Ferkans — Interactive Telecom Tutor

Why Transform Methods?

We have spent several chapters learning to compute with distributions directly — PDFs, CDFs, convolutions. This works well for one or two random variables, but the moment you need the distribution of a sum $S_n = X_1 + \cdots + X_n$ of independent random variables, you face an $n$ -fold convolution that becomes unwieldy even for $n = 3$ .

Transform methods offer an elegant alternative: encode the distribution into a single function, and the convolution becomes a product. This chapter develops three transforms — the MGF, the characteristic function, and the PGF — each suited to different contexts. Together, they form the analytical backbone that powers the great limit theorems: the law of large numbers and the central limit theorem.

Definition:
Moment Generating Function (MGF)

Let $X$ be a random variable with CDF $F(x)$ . The moment generating function (MGF) of $X$ is

$M_X(t) = \mathbb{E}[e^{tX}] = \int_{-\infty}^{\infty} e^{tx}\,dF(x), \qquad t \in \mathbb{R}.$

The MGF may be finite only on a subset of $\mathbb{R}$ . We say that the MGF exists if there is an open interval $(-a, a)$ with $a > 0$ such that $M_X(t) < \infty$ for all $|t| < a$ .

The Riemann-Stieltjes formulation $\int e^{tx}\,dF(x)$ unifies the discrete case ( $\sum_x e^{tx} p_X(x)$ ) and the continuous case ( $\int e^{tx} f(x)\,dx$ ).

Moment Generating Function (MGF)

The function $M_X(t) = \mathbb{E}[e^{tX}]$ , which encodes all moments of $X$ through its Taylor coefficients: $\mathbb{E}[X^k] = M_X^{(k)}(0)$ .

MGF as Bilateral Laplace Transform

For a continuous random variable with PDF $f(x)$ , the MGF is the bilateral Laplace transform of the density evaluated at $s = -t$ :

$M_X(t) = \mathcal{L}[f](s)\big|_{s = -t}, \qquad \mathcal{L}[f](s) = \int_{-\infty}^{\infty} e^{-sx}f(x)\,dx.$

This connection to the Laplace transform is why the MGF inherits all the algebraic machinery of transform calculus — in particular, the conversion of convolution to multiplication.

Theorem: Moments from Derivatives of the MGF

If $M_X(t) < \infty$ for $|t| < a$ with $a > 0$ , then all moments of $X$ exist and

$\mathbb{E}[X^k] = \frac{d^k}{dt^k}M_X(t)\bigg|_{t=0} = M_X^{(k)}(0), \qquad k = 1, 2, 3, \ldots$

Moreover, $M_X(t)$ admits the Taylor expansion

$M_X(t) = \sum_{k=0}^{\infty} \frac{\mathbb{E}[X^k]}{k!}\,t^k$

for $|t| < a$ .

Differentiate under the integral: $\frac{d}{dt}\mathbb{E}[e^{tX}] = \mathbb{E}[X e^{tX}]$ , and evaluate at $t = 0$ to extract moments one at a time.

Proof

Interchange differentiation and expectation

Since $M_X(t) < \infty$ in a neighborhood of $0$ , the dominated convergence theorem justifies:

$\frac{d^k}{dt^k}M_X(t) = \frac{d^k}{dt^k}\int e^{tx}\,dF(x) = \int x^k e^{tx}\,dF(x).$

Evaluate at $t = 0$

Setting $t = 0$ :

$M_X^{(k)}(0) = \int x^k\,dF(x) = \mathbb{E}[X^k].$

Taylor expansion

Since $M_X(t)$ is analytic in $|t| < a$ , it equals its Taylor series. The $k$ -th coefficient is $M_X^{(k)}(0)/k! = \mathbb{E}[X^k]/k!$ .

Theorem: MGF of a Sum of Independent Random Variables

If $X$ and $Y$ are independent random variables whose MGFs exist, then

$M_{X+Y}(t) = M_X(t) \cdot M_Y(t).$

More generally, if $X_1, \ldots, X_n$ are independent, then

$M_{S_n}(t) = \prod_{i=1}^n M_{X_i}(t), \qquad S_n = \sum_{i=1}^n X_i.$

The exponential converts a sum into a product: $e^{t(X+Y)} = e^{tX} \cdot e^{tY}$ . Independence then factors the expectation: $\mathbb{E}[e^{tX} e^{tY}] = \mathbb{E}[e^{tX}]\,\mathbb{E}[e^{tY}]$ .

Proof

Factor the exponential

$M_{X+Y}(t) = \mathbb{E}[e^{t(X+Y)}] = \mathbb{E}[e^{tX} e^{tY}]$ .

Apply independence

Since $X \perp Y$ , and $g(X) = e^{tX}$ , $h(Y) = e^{tY}$ are measurable functions of independent RVs:

$\mathbb{E}[e^{tX} e^{tY}] = \mathbb{E}[e^{tX}]\,\mathbb{E}[e^{tY}] = M_X(t) \cdot M_Y(t).$

Key Takeaway

The MGF converts the convolution of densities into a product of functions. This is the single most important algebraic property of transforms: it reduces the problem of finding the distribution of a sum to multiplying known functions and inverting.

Example: MGF of the Gaussian Distribution

Let $X \sim \mathcal{N}(\mu, \sigma^2)$ . Compute $M_X(t)$ .

Solution

Write the expectation

$M_X(t) = \int_{-\infty}^{\infty} e^{tx} \frac{1}{\sqrt{2\pi\sigma^2}}\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)dx.$ $

Complete the square

The exponent is $tx - \frac{(x-\mu)^2}{2\sigma^2} = -\frac{1}{2\sigma^2}\bigl(x - (\mu + t\sigma^2)\bigr)^2 + \mu t + \frac{t^2\sigma^2}{2}$ .

Evaluate the Gaussian integral

The integral over $x$ of the completed-square Gaussian density is $1$ , leaving:

$M_X(t) = \exp\!\left(\mu t + \frac{\sigma^2 t^2}{2}\right).$

Verify moments

$M_X'(0) = \mu = \mathbb{E}[X]$ . Also $M_X''(0) = \mu^2 + \sigma^2 = \mathbb{E}[X^2]$ , so $\text{Var}(X) = \sigma^2$ . Correct.

Example: MGF of the Exponential Distribution

Let $X \sim \text{Exp}(\lambda)$ with PDF $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$ . Find $M_X(t)$ and identify its domain.

Solution

Compute the integral

$M_X(t) = \int_0^{\infty} e^{tx} \lambda e^{-\lambda x}\,dx = \lambda \int_0^{\infty} e^{-({\lambda - t})x}\,dx = \frac{\lambda}{\lambda - t}, \qquad t < \lambda.$ $

Extract moments

$M_X'(t) = \frac{\lambda}{(\lambda - t)^2}$ , so $\mathbb{E}[X] = M_X'(0) = 1/\lambda$ . Similarly, $\mathbb{E}[X^2] = 2/\lambda^2$ , giving $\text{Var}(X) = 1/\lambda^2$ .

MGFs of Common Distributions

Distribution	Parameters	$M_X(t)$	Domain
$\text{Bernoulli}(p)$	$p \in (0,1)$	$1 - p + pe^t$	$\mathbb{R}$
$\text{Bin}(n, p)$	$n \in \mathbb{N},\; p \in (0,1)$	$(1 - p + pe^t)^n$	$\mathbb{R}$
$\text{Poi}(\lambda)$	$\lambda > 0$	$e^{\lambda(e^t - 1)}$	$\mathbb{R}$
$\text{Exp}(\lambda)$	$\lambda > 0$	$\frac{\lambda}{\lambda - t}$	$t < \lambda$
$\text{Gamma}(\alpha, \beta)$	$\alpha, \beta > 0$	$\left(\frac{\beta}{\beta - t}\right)^\alpha$	$t < \beta$
$\mathcal{N}(\mu, \sigma^2)$	$\mu \in \mathbb{R},\; \sigma^2 > 0$	$e^{\mu t + \sigma^2 t^2/2}$	$\mathbb{R}$

Common Mistake: The MGF Does Not Always Exist

Mistake:

Assuming that $M_X(t)$ is finite for all $t$ — or even for any $t \neq 0$ — without checking. For example, the Cauchy distribution has $M_X(t) = \infty$ for all $t \neq 0$ .

Correction:

Always verify the domain of finiteness before using the MGF. Heavy-tailed distributions (Cauchy, Pareto with small shape parameter, log-normal) have MGFs that diverge everywhere except at $t = 0$ . For such distributions, use the characteristic function instead — it always exists.

MGF Explorer for Common Distributions

Select a distribution and adjust its parameters to see how the MGF $M_X(t)$ changes shape. Notice how the slope at $t = 0$ equals the mean and the curvature relates to the variance.

Parameters

Distribution

Parameter 1 (mean or rate)1

Parameter 2 (variance or shape)1

Historical Note: Laplace and the Birth of Transform Methods

18th-19th century

The idea of encoding a function through an integral transform dates to Pierre-Simon Laplace (1749--1827), who used what we now call the Laplace transform to solve differential equations. The connection to probability was recognized early: the MGF $M_X(t) = \mathbb{E}[e^{tX}]$ is precisely the Laplace transform of the density evaluated on the negative real axis. Laplace himself used generating functions (the discrete analogue) to study random walks and gambler's ruin problems in his Theorie analytique des probabilites (1812).

Laplace Transform

The integral transform $\mathcal{L}[f](s) = \int_0^{\infty} e^{-sx} f(x)\,dx$ . For probability, the bilateral version $\int_{-\infty}^{\infty} e^{-sx} f(x)\,dx$ connects to the MGF via $M_X(t) = \mathcal{L}[f](-t)$ .

Quick Check

If $X \sim \text{Poi}(\lambda)$ and $Y \sim \text{Poi}(\mu)$ are independent, what is the MGF of $Z = X + Y$ ?

$e^{(\lambda + \mu)(e^t - 1)}$

$e^{\lambda\mu(e^t - 1)}$

$e^{(\lambda + \mu)t}$

$\frac{\lambda\mu}{(\lambda-t)(\mu-t)}$

Correction:

e^{(\lambda + \mu)(e^t - 1)}

By the product property: $M_Z(t) = M_X(t) M_Y(t) = e^{\lambda(e^t-1)} e^{\mu(e^t-1)} = e^{(\lambda+\mu)(e^t-1)}$ . This is the MGF of $\text{Poi}(\lambda + \mu)$ .

🔧Engineering Note

MGF Approach to BER Analysis over Fading Channels

In digital communications over fading channels, the bit error rate (BER) conditioned on the instantaneous SNR $\gamma$ is typically $P_e(\gamma) = Q(\sqrt{2\gamma})$ . The average BER requires integrating $P_e(\gamma)$ over the fading distribution. Using the alternative form $Q(x) = \frac{1}{\pi}\int_0^{\pi/2} \exp\!\bigl(-\frac{x^2}{2\sin^2\theta}\bigr)d\theta$ (Craig's representation), the inner integral becomes the MGF of $\gamma$ evaluated at $-1/\sin^2\theta$ . This converts the BER averaging problem into an MGF evaluation — a technique used extensively in wireless communications.

Practical Constraints

•
Requires that the MGF of the fading distribution exists
•
Craig's representation applies to the Q-function specifically

Why This Matters: MGF and Fading Channel Analysis

The MGF of the SNR distribution plays a central role in wireless communications. For Rayleigh fading ( $\gamma \sim \text{Exp}(1/\bar{\gamma})$ ), the MGF is $M_\gamma(t) = (1 - \bar{\gamma}t)^{-1}$ , which directly yields closed-form BER expressions for most modulation schemes. The MGF approach extends naturally to diversity combining (MRC, EGC) and MIMO systems where the SNR is a sum or function of channel gains.

The Moment Generating Function

Why Transform Methods?

Definition: Moment Generating Function (MGF)

Moment Generating Function (MGF)

MGF as Bilateral Laplace Transform

Theorem: Moments from Derivatives of the MGF

Interchange differentiation and expectation

Evaluate at $t = 0$

Taylor expansion

Theorem: MGF of a Sum of Independent Random Variables

Factor the exponential

Apply independence

Key Takeaway

Example: MGF of the Gaussian Distribution

Write the expectation

Complete the square

Evaluate the Gaussian integral

Verify moments

Example: MGF of the Exponential Distribution

Compute the integral

Extract moments

MGFs of Common Distributions

Common Mistake: The MGF Does Not Always Exist

MGF Explorer for Common Distributions

Parameters

Historical Note: Laplace and the Birth of Transform Methods

Laplace Transform

Quick Check

MGF Approach to BER Analysis over Fading Channels

Why This Matters: MGF and Fading Channel Analysis

Definition:
Moment Generating Function (MGF)