Ferkans — Interactive Telecom Tutor

Convolution: The Hard Way and the Easy Way

Finding the distribution of $Z = X + Y$ for independent continuous RVs requires the convolution integral: $f_{Z}(z) = \int_{-\infty}^{\infty}f_{X}(x)\,f_{Y}(z-x)\,dx$ . For a sum of $n$ independent RVs, this becomes an $(n-1)$ -fold iterated integral — increasingly unmanageable.

Transforms convert convolution to multiplication: ${\phi_X}_{Z} = {\phi_X}_{X} \cdot {\phi_X}_{Y}$ . Multiply, then invert. This is the fundamental strategy of this section.

Theorem: The Convolution Theorem

Let $X$ and $Y$ be independent random variables. The PDF of $Z = X + Y$ is the convolution $f_{Z} = f_{X} * f_{Y}$ , and the CF satisfies

$\phi_Z(u) = \phi_X(u) \cdot \phi_Y(u).$

Conversely, if $\phi_Z(u) = \phi_X(u)\phi_Y(u)$ and $\phi_X(u) \neq 0$ in a neighborhood of $0$ , then $\phi_Y(u) = \phi_Z(u)/\phi_X(u)$ — deconvolution in the transform domain.

Convolution in the "time" (probability) domain corresponds to multiplication in the "frequency" (transform) domain. This is the same principle that makes Fourier analysis powerful in signal processing.

Proof

Write the expectation

$\phi_Z(u) = \mathbb{E}[e^{ju(X+Y)}] = \mathbb{E}[e^{juX}e^{juY}]$ .

Factor by independence

$= \mathbb{E}[e^{juX}]\,\mathbb{E}[e^{juY}] = \phi_X(u)\,\phi_Y(u)$ .

Example: Sum of Independent Gaussian Random Variables

Let $X_i \sim \mathcal{N}(\mu_i, \sigma_i^2)$ be independent for $i = 1, \ldots, n$ . Find the distribution of $S_n = \sum_{i=1}^n X_i$ .

Solution

Multiply the CFs

$\phi_{S_n}(u) = \prod_{i=1}^n \exp\!\left(j\mu_i u - \frac{\sigma_i^2 u^2}{2}\right) = \exp\!\left(j\left(\sum\mu_i\right)u - \frac{\left(\sum\sigma_i^2\right)u^2}{2}\right).$

Identify by uniqueness

This is the CF of $\mathcal{N}\!\left(\sum_{i=1}^n\mu_i, \sum_{i=1}^n\sigma_i^2\right)$ .

The Gaussian family is closed under convolution: sums of independent Gaussians are Gaussian. The mean and variance simply add.

Key Takeaway

The Gaussian distribution is stable under addition: a sum of independent Gaussians is Gaussian. This is the reason the Gaussian plays such a central role — it is the only distribution (up to trivial scaling) that is a fixed point of the operation "sum, then rescale," which is exactly what the CLT describes.

Example: Sum of Independent Exponentials is Gamma

Let $X_1, \ldots, X_n$ be i.i.d. $\text{Exp}(\lambda)$ . Find the distribution of $S_n = \sum_{i=1}^n X_i$ .

Solution

Multiply MGFs

$M_{S_n}(t) = \left(\frac{\lambda}{\lambda - t}\right)^n, \qquad t < \lambda$ .

Identify

This is the MGF of $\text{Gamma}(n, \lambda)$ , confirming that $S_n \sim \text{Gamma}(n, \lambda)$ . The sum of $n$ i.i.d. exponentials with rate $\lambda$ is a Gamma with shape $n$ and rate $\lambda$ .

For $n$ integer, this is also the Erlang distribution, which models the waiting time until the $n$ -th arrival in a Poisson process with rate $\lambda$ .

Convolution via Characteristic Functions

This plot shows two independent distributions $X$ and $Y$ (top), their characteristic functions (middle), and the distribution of $Z = X + Y$ obtained by multiplying the CFs and inverting (bottom). Adjust the distributions to see how the transform domain simplifies convolution.

Parameters

Distribution of X

Distribution of Y

Scale of X1

Scale of Y1

🔧Engineering Note

Thermal Noise as a Sum of Independent Contributions

In electronic systems, thermal noise arises from the random motion of electrons in resistive elements. The total noise voltage is a sum of contributions from many independent sources — different circuit components, amplifier stages, and antenna elements. By the CLT (Section §The Law of Large Numbers and Central Limit Theorem), this sum is approximately Gaussian regardless of the distribution of individual contributions. The transform approach makes this rigorous: the CF of the sum converges to a Gaussian CF as the number of terms grows.

This is the physical justification for the AWGN (additive white Gaussian noise) model that pervades communications theory.

Practical Constraints

•
Requires many independent contributions of comparable magnitude
•
Fails when one dominant interferer is present (impulsive noise)

Common Mistake: Convolution Requires Independence

Mistake:

Writing $\phi_{X+Y}(u) = \phi_X(u)\phi_Y(u)$ without verifying that $X$ and $Y$ are independent. If $X$ and $Y$ are dependent, the CF of the sum is $\phi_{X+Y}(u) = \mathbb{E}[e^{ju(X+Y)}]$ , which does not factor.

Correction:

The product rule applies only when $X \perp Y$ . For dependent random variables, you must work with the joint CF $\phi_{X,Y}(u, v)$ and set $v = u$ to get $\phi_{X+Y}(u)$ .

Sums of Independent Random Variables and Convolution