Ferkans — Interactive Telecom Tutor

Why the Gaussian Appears Everywhere

The LLN tells us that $\bar{X}_n \approx \mu$ for large $n$ . The CLT answers the follow-up question: how is $\bar{X}_n$ distributed around $\mu$ ? The answer — Gaussian, regardless of the underlying distribution — is one of the most remarkable facts in all of mathematics.

For telecommunications, this has a profound operational consequence. Thermal noise is the sum of tiny random contributions from many electrons. Aggregate interference in a dense network is the sum of many weak signals. Channel estimation errors accumulate over many pilot symbols. In all these cases, the CLT explains why Gaussian models work so well: the sum of many small independent effects is approximately Gaussian, no matter what the individual effects look like.

Theorem: Central Limit Theorem (CLT)

Let $X_1, X_2, \ldots$ be i.i.d. with mean $\mu$ , variance $\sigma^2 \in (0, \infty)$ . Define the standardized sum:

$Z_n = \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} = \frac{\sum_{i=1}^n X_i - n\mu}{\sigma\sqrt{n}}.$

Then:

$Z_n \xrightarrow{d} \mathcal{N}(0,1),$

that is, $\lim_{n \to \infty} \mathbb{P}(Z_n \leq z) = \Phi(z)$ for all $z \in \mathbb{R}$ , where $\Phi$ is the standard normal CDF.

The LLN says $\bar{X}_n \approx \mu$ with fluctuations of order $\sigma/\sqrt{n}$ . The CLT says the shape of those fluctuations is Gaussian, regardless of the shape of the original distribution. The characteristic function proof reveals why: the Ch.F of the standardized sum converges to $e^{-u^2/2}$ because all higher-order cumulants vanish after division by $\sqrt{n}$ .

Proof

Standardize

Let $Y_i = (X_i - \mu)/\sigma$ , so $\mathbb{E}[Y_i] = 0$ , $\text{Var}(Y_i) = 1$ . Then $Z_n = \frac{1}{\sqrt{n}}\sum_{i=1}^n Y_i$ .

Compute the Ch.F of $Z_n$

Let ${\phi_X}_{Y}$ be the common Ch.F of the $Y_i$ . Since they are i.i.d. and $Z_n = \frac{1}{\sqrt{n}}\sum Y_i$ :

${\phi_X}_{Z_n}(u) = \left({\phi_X}_{Y}\!\left(\frac{u}{\sqrt{n}}\right)\right)^n.$

Taylor-expand the Ch.F

Since $\mathbb{E}[Y] = 0$ and $\mathbb{E}[Y^2] = 1$ , the Taylor expansion (Theorem 10 from Ch. 10) gives:

${\phi_X}_{Y}(v) = 1 + jv \cdot 0 + \frac{(jv)^2}{2} \cdot 1 + o(v^2) = 1 - \frac{v^2}{2} + o(v^2).$

Substituting $v = u/\sqrt{n}$ :

${\phi_X}_{Y}\!\left(\frac{u}{\sqrt{n}}\right) = 1 - \frac{u^2}{2n} + o\!\left(\frac{1}{n}\right).$

Take the $n$-th power

Using the limit $(1 + a/n + o(1/n))^n \to e^a$ :

${\phi_X}_{Z_n}(u) = \left(1 - \frac{u^2}{2n} + o\!\left(\frac{1}{n}\right)\right)^n \to e^{-u^2/2}.$

Identify the limit and conclude

The function $e^{-u^2/2}$ is the Ch.F of $\mathcal{N}(0,1)$ . It is continuous at $u = 0$ . By the Levy continuity theorem:

$Z_n \xrightarrow{d} \mathcal{N}(0,1). \quad \blacksquare$

,

The Operational Content of the CLT

The CLT gives us a practical approximation: for large $n$ ,

$\bar{X}_n \approx \mathcal{N}\!\left(\mu, \frac{\sigma^2}{n}\right).$

This means:

$\mathbb{P}(\bar{X}_n > \mu + \delta) \approx Q(\delta\sqrt{n}/\sigma)$
The 95% confidence interval for $\mu$ is approximately $\bar{X}_n \pm 1.96\,\sigma/\sqrt{n}$
The "error" is of order $\sigma/\sqrt{n}$ , the universal convergence rate for i.i.d. averaging

The question "how large must $n$ be for this approximation to be good?" is answered by the Berry-Esseen theorem below.

Theorem: Berry-Esseen Theorem

Let $X_1, X_2, \ldots$ be i.i.d. with mean $\mu$ , variance $\sigma^2 > 0$ , and finite third absolute moment $\rho = \mathbb{E}[|X_1 - \mu|^3] < \infty$ . Then for all $n \geq 1$ and all $z \in \mathbb{R}$ :

$\left|\mathbb{P}\!\left(\frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \leq z\right) - \Phi(z)\right| \leq \frac{C\rho}{\sigma^3 \sqrt{n}},$

where $C$ is a universal constant. The best known value is $C \leq 0.4748$ (Shevtsova, 2011).

The CLT says the CDF converges to the Gaussian CDF, but how fast? Berry-Esseen says the convergence rate is $O(1/\sqrt{n})$ , uniformly over all $z$ . The constant depends on $\rho/\sigma^3$ , which measures how "non-Gaussian" the original distribution is. For symmetric distributions, the bound is tighter.

, ,

CLT Convergence: Histogram of $Z_n$ Approaching the Gaussian

For i.i.d. samples from a chosen distribution, observe how the histogram of the standardized sum $Z_n$ approaches the standard normal bell curve as $n$ grows.

Parameters

Distribution

n

(number of summands)5

Berry-Esseen Rate: $\sup_z |F_n(z) - \Phi(z)|$ vs. $n$

Observe how the maximum CDF error between the standardized sum and the Gaussian decreases as $O(1/\sqrt{n})$ , matching the Berry-Esseen prediction.

Parameters

Distribution

Distribution parameter0.5

n_{\max}

200

Example: CLT for Coin Flips: The de Moivre-Laplace Theorem

Let $X_1, \ldots, X_n$ be i.i.d. $\text{Bernoulli}(1/2)$ . Use the CLT to approximate $\mathbb{P}(S_n \geq 60)$ where $S_n = \sum_{i=1}^{100} X_i$ (number of heads in 100 fair coin flips).

Solution

Compute mean and variance

$\mu = 1/2$ , $\sigma^2 = 1/4$ , so $S_n$ has mean $n\mu = 50$ and standard deviation $\sigma\sqrt{n} = 5$ .

Standardize

$\mathbb{P}(S_{100} \geq 60) = \mathbb{P}\!\left(\frac{S_{100} - 50}{5} \geq \frac{60 - 50}{5}\right) = \mathbb{P}(Z_{100} \geq 2).$ $

Apply the CLT approximation

$\mathbb{P}(Z_{100} \geq 2) \approx Q(2) = 1 - \Phi(2) \approx 0.0228.$ $The exact value (by summing the binomial PMF) is$ 0.0284 $, so the CLT approximation is quite accurate even for$ n = 100$ with a discrete distribution.

Example: CLT for Waiting Times

A call center receives calls with i.i.d. exponential inter-arrival times with rate $\lambda = 2$ calls/minute. Approximate the probability that the total time for 100 calls exceeds 55 minutes.

Solution

Identify parameters

Each inter-arrival time $X_i \sim \text{Exp}(2)$ with $\mu = 1/\lambda = 0.5$ minutes and $\sigma^2 = 1/\lambda^2 = 0.25$ .

Apply CLT

$S_{100} = \sum_{i=1}^{100} X_i$ has mean $100 \times 0.5 = 50$ and standard deviation $\sqrt{100 \times 0.25} = 5$ . Then:

$\mathbb{P}(S_{100} > 55) = \mathbb{P}\!\left(\frac{S_{100} - 50}{5} > 1\right) \approx Q(1) \approx 0.1587.$

Common Mistake: The CLT Is an Asymptotic Statement — Small $n$ Requires Caution

Mistake:

Applying the CLT with $n = 5$ or $n = 10$ and trusting the Gaussian approximation for tail probabilities.

Correction:

The CLT guarantees convergence as $n \to \infty$ , but the rate depends on the underlying distribution. For heavy-tailed distributions (large $\rho/\sigma^3$ ), the Berry-Esseen bound shows convergence can be slow. In particular:

Bernoulli with $p = 0.5$ : excellent by $n = 30$
Exponential: reasonable by $n = 50$
Chi-squared with 1 d.f. (very skewed): may need $n > 100$

For tail probabilities ( $z > 2$ ), the approximation degrades faster than at the center.

Historical Note: The Central Limit Theorem: 200 Years of Refinement

1733–1942

Abraham de Moivre (1733) proved the CLT for fair coin flips. Pierre-Simon Laplace (1812) extended it to general distributions, though his proof lacked rigor by modern standards. The first rigorous proof using characteristic functions was given by Aleksandr Lyapunov (1901). Lindeberg (1922) and Feller (1935) established the definitive necessary and sufficient conditions for the CLT to hold for independent (not necessarily identically distributed) summands. The Berry-Esseen theorem (1941-42) finally quantified the rate of convergence.

The name "central" was coined by George Polya in 1920, reflecting its central importance in probability theory — not any geometric meaning.

🔧Engineering Note

Why Gaussian Noise Models Work in Communications

In a communication receiver, the thermal noise at the antenna is the aggregate effect of random electron motion across billions of charge carriers. Each contributes a tiny random voltage, and the CLT guarantees that their sum is approximately Gaussian. This justifies the $\mathcal{N}(0, \sigma^2)$ noise model used throughout signal processing and information theory.

More precisely, the noise in a bandwidth $W$ over a time interval $T$ is the sum of roughly $2WT$ independent noise "samples" (by the sampling theorem). For typical values ( $W = 10$ MHz, $T = 1$ ms), this is $20{,}000$ independent contributions — more than enough for the CLT to provide an excellent approximation.

Practical Constraints

•
The Gaussian model breaks down for impulsive noise (e.g., lightning, power line interference)
•
Non-Gaussian interference arises in ultra-dense networks where a few strong interferers dominate

Quick Check

The CLT says that $Z_n = \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \xrightarrow{d} \mathcal{N}(0,1)$ . What convergence mode is this?

Almost sure convergence

Convergence in probability

Convergence in distribution

$L^2$ convergence

Correction:

Convergence in distribution

Correct. The CLT is a statement about convergence of CDFs: $\mathbb{P}(Z_n \leq z) \to \Phi(z)$ .

Central Limit Theorem

The standardized sum of i.i.d. random variables with finite variance converges in distribution to $\mathcal{N}(0,1)$ . The convergence rate is $O(1/\sqrt{n})$ by Berry-Esseen.

Berry-Esseen Theorem

Quantifies the CLT convergence rate: $\sup_z |F_{Z_n}(z) - \Phi(z)| \leq C\rho/(\sigma^3\sqrt{n})$ where $\rho = \mathbb{E}[|X - \mu|^3]$ and $C \leq 0.4748$ .

Related: Central Limit Theorem

Key Takeaway

The CLT is the reason Gaussian models dominate communications engineering. Whenever a quantity is the sum of many small independent contributions — noise, interference, estimation errors — it is approximately Gaussian, regardless of the individual distributions. The Berry-Esseen theorem tells us the approximation error is $O(1/\sqrt{n})$ .

The Central Limit Theorem

Why the Gaussian Appears Everywhere

Theorem: Central Limit Theorem (CLT)

Standardize

Compute the Ch.F of $Z_n$

Taylor-expand the Ch.F

Take the $n$-th power

Identify the limit and conclude

The Operational Content of the CLT

Theorem: Berry-Esseen Theorem

CLT Convergence: Histogram of ZnZ_nZn​ Approaching the Gaussian

Parameters

Berry-Esseen Rate: sup⁡z∣Fn(z)−Φ(z)∣\sup_z |F_n(z) - \Phi(z)|supz​∣Fn​(z)−Φ(z)∣ vs. nnn

Parameters

Example: CLT for Coin Flips: The de Moivre-Laplace Theorem

Compute mean and variance

Standardize

Apply the CLT approximation

Example: CLT for Waiting Times

Identify parameters

Apply CLT

Common Mistake: The CLT Is an Asymptotic Statement — Small nnn Requires Caution

Historical Note: The Central Limit Theorem: 200 Years of Refinement

Why Gaussian Noise Models Work in Communications

Quick Check

Central Limit Theorem

Berry-Esseen Theorem

Key Takeaway

CLT Convergence: Histogram of $Z_n$ Approaching the Gaussian

Berry-Esseen Rate: $\sup_z |F_n(z) - \Phi(z)|$ vs. $n$

Common Mistake: The CLT Is an Asymptotic Statement — Small $n$ Requires Caution