Chapter Summary

Chapter Summary

Key Points

• 1.

  There are four modes of convergence for random variables: almost sure, in probability, in $L^r$, and in distribution. The implications are a.s. $\Rightarrow$ in probability $\Rightarrow$ in distribution, and $L^r \Rightarrow$ in probability. No other general implications hold.

• 2.

  The Weak Law of Large Numbers (WLLN) states that $\bar{X}_n \xrightarrow{P} \mu$ for i.i.d. sequences with finite variance. The proof is a direct application of Chebyshev's inequality: $\mathbb{P}(|\bar{X}_n - \mu| \geq \epsilon) \leq \sigma^2/(n\epsilon^2)$.

• 3.

  The Strong Law of Large Numbers (SLLN) upgrades to almost sure convergence: $\bar{X}_n \xrightarrow{\text{a.s.}} \mu$ under only finite mean. The Borel-Cantelli proof for finite fourth moments shows $\sum_n \mathbb{P}(|\bar{X}_n - \mu| \geq \epsilon) < \infty$.

• 4.

  The Central Limit Theorem (CLT) characterizes the fluctuations around $\mu$: $(\bar{X}_n - \mu)/(\sigma/\sqrt{n}) \xrightarrow{d} \mathcal{N}(0,1)$. The proof via characteristic functions exploits the Taylor expansion $\phi_Y(u) = 1 - u^2/2 + o(u^2)$ and the limit $(1 + a/n)^n \to e^a$.

• 5.

  The Berry-Esseen theorem quantifies the CLT convergence rate at $O(1/\sqrt{n})$, depending on the third absolute moment $\rho/\sigma^3$ of the underlying distribution.

• 6.

  The multivariate CLT, delta method, and Slutsky's theorem extend the CLT to vector averages and smooth functions thereof — the foundation of asymptotic statistics and performance analysis in communications.