Chapter Summary

Chapter Summary

Key Points

• 1.

  The AWGN channel $Y = X + Z$, $Z \sim \mathcal{N}(0, N)$, with power constraint $P$ has capacity $C = \frac{1}{2}\log(1 + P/N)$ bits per real channel use. The Gaussian input is uniquely optimal because it maximizes the output entropy under a variance constraint.

• 2.

  The Shannon limit $E_b/N_0 \geq \ln 2 \approx -1.59$ dB is the ultimate energy efficiency bound, achieved in the wideband regime ($W \to \infty$). No coding scheme can beat this limit; modern turbo and LDPC codes approach it within 0.5 dB.

• 3.

  For $K$ parallel Gaussian sub-channels with different gains, the capacity-achieving power allocation is water-filling: ${E_s}_{k}^* = [\nu - N_0/|G_k|^2]_+$. This is the unique global optimum of a convex optimization, found via KKT conditions.

• 4.

  Colored noise is handled by diagonalizing the noise covariance (Karhunen-Lo`eve expansion), which converts the channel to parallel sub-channels with unequal noise variances. Water-filling over the noise spectrum then gives the optimal power allocation.

• 5.

  The Nyquist-Shannon sampling theorem converts the continuous-time band-limited AWGN channel into a discrete-time model, yielding the Shannon-Hartley formula: $C = W\log_2(1 + P/(N_0W))$ bits/s. This reveals two operating regimes: bandwidth-limited (high SNR) and power-limited (low SNR).

• 6.

  OFDM converts a frequency-selective ISI channel into parallel Gaussian sub-channels via the cyclic prefix and DFT, making water-filling directly applicable. This is the information-theoretic foundation of 4G LTE and 5G NR.

• 7.

  The sphere-packing interpretation provides geometric intuition: capacity equals the logarithmic ratio of the volume of the received sphere to the noise sphere, counting how many distinguishable codewords can be packed.