Prerequisites & Notation

Before You Begin

This chapter applies the continuous-alphabet channel coding theorem (Chapter 2) to the most important channel model in communications: the additive white Gaussian noise (AWGN) channel. We rely heavily on differential entropy, the Gaussian entropy maximizer theorem, and the connection between MMSE and differential entropy. Convex optimization (Lagrange multipliers, KKT conditions) appears prominently in the water-filling solution.

Differential entropy $h(X)$ and its properties (Chapter 2)(Review ch02)
Self-check: Can you compute $h(X)$ for $X \sim \mathcal{N}(\mu, \sigma^2)$ ?
Gaussian maximizes differential entropy under a covariance constraint (Chapter 2)(Review ch02)
Self-check: Can you state and prove why Gaussian input maximizes $h(Y)$ for a given variance?
Mutual information for continuous random variables: $I(X;Y) = h(Y) - h(Y|X)$ (Review ch02)
Self-check: Can you express $I(X;Y)$ in terms of differential entropies?
Channel coding theorem for continuous-alphabet channels (Chapter 9)(Review ch09)
Self-check: Can you state the capacity formula $C = \max_{p_X} I(X;Y)$ and what achievability and converse mean?
Lagrange multipliers and KKT conditions for constrained optimization
Self-check: Can you solve $\max f(x)$ subject to $g(x) \leq 0$ using the KKT conditions?
Basic Fourier analysis: DFT, DTFT, circulant matrices
Self-check: Do you know that circulant matrices are diagonalized by the DFT matrix?

Notation for This Chapter

Symbols introduced in this chapter. All logarithms are base 2 (bits) unless stated otherwise. We use both real and complex channel models; the context will always be clear.

Symbol	Meaning	Introduced
$Y = X + Z$	Scalar AWGN channel model	s01
$Z \sim \mathcal{N}(0, N)$	Additive Gaussian noise with variance $N$	s01
$P$	Average transmit power constraint: $\frac{1}{n}\sum_i x_i^2 \leq P$	s01
$\text{SNR} = P/N$	Signal-to-noise ratio	s01
$C(\text{SNR})$	AWGN channel capacity as a function of SNR	s01
$E_b/N_0$	Energy per bit to noise spectral density ratio	s02
$W$	Channel bandwidth in Hz	s02
$G_k$	Channel gain on sub-channel $k$ (parallel Gaussian model)	s03
$\nu$	Water-filling level (Lagrange multiplier)	s03
$[x]_+ = \max(x, 0)$	Positive part operator	s03
$N_0$	One-sided noise power spectral density	s05
$G(\xi)$	DTFT of the channel impulse response	s04

← Ch 9 The AWGN Channel