Ferkans — Interactive Telecom Tutor

From One Channel to Many

Real wideband systems rarely face a single flat channel. An OFDM system sees $K$ parallel sub-channels with different gains; a MIMO system decomposes (via SVD) into parallel spatial streams. The fundamental question becomes: given a total power budget, how should we allocate power across sub-channels of unequal quality?

The answer — water-filling — is one of the most beautiful results in information theory, and it emerges naturally from the KKT conditions of a convex optimization. The same structure reappears in MIMO capacity (Chapter 13), OFDM power loading (Book telecom, Ch. 14), and even rate-distortion theory (Chapter 6) in its "reverse water-filling" form.

Definition:
Parallel Gaussian Channel

The parallel Gaussian channel consists of $K$ independent sub-channels:

$Y_k = G_k X_k + Z_{k}, \quad k = 1, \ldots, K,$

where $G_k \in \mathbb{C}$ is the (known) gain of sub-channel $k$ and $Z_{k} \sim \mathcal{CN}(0, N_0)$ are independent noise samples.

The total power is constrained: $\sum_{k=1}^K \mathbb{E}[|X_k|^2] \leq E_s$ , where ${E_s}_{k} = \mathbb{E}[|X_k|^2]$ is the power allocated to sub-channel $k$ .

Parallel Gaussian channel

A set of $K$ independent AWGN sub-channels with different gains $G_1, \ldots, G_K$ sharing a total power budget. Arises naturally in OFDM (frequency sub-carriers), MIMO (spatial streams after SVD), and DSL (tones).

Related: Water-filling, AWGN channel

Theorem: Capacity of the Parallel Gaussian Channel

The capacity of the parallel Gaussian channel is

$C(E_s) = \max_{\substack{{E_s}_{1}, \ldots, {E_s}_{K} \geq 0 \\ \sum_k {E_s}_{k} \leq E_s}} \sum_{k=1}^K \log\!\left(1 + \frac{|G_k|^2 {E_s}_{k}}{N_0}\right).$

The optimal power allocation is given by the water-filling solution:

${E_s}_{k}^* = \left[\nu - \frac{N_0}{|G_k|^2}\right]_+,$

where $\nu$ is chosen so that $\sum_k {E_s}_{k}^* = E_s$ . The resulting capacity is

$C(E_s) = \sum_{k=1}^K \left[\log\!\left(\frac{\nu\, |G_k|^2}{N_0}\right)\right]_+.$

Water-filling allocates more power to stronger sub-channels and less (or zero) to weak ones. The geometric picture is vivid: invert the channel gains to form a "bowl" with depth $N_0/|G_k|^2$ , then "pour water" up to a common level $\nu$ . The water depth at each position is the allocated power. Deep nulls (weak sub-channels) receive no water — it is better to shut them off entirely.

Proof

Independence gives additive capacity

Since the sub-channels are independent, the mutual information decomposes:

$I(\mathbf{X}; \mathbf{Y}) = \sum_{k=1}^K I(X_k; Y_k) = \sum_{k=1}^K \log\!\left(1 + \frac{|G_k|^2 {E_s}_{k}}{N_0}\right),$

where the last equality uses the fact that the capacity-achieving input for each sub-channel is Gaussian: $X_k \sim \mathcal{CN}(0, {E_s}_{k})$ .

Formulate as convex optimization

We maximize a sum of concave functions (each $\log(1 + a_k {E_s}_{k})$ is concave in ${E_s}_{k}$ ) over a convex constraint set (the simplex $\sum_k {E_s}_{k} \leq E_s$ , ${E_s}_{k} \geq 0$ ).

This is a convex problem, so the KKT conditions are necessary and sufficient. The point is that convexity guarantees a unique global optimum — the water-filling solution is not a heuristic but provably optimal.

KKT conditions yield water-filling

The Lagrangian is

$\mathcal{L} = \sum_k \log\!\left(1 + \frac{|G_k|^2 {E_s}_{k}}{N_0}\right) - \lambda\!\left(\sum_k {E_s}_{k} - E_s\right) + \sum_k \mu_k {E_s}_{k}.$

Setting $\partial \mathcal{L}/\partial {E_s}_{k} = 0$ :

$\frac{|G_k|^2/N_0}{1 + |G_k|^2 {E_s}_{k}/N_0} = \lambda - \mu_k.$

Complementary slackness: $\mu_k \geq 0$ , $\mu_k {E_s}_{k} = 0$ . If ${E_s}_{k} > 0$ , then $\mu_k = 0$ and

${E_s}_{k} = \frac{1}{\lambda} - \frac{N_0}{|G_k|^2}.$

Setting $\nu = 1/\lambda$ gives ${E_s}_{k}^* = [\nu - N_0/|G_k|^2]_+$ .

Water-filling

The optimal power allocation for parallel Gaussian channels: ${E_s}_{k}^* = [\nu - N_0/|G_k|^2]_+$ . Allocates more power to stronger sub-channels, shutting off the weakest ones entirely. Named for the geometric analogy of pouring water over an uneven surface.

Related: Parallel Gaussian channel

Example: Water-Filling with Three Sub-Channels

Consider $K = 3$ sub-channels with gains $|G_1|^2 = 10$ , $|G_2|^2 = 2$ , $|G_3|^2 = 0.1$ , noise power $N_0 = 1$ , and total power $E_s = 1$ . Find the water-filling power allocation and the capacity.

Solution

Compute the inverse gains

$N_0/|G_1|^2 = 0.1$ , $N_0/|G_2|^2 = 0.5$ , $N_0/|G_3|^2 = 10$ .

Try all three channels active

If all channels are active: $3\nu - (0.1 + 0.5 + 10) = 1$ , so $\nu = 3.87$ . But then ${E_s}_{3} = 3.87 - 10 < 0$ . Contradiction — channel 3 cannot be active.

Try two channels active

With channels 1 and 2 only: $2\nu - (0.1 + 0.5) = 1$ , so $\nu = 0.8$ .

${E_s}_{1} = 0.8 - 0.1 = 0.7$ , ${E_s}_{2} = 0.8 - 0.5 = 0.3$ , ${E_s}_{3} = 0$ . Check: $0.7 + 0.3 = 1 = E_s$ . $\checkmark$

Compute capacity

$C = \log_2(1 + 10 \times 0.7) + \log_2(1 + 2 \times 0.3) = \log_2(8) + \log_2(1.6) = 3 + 0.678 = 3.678 \text{ bits}.$ $The weakest channel ($ |G_3|^2 = 0.1$) is shut off entirely — the power is better spent strengthening the already-good channels.

Why Convexity Matters Here

The water-filling solution is not just "a good idea" — it is provably optimal because the underlying optimization is convex. This has important consequences:

Uniqueness. There is exactly one optimal power allocation (up to degenerate cases where a channel is exactly at the threshold).
KKT sufficiency. Any point satisfying the KKT conditions is globally optimal — no need to check second-order conditions or worry about local maxima.
Efficient computation. The water level $\nu$ can be found by a simple bisection on the power constraint, or even in closed form for small $K$ .

This "convexity reflex" — recognizing when a problem is convex and exploiting it — is a recurring theme throughout information theory.

Water-Filling Power Allocation

Watch water pour over the inverted channel gains

N_0/|G_k|^2

. As total power increases, the water level

\nu

rises and progressively activates weaker sub-channels. The optimal power on each sub-channel equals the water depth above its bowl level.

Water-Filling Power Allocation

Visualize the water-filling solution for parallel Gaussian channels. The "bowl" shows the inverted channel gains $N_0/|G_k|^2$ , and the water level $\nu$ determines the power allocation. Adjust the total power to see how weak channels are progressively activated as power increases.

Parameters

Number of sub-channels

K

8

Total power

E_s

5

Channel gain profile

From ISI Channels to Parallel Channels via OFDM

The parallel Gaussian channel model is not merely theoretical — it is the operational model for OFDM (Orthogonal Frequency Division Multiplexing), the dominant wideband modulation in 4G, 5G, Wi-Fi, and DSL.

Consider a discrete-time channel with intersymbol interference (ISI): $Y_i = \sum_{\ell=0}^L g_\ell X_{i-\ell} + Z_{i}$ , where $(g_0, \ldots, g_L)$ is the finite impulse response. By prepending a cyclic prefix (CP) of length $L$ to each block of $K$ symbols, linear convolution becomes circular convolution. The resulting $K \times K$ circulant matrix is diagonalized by the DFT:

$\mathbf{G} = \mathbf{F}^H \text{diag}(G_0, \ldots, G_{K-1}) \mathbf{F},$

where $G_k = \sum_{\ell=0}^L g_\ell e^{-j2\pi k\ell/K}$ . Applying IDFT at the transmitter and DFT at the receiver converts the ISI channel into $K$ parallel Gaussian sub-channels — and water-filling gives the optimal power allocation.

Definition:
Capacity of the ISI Channel via OFDM

For a discrete-time ISI channel with $K$ subcarriers and cyclic prefix of length $L$ , the capacity per channel use is

$C_{K}(E_s) = \frac{K}{K+L} \cdot \frac{1}{K}\sum_{k=0}^{K-1} \left[\log\!\left(\frac{\nu\, |G_k|^2}{N_0}\right)\right]_+,$

where the factor $K/(K+L)$ accounts for the rate loss due to the cyclic prefix overhead.

In the limit $K \to \infty$ :

$C(\mathcal{P}) = \int_{-1/2}^{1/2} \left[\log\!\left(\frac{\nu\, |G(\xi)|^2}{N_0}\right)\right]_+ d\xi,$

where $G(\xi) = \sum_{\ell=0}^L g_\ell e^{-j2\pi\xi\ell}$ is the DTFT of the impulse response and $\nu$ satisfies $\int_{-1/2}^{1/2} [\nu - N_0/|G(\xi)|^2]_+ d\xi = \mathcal{P}$ .

Water-Filling Algorithm

Complexity:

O(K \log K)

due to the initial sort. The repeat loop runs at most

K

iterations.

Input: Channel gains

|G_1|^2, \ldots, |G_K|^2

, noise power

N_0

, total power

E_s

Output: Power allocation

{E_s}_{1}^*, \ldots, {E_s}_{K}^*

1. Sort sub-channels by effective noise level:

N_0/|G_{\pi(1)}|^2 \leq \cdots \leq N_0/|G_{\pi(K)}|^2

2. Set

\mathcal{A} \leftarrow \{1, \ldots, K\}

(active set)

3. repeat

a. Compute water level:

\nu = \frac{1}{|\mathcal{A}|}\left(E_s + \sum_{k \in \mathcal{A}} \frac{N_0}{|G_k|^2}\right)

b. Compute power:

{E_s}_{k} = \nu - N_0/|G_k|^2

for

k \in \mathcal{A}

c. if

\exists\, k \in \mathcal{A}

with

{E_s}_{k} < 0

:

Remove the worst channel from

\mathcal{A}

d. else: STOP

4. Set

{E_s}_{k} = 0

for

k \notin \mathcal{A}

5. return

({E_s}_{1}^*, \ldots, {E_s}_{K}^*)

In practice, a bisection search on $\nu$ is numerically simpler.

OFDM Water-Filling over a Frequency-Selective Channel

Visualize water-filling over the frequency response of a multipath channel. The inverted channel $N_0/|G(\xi)|^2$ forms the "bowl," and water is poured to level $\nu$ . Deep fades receive no power. Adjust total power to see how sub-carriers are activated.

Parameters

Number of channel taps

L+1

4

Number of OFDM subcarriers

K

64

Total power (dB)10

⚠️Engineering Note

Cyclic Prefix Overhead in Practice

The factor $K/(K+L)$ in the OFDM capacity accounts for the rate loss due to the cyclic prefix. In 5G NR, the normal CP length is approximately $L = K/14$ for the 15 kHz subcarrier spacing, giving an overhead of about 7%. For extended CP (used in high-delay-spread environments), the overhead increases to about 25%.

The choice of $K$ (FFT size) involves a tradeoff: larger $K$ reduces CP overhead but increases sensitivity to Doppler and requires longer processing blocks. 5G NR supports FFT sizes from 128 to 4096 with subcarrier spacings from 15 to 240 kHz.

Practical Constraints

•
Normal CP overhead in 5G NR: ~7% for 15 kHz SCS
•
Extended CP overhead: ~25%, used for high delay spread
•
Larger FFT reduces CP overhead but increases Doppler sensitivity

📋 Ref: 3GPP TS 38.211

Common Mistake: Equal Power Allocation Is Not Optimal

Mistake:

Distributing power equally across all sub-channels: ${E_s}_{k} = E_s/K$ for all $k$ .

Correction:

Equal power allocation ignores the channel gains and wastes power on weak sub-channels. Water-filling allocates more power to stronger channels and shuts off the weakest ones. The capacity gap between equal power and water-filling grows with channel variability. However, at high SNR the gap shrinks because all channels become active and the power differences become small relative to $N_0/|G_k|^2$ .

🎓CommIT Contribution(1998)

BICM Capacity Analysis

G. Caire, G. Taricco, E. Biglieri — IEEE Trans. Inform. Theory, vol. 44, no. 3, pp. 927-946

Caire, Taricco, and Biglieri introduced the mutual information analysis of Bit-Interleaved Coded Modulation (BICM), showing that BICM achieves a different (generally lower) mutual information than ideal coded modulation, but with significant practical advantages in complexity and flexibility. The BICM mutual information analysis is directly related to the parallel Gaussian channel capacity framework developed in this section — each bit level of the modulation can be viewed as a parallel sub-channel with its own effective SNR.

This work was later extended by Guill'en i F`abregas, Mart'inez, and Caire, who showed that BICM can be viewed as mismatched decoding and derived the corresponding error exponents.

BICMcoded modulationparallel channelsView Paper →

Why This Matters: OFDM in 4G LTE and 5G NR

The parallel Gaussian channel and water-filling framework is the information-theoretic foundation of OFDM-based systems. In 4G LTE and 5G NR, each OFDM subcarrier is a parallel sub-channel, and the resource allocation problem (which subcarriers to use, how much power on each) is a practical instantiation of water-filling.

In practice, perfect water-filling is approximated by adaptive modulation and coding (AMC): the base station measures the channel quality on each subcarrier group (resource block) and selects the modulation order and code rate accordingly. This is "quantized water-filling" — a practical version of the continuous solution.

See Book telecom, Ch. 14 for the full treatment of OFDM and Book telecom, Ch. 17 for multiuser OFDMA resource allocation.

Quick Check

In a water-filling solution with $K = 5$ sub-channels and total power $E_s = 2$ , suppose two channels have $N_0/|G_k|^2 > \nu$ . How many channels are active?

5

3

2

Cannot determine without the exact gains

Correction:

3

A channel is active if and only if $N_0/|G_k|^2 < \nu$ . Since two channels have inverse gains above the water level, they receive zero power. The remaining $5 - 2 = 3$ channels are active.

Parallel Gaussian Channels and Water-Filling

From One Channel to Many

Definition: Parallel Gaussian Channel

Parallel Gaussian channel

Theorem: Capacity of the Parallel Gaussian Channel

Independence gives additive capacity

Formulate as convex optimization

KKT conditions yield water-filling

Water-filling

Example: Water-Filling with Three Sub-Channels

Compute the inverse gains

Try all three channels active

Try two channels active

Compute capacity

Why Convexity Matters Here

Water-Filling Power Allocation

Water-Filling Power Allocation

Parameters

From ISI Channels to Parallel Channels via OFDM

Definition: Capacity of the ISI Channel via OFDM

Water-Filling Algorithm

OFDM Water-Filling over a Frequency-Selective Channel

Parameters

Cyclic Prefix Overhead in Practice

Common Mistake: Equal Power Allocation Is Not Optimal

BICM Capacity Analysis

Why This Matters: OFDM in 4G LTE and 5G NR

Quick Check

Definition:
Parallel Gaussian Channel

Definition:
Capacity of the ISI Channel via OFDM