Ferkans — Interactive Telecom Tutor

Capacity as the Ultimate Limit

Having established the MIMO input-output model and the SVD decomposition into parallel channels, we can now answer the fundamental question: what is the maximum achievable rate?

For a known (deterministic) channel $\mathbf{H}$ , the capacity is achieved by choosing the optimal input covariance matrix $\mathbf{Q} = \mathbb{E}[\mathbf{x}\mathbf{x}^H]$ subject to a power constraint. The solution combines the SVD decomposition from Section 15.1 with the water-filling principle from Chapter 11, yielding one of the most elegant results in information theory.

Definition:
MIMO Capacity (Deterministic Channel)

For a deterministic MIMO channel $\mathbf{y} = \mathbf{H}\mathbf{x} + \mathbf{n}$ with $\mathbf{n} \sim \mathcal{CN}(\mathbf{0}, \sigma^2 \mathbf{I})$ and power constraint $\mathrm{tr}(\mathbf{Q}) \leq P$ , the MIMO capacity is

$C = \max_{\mathbf{Q} \succeq 0,\; \mathrm{tr}(\mathbf{Q}) \leq P} \log_2 \det\!\left(\mathbf{I}_{n_r} + \frac{1}{\sigma^2} \mathbf{H}\mathbf{Q}\mathbf{H}^{H}\right) \quad \text{bits/s/Hz}$

where $\mathbf{Q} \succeq 0$ means $\mathbf{Q}$ is positive semidefinite (valid covariance matrix).

This is the mutual information $I(\mathbf{x}; \mathbf{y})$ maximised over Gaussian inputs. Gaussian inputs are optimal because the noise is Gaussian (maximum entropy property).

,

Definition:
Parallel Channel Decomposition

Using the SVD $\mathbf{H} = \mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^H$ and the transformation from Theorem 15.1.1, the MIMO channel decomposes into $r = \mathrm{rank}(\mathbf{H})$ parallel scalar channels:

$\tilde{y}_i = \sigma_i \tilde{x}_i + \tilde{n}_i, \qquad i = 1, \ldots, r$

Each sub-channel has gain $\sigma_i^2$ (the $i$ -th eigenvalue of $\mathbf{H}^{H}\mathbf{H}$ ). The MIMO capacity problem reduces to optimally allocating total power $P$ across these $r$ parallel Gaussian channels, which is solved by water-filling.

Theorem: MIMO Capacity via SVD and Water-Filling

The capacity of the deterministic MIMO channel $\mathbf{y} = \mathbf{H}\mathbf{x} + \mathbf{n}$ with $\mathbf{H} = \mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^H$ and power constraint $P$ is

$C = \sum_{i=1}^{r} \log_2\!\left(1 + \frac{p_i^* \sigma_i^2}{\sigma^2}\right) \quad \text{bits/s/Hz}$

where the optimal power allocation $\{p_i^*\}$ is given by water-filling:

$p_i^* = \left(\mu - \frac{\sigma^2}{\sigma_i^2}\right)^+$

and $\mu$ is chosen so that $\sum_{i=1}^{r} p_i^* = P$ . Here $(x)^+ = \max(x, 0)$ .

The optimal input covariance matrix is $\mathbf{Q}^* = \mathbf{V}\, \mathrm{diag}(p_1^*, \ldots, p_r^*, 0, \ldots)\, \mathbf{V}^H$ .

Water-filling allocates more power to stronger sub-channels (large $\sigma_i^2$ ) and less to weaker ones, even shutting off the weakest sub-channels entirely. At high SNR, the water level $\mu$ is high relative to all $\sigma^2/\sigma_i^2$ , so power is approximately equal across sub-channels. At low SNR, it is better to concentrate power on the strongest sub-channel (beamforming).

Proof

Reduce to parallel channels

Apply the SVD decomposition from Theorem 15.1.1. The MIMO mutual information under Gaussian inputs with covariance $\mathbf{Q}$ is

$I(\mathbf{x}; \mathbf{y}) = \log_2 \det\!\left(\mathbf{I} + \frac{1}{\sigma^2}\mathbf{H}\mathbf{Q}\mathbf{H}^{H}\right)$

In the SVD basis, the optimal $\mathbf{Q}$ is diagonal (the off-diagonal elements cannot increase mutual information because the sub-channels are decoupled), so $\tilde{\mathbf{Q}} = \mathrm{diag}(p_1, \ldots, p_r)$ .

Decompose the determinant

With diagonal $\tilde{\mathbf{Q}}$ :

$\log_2 \det\!\left(\mathbf{I} + \frac{1}{\sigma^2}\boldsymbol{\Sigma}\tilde{\mathbf{Q}}\boldsymbol{\Sigma}^H\right) = \sum_{i=1}^{r} \log_2\!\left(1 + \frac{p_i \sigma_i^2}{\sigma^2}\right)$

This is a sum of $r$ independent terms, each depending on one power variable $p_i$ .

Apply water-filling (KKT conditions)

Maximise $\sum_i \log_2(1 + p_i \sigma_i^2/\sigma^2)$ subject to $\sum_i p_i = P$ and $p_i \geq 0$ . The Lagrangian is

$\mathcal{L} = \sum_i \log_2\!\left(1 + \frac{p_i \sigma_i^2}{\sigma^2}\right) - \mu\!\left(\sum_i p_i - P\right)$

Setting $\partial \mathcal{L}/\partial p_i = 0$ :

$\frac{\sigma_i^2/\sigma^2}{1 + p_i \sigma_i^2/\sigma^2} = \mu \ln 2$

Solving for $p_i$ with the non-negativity constraint:

$p_i^* = \left(\frac{1}{\mu \ln 2} - \frac{\sigma^2}{\sigma_i^2}\right)^+$

Redefining $\mu' = 1/(\mu \ln 2)$ as the "water level" gives the standard water-filling form. The water level is found by the constraint $\sum_i p_i^* = P$ . $\blacksquare$

,

SVD Decomposition into Parallel Sub-Channels — The SVD transforms the MIMO channel $\mathbf{H}$ into $r = \mathrm{rank}(\mathbf{H})$ parallel scalar sub-channels. Precoding with $\mathbf{V}$ at the transmitter and combining with $\mathbf{U}^H$ at the receiver diagonalises the channel. Water-filling allocates power $p_i^*$ to each sub-channel based on its gain $\sigma_i$ .

SVD Decomposition of a MIMO Channel

Watch how the SVD decomposes a

4 \times 4

MIMO channel into parallel sub-channels, with water-filling power allocation adapting to the singular value spread.

The animation shows the singular values of a random channel realisation, the resulting parallel sub-channels, and how water-filling allocates more power to stronger modes.

SVD Parallel Channels and Water-Filling

Visualise the SVD decomposition of a MIMO channel into parallel sub-channels and the water-filling power allocation. Adjust the channel singular values and total SNR to see how power is distributed.

Parameters

n_t

4

n_r

4

SNR (dB)15

Channel type

Type of channel realisation to generate

Example: Water-Filling Power Allocation

A $3 \times 3$ MIMO channel has singular values $\sigma_1 = 2.0$ , $\sigma_2 = 1.0$ , $\sigma_3 = 0.3$ . The noise variance is $\sigma^2 = 1$ and total power is $P = 10$ .

(a) Find the water-filling power allocation.

(b) Compute the capacity.

(c) Compare with equal power allocation.

Solution

Set up the water-filling equations

The channel gains are $\sigma_1^2 = 4.0$ , $\sigma_2^2 = 1.0$ , $\sigma_3^2 = 0.09$ . The "inverse gains" (water-filling floors) are

$\frac{\sigma^2}{\sigma_1^2} = 0.25, \qquad \frac{\sigma^2}{\sigma_2^2} = 1.0, \qquad \frac{\sigma^2}{\sigma_3^2} = 11.11$

Find the water level

First try all 3 channels active. The water level must satisfy $\sum_{i=1}^{3}(\mu - \sigma^2/\sigma_i^2)^+ = P = 10$ :

$3\mu - (0.25 + 1.0 + 11.11) = 10 \implies \mu = 7.45$

Check: $p_3 = 7.45 - 11.11 = -3.66 < 0$ . So channel 3 is shut off.

Try 2 channels active: $2\mu - (0.25 + 1.0) = 10 \implies \mu = 5.625$

$p_1 = 5.625 - 0.25 = 5.375$ , $p_2 = 5.625 - 1.0 = 4.625$ , $p_3 = 0$ . All non-negative, so this is the solution.

Compute capacity

$C = \log_2(1 + 5.375 \times 4) + \log_2(1 + 4.625 \times 1) + 0KATEXPLACEHOLDER0END= \log_2(22.5) + \log_2(5.625) = 4.49 + 2.49 = 6.98 \;\text{bits/s/Hz}$ $

Compare with equal power

Equal power: $p_1 = p_2 = p_3 = 10/3 \approx 3.33$ .

$C_{\mathrm{equal}} = \log_2(1 + 3.33 \times 4) + \log_2(1 + 3.33 \times 1) + \log_2(1 + 3.33 \times 0.09)$

$= \log_2(14.33) + \log_2(4.33) + \log_2(1.30) = 3.84 + 2.11 + 0.38 = 6.33 \;\text{bits/s/Hz}$

Water-filling gains $6.98 - 6.33 = 0.65$ bits/s/Hz ( $10\%$ improvement) by redirecting power from the weak third sub-channel to the stronger first and second. $\blacksquare$

Common Mistake: Equal Power Allocation Is Near-Optimal at High SNR

Mistake:

Spending significant effort on water-filling at high SNR, or conversely, using equal power allocation at low SNR where it is highly suboptimal.

Correction:

At high SNR, the water level $\mu$ is much larger than all $\sigma^2/\sigma_i^2$ , so $p_i^* \approx P/r$ for all active sub-channels. Equal power allocation loses very little.

At low SNR, water-filling concentrates all power on the strongest sub-channel (beamforming), which can provide several dB of gain over equal allocation. The transition occurs around $\text{SNR} \approx \sigma_1^2/\sigma_r^2$ (ratio of strongest to weakest eigenvalue).

Quick Check

A $2 \times 2$ MIMO channel has singular values $\sigma_1 = 3$ and $\sigma_2 = 0$ . What is the capacity at SNR $= P/\sigma^2 = 20$ dB with optimal power allocation?

$C = \log_2(1 + 100 \times 9) = \log_2(901) \approx 9.82$ bits/s/Hz

$C = 2 \log_2(1 + 50 \times 9) \approx 17.6$ bits/s/Hz

$C = \log_2(1 + 100 \times 3) = \log_2(301) \approx 8.23$ bits/s/Hz

$C = 0$ bits/s/Hz because one singular value is zero

Correction:

C = \log_2(1 + 100 \times 9) = \log_2(901) \approx 9.82

bits/s/Hz

The channel has rank 1 ( $\sigma_2 = 0$ ), so all power goes to the single active sub-channel. $C = \log_2(1 + P \sigma_1^2/\sigma^2) = \log_2(1 + 100 \times 9)$ .

Water-filling

The optimal power allocation strategy for parallel Gaussian channels that allocates more power to stronger sub-channels and less (or none) to weaker ones: $p_i^* = (\mu - \sigma^2/\sigma_i^2)^+$ .

MIMO capacity

The maximum mutual information achievable over a MIMO channel, optimised over the input covariance matrix: $C = \max_{\mathbf{Q}} \log_2\det(\mathbf{I} + \frac{1}{\sigma^2}\mathbf{H}\mathbf{Q}\mathbf{H}^{H})$ .

Related: Water-filling, Channel rank

MIMO Capacity: Deterministic Channels