Exercises

$\mathbf{y} = \mathbf{H}\mathbf{x} + \mathbf{n} = \begin{bmatrix} 1 & 2 & 0 \\ 0 & 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} + \begin{bmatrix} n_1 \\ n_2 \end{bmatrix}$$So$y_1 = x_1 + 2x_2 + n_1$and$y_2 = x_2 + x_3 + n_2$.

The two rows $[1, 2, 0]$ and $[0, 1, 1]$ are linearly independent (neither is a scalar multiple of the other), so $\mathrm{rank}(\mathbf{H}) = 2 = \min(n_r, n_t) = \min(2, 3)$. The channel is full-rank.

$\mathbf{H}\mathbf{H}^{H} = \begin{bmatrix} 5 & 2 \\ 2 & 2 \end{bmatrix}$$Eigenvalues:$\lambda = \frac{7 \pm \sqrt{9+16}}{2} = \frac{7 \pm 5}{2}$, giving$\lambda_1 = 6$and$\lambda_2 = 1$. Singular values:$\sigma_1 = \sqrt{6} \approx 2.45$,$\sigma_2 = 1$. Condition number:$\kappa = \sigma_1/\sigma_2 = \sqrt{6} \approx 2.45$.$\blacksquare$

For an $n \times n$ matrix with i.i.d. continuous entries (including $\mathcal{CN}(0,1)$), the matrix is full-rank with probability 1. This is because the set of rank-deficient matrices has measure zero in $\mathbb{C}^{n \times n}$ (it is a lower-dimensional algebraic variety).

Therefore, $\mathrm{rank}(\mathbf{H}) = 4$ with probability 1, and $\Pr[\mathrm{rank}(\mathbf{H}) < 4] = 0$. $\blacksquare$

Let $\mathbf{A} = \frac{1}{\sigma}\mathbf{H}\mathbf{Q}^{1/2}$ (size $n_r \times n_t$) and $\mathbf{B} = \frac{1}{\sigma}\mathbf{Q}^{1/2}\mathbf{H}^{H}$ (size $n_t \times n_r$). Then

$$\det\!\left(\mathbf{I}_{n_r} + \mathbf{A}\mathbf{B}\right) = \det\!\left(\mathbf{I}_{n_r} + \frac{1}{\sigma^2}\mathbf{H}\mathbf{Q}\mathbf{H}^{H}\right)$$

By Sylvester's identity:

$$= \det\!\left(\mathbf{I}_{n_t} + \mathbf{B}\mathbf{A}\right) = \det\!\left(\mathbf{I}_{n_t} + \frac{1}{\sigma^2}\mathbf{Q}^{1/2}\mathbf{H}^{H}\mathbf{H}\mathbf{Q}^{1/2}\right)$$

Since $\det(\mathbf{I} + \mathbf{Q}^{1/2}\mathbf{M}\mathbf{Q}^{1/2}) = \det(\mathbf{I} + \mathbf{M}\mathbf{Q})$ for positive semidefinite $\mathbf{Q}$, we get

$$= \det\!\left(\mathbf{I}_{n_t} + \frac{1}{\sigma^2}\mathbf{H}^{H}\mathbf{H}\mathbf{Q}\right) \qquad \blacksquare$$

With $\rho = 0$, $\mathbf{R}_{t} = \mathbf{I}_2$ and the channel is i.i.d. Rayleigh. By Telatar's formula with $n_t = n_r = 2$, $\text{SNR} = 100$:

$$C_{\mathrm{iid}} \approx 11.5 \;\text{bits/s/Hz}$$

(from Monte Carlo or using the known mean of the Wishart eigenvalues).

The eigenvalues of $\mathbf{R}_{t}$ are $1 + |\rho| = 1.9$ and $1 - |\rho| = 0.1$. The effective channel has one strong and one weak transmit mode.

The capacity loss at high SNR is approximately $\Delta C \approx -\log_2\det(\mathbf{R}_{t}) = -\log_2((1-|\rho|^2)) = -\log_2(0.19) \approx 2.4$ bits/s/Hz.

So $C_{\mathrm{corr}} \approx 11.5 - 2.4 = 9.1$ bits/s/Hz.

For $C_{\mathrm{corr}} = C_{\mathrm{iid}}/2 \approx 5.75$ bits/s/Hz, the system degenerates to essentially rank-1 (SISO-like). This occurs around $|\rho| \to 1$, where $\mathbf{R}_{t}$ becomes rank-1. Numerically, $|\rho| \approx 0.97$ gives approximately 50% capacity loss at 20 dB. $\blacksquare$

With $\mathbf{Q} = (P/n_t)\mathbf{I}$:

$$C_{\mathrm{equal}} = \log_2\det\!\left(\mathbf{I} + \frac{P}{n_t \sigma^2}\mathbf{H}\mathbf{H}^{H}\right) = \sum_{i=1}^{r} \log_2\!\left(1 + \frac{P}{n_t \sigma^2}\sigma_i^2\right)$$

where the last step uses the eigenvalue decomposition.

At high SNR, water-filling gives $p_i^* \approx P/r$ for all $r$ active sub-channels (the water level is much higher than all floor levels $\sigma^2/\sigma_i^2$). Thus

$$C_{\mathrm{WF}} \approx \sum_{i=1}^{r}\log_2\!\left(\frac{P\sigma_i^2}{r\sigma^2}\right)$$

With $P/(n_t\sigma^2) \gg 1$, the equal power allocation gives the same leading term $\sum_i \log_2(P\sigma_i^2/(n_t\sigma^2))$, with only a constant gap of $r\log_2(n_t/r)$ bits.

As $\text{SNR} \to \infty$, this gap becomes negligible relative to the growing $r\log_2(\text{SNR})$ terms. $\blacksquare$

Water-filling shuts off channel 2 when the water level $\mu \leq 1/\sigma_2^2$. At the boundary, $\mu = 1/\sigma_2^2$ and all power goes to channel 1:

$$p_1 = \mu - \frac{1}{\sigma_1^2} = \frac{1}{\sigma_2^2} - \frac{1}{\sigma_1^2} = P$$

So the threshold is

$$P_{\mathrm{th}} = \frac{1}{\sigma_2^2} - \frac{1}{\sigma_1^2} = \frac{\sigma_1^2 - \sigma_2^2}{\sigma_1^2 \sigma_2^2}$$

For $\sigma_1 = 2$, $\sigma_2 = 1$:

$$P_{\mathrm{th}} = \frac{4 - 1}{4 \times 1} = \frac{3}{4} = 0.75$$

In dB: $\text{SNR}_{\mathrm{th}} = 10\log_{10}(0.75) \approx -1.25$ dB.

For $\text{SNR} < -1.25$ dB, it is optimal to beamform (single-stream transmission). Above this threshold, both sub-channels should be used. $\blacksquare$

Let $p_i^*$ be the water-filling powers and $p_i^0 = P/n_t$ be equal powers. The gap is

$$\Delta C = \sum_{i=1}^{r}\left[\log_2\!\left(1 + \frac{p_i^* \sigma_i^2}{\sigma^2}\right) - \log_2\!\left(1 + \frac{P\sigma_i^2}{n_t \sigma^2}\right)\right]$$

At high SNR, $\Delta C \approx \sum_i \log_2(p_i^*/p_i^0)$. With $\sum p_i^* = P$ and $p_i^0 = P/n_t$:

$$\Delta C \leq \sum_{i=1}^{r}\log_2\!\left(\frac{P/r}{P/n_t}\right) = r\log_2\!\left(\frac{n_t}{r}\right)$$

using the fact that equal allocation among $r$ active channels maximises $\sum \log_2(p_i)$ for fixed $\sum p_i$.

The bound is zero when $r = n_t$ (all channels active, equal power is optimal) and maximum when $r = 1$ (beamforming regime, $\log_2(n_t)$ bits gain).

CSIT is most valuable when: (a) the channel is ill-conditioned (few active sub-channels), (b) the SNR is low (beamforming regime), or (c) $n_t \gg r$ (many antennas but few useful modes). $\blacksquare$

With isotropic input ($\mathbf{Q} = (P/n_t)\mathbf{I}_2$):

$$C = \mathbb{E}\!\left[\log_2\!\left(1 + \frac{P}{2\sigma^2}\|\mathbf{h}\|^2\right)\right]$$

Since $h_1, h_2 \sim \mathcal{CN}(0,1)$ independently, $\|\mathbf{h}\|^2 = |h_1|^2 + |h_2|^2 \sim \mathrm{Gamma}(2,1)$, which is $\chi^2_4/2$ (chi-squared with 4 real DoF, scaled).

With $\text{SNR} = 10$ (10 dB):

$$C = \mathbb{E}\!\left[\log_2\!\left(1 + 5\|\mathbf{h}\|^2\right)\right]$$

Using the known result for $\|\mathbf{h}\|^2 \sim \mathrm{Gamma}(2,1)$:

$$C = \int_0^\infty \log_2(1 + 5\gamma)\, \gamma e^{-\gamma}\, d\gamma$$

Numerical evaluation gives $C \approx 3.56$ bits/s/Hz.

For $1 \times 2$ SIMO with MRC, $y = \|\mathbf{h}\|^2 x + \tilde{n}$:

$$C_{\mathrm{SIMO}} = \mathbb{E}\!\left[\log_2\!\left(1 + \text{SNR} \cdot \|\mathbf{h}\|^2\right)\right]$$

At SNR = 10 dB: $C_{\mathrm{SIMO}} = \mathbb{E}[\log_2(1 + 10\|\mathbf{h}\|^2)] \approx 4.53$ bits/s/Hz.

The SIMO system is better because it uses full power on one antenna ($\text{SNR}$ vs. $\text{SNR}/n_t$ per mode), providing full array gain without the $1/n_t$ splitting loss. $\blacksquare$

At high SNR:

$$C \approx \sum_{i=1}^{n}\mathbb{E}\!\left[\log_2\!\left(\frac{\text{SNR}}{n}\lambda_i\right)\right] = n\log_2\!\left(\frac{\text{SNR}}{n}\right) + \sum_{i=1}^{n}\mathbb{E}[\log_2 \lambda_i]$$

For an $n \times n$ Wishart matrix $\mathbf{W} = \mathbf{H}\mathbf{H}^{H}$:

$$\sum_{i=1}^{n}\mathbb{E}[\ln \lambda_i] = \mathbb{E}[\ln\det \mathbf{W}] = \sum_{k=1}^{n}\psi(k)$$

This follows from the known formula for the expected log-determinant of a Wishart matrix (using the moment generating function of the log-determinant). Converting to $\log_2$:

$$\sum_{i=1}^{n}\mathbb{E}[\log_2 \lambda_i] = \frac{1}{\ln 2}\sum_{k=1}^{n}\psi(k)$$

Combining: $C \approx n\log_2(\text{SNR}) - n\log_2(n) + \frac{n}{\ln 2}\sum_{k=1}^{n}\psi(k)/n$.

More precisely: $C \approx n\log_2(\text{SNR}) - n\log_2(n) + \frac{1}{\ln 2}\sum_{k=1}^{n}\psi(k)$. $\blacksquare$

Generate $N = 10{,}000$ realisations of $\mathbf{H}$ with i.i.d. $\mathcal{CN}(0,1)$ entries. For each, compute $I_k = \log_2\det(\mathbf{I}_2 + \frac{31.6}{2}\mathbf{H}_{k}\mathbf{H}_{k}^{H})$.

Sort the values and find the 10th percentile. Typical results:

• Ergodic capacity: $\bar{C} \approx 8.8$ bits/s/Hz
• 10% outage capacity: $C_{0.1} \approx 5.2$ bits/s/Hz

The mutual information $I$ is approximately Gaussian with mean $\mu_I \approx 8.8$ and standard deviation $\sigma_I \approx 2.8$ (for $2 \times 2$ at 15 dB).

The 10% outage point is $C_{0.1} \approx \mu_I - 1.28\sigma_I \approx 8.8 - 3.6 = 5.2$ bits/s/Hz.

The ergodic-to-outage gap of 3.6 bits/s/Hz (41%) shows the importance of diversity: with more antennas (higher diversity), the variance $\sigma_I$ decreases and the gap shrinks. $\blacksquare$

Note that the $1 \times 16$ (SIMO) system has the same DoF as SISO despite 16 receive antennas. The extra antennas only provide array gain (a constant offset). $\blacksquare$

SISO: $C_{\mathrm{SISO}} = \mathbb{E}[\log_2(1 + \text{SNR}|h|^2)]$.

$4 \times 4$ MIMO: $C_{\mathrm{MIMO}} = \mathbb{E}[\sum_{i=1}^{4}\log_2(1 + \frac{\text{SNR}}{4}\lambda_i)]$.

At very low SNR ($\text{SNR} \to 0$): $C_{\mathrm{SISO}} \approx \frac{\text{SNR}}{\ln 2}\mathbb{E}[|h|^2] = \frac{\text{SNR}}{\ln 2}$.

$C_{\mathrm{MIMO}} \approx \frac{\text{SNR}}{4\ln 2}\mathbb{E}[\mathrm{tr}(\mathbf{H}\mathbf{H}^{H})] = \frac{\text{SNR}}{4\ln 2} \times 16 = \frac{4\text{SNR}}{\ln 2}$.

At low SNR, the MIMO system is actually $4\times$ better in capacity because $\mathbb{E}[\mathrm{tr}(\mathbf{H}\mathbf{H}^{H})] = n_t n_r = 16$ while SISO has $\mathbb{E}[|h|^2] = 1$. After dividing by $n_t = 4$, the $4 \times 4$ still has $4\times$ the first-order capacity.

Therefore, $4 \times 4$ MIMO outperforms $1 \times 1$ SISO at all SNR values, not just high SNR. The crossover never occurs --- MIMO is always better (or equal)! The multiplexing gain at high SNR is an additional benefit on top of the array gain at low SNR. $\blacksquare$

$C = \mathbb{E}\!\left[\sum_{i=1}^{n}\log_2\!\left(1+\frac{\text{SNR}}{n}\lambda_i\right)\right] \approx \frac{\text{SNR}}{n\ln 2}\sum_{i=1}^{n}\mathbb{E}[\lambda_i] = \frac{\text{SNR}}{n\ln 2}\mathbb{E}[\mathrm{tr}(\mathbf{H}\mathbf{H}^{H})]$$Since$\mathbb{E}[\mathrm{tr}(\mathbf{H}\mathbf{H}^{H})] = n^2$for an$n \times n$i.i.d. matrix:$C \approx n \cdot \text{SNR}/\ln 2$.

$\frac{E_b}{N_0} = \frac{\text{SNR}}{C} \to \frac{\text{SNR}}{n \cdot \text{SNR}/\ln 2} = \frac{\ln 2}{n}$$For SISO:$(E_b/N_0){\min} = \ln 2 \approx -1.59$dB. For$n \times n$MIMO:$(E_b/N_0){\min} = \ln 2/n$, which is$10\log_{10}(n)$dB below the SISO limit. For$4 \times 4$:$-1.59 - 10\log_{10}(4) = -1.59 - 6.02 = -7.61$dB. MIMO provides an energy efficiency gain proportional to$n$.$\blacksquare$

$\min(n_t, n_r) = 2$, so $r \in [0, 2]$:

The $4 \times 2$ has twice the full diversity of $2 \times 2$ ($d^*(0) = 8$ vs. 4) but the same maximum multiplexing gain ($r = 2$).

The $3 \times 3$ provides 9 diversity levels and 3 multiplexing streams. $\blacksquare$

$r = 3/2$. This is a non-integer value between $r = 1$ and $r = 2$.

For $n_t = n_r = 2$, $d^*(1) = 1$ and $d^*(2) = 0$. Linear interpolation:

$$d^*(3/2) = d^*(1) + \frac{3/2 - 1}{2 - 1}(d^*(2) - d^*(1)) = 1 + \frac{1}{2}(0 - 1) = \frac{1}{2}$$

The outage probability decays as $P_{\mathrm{out}} \doteq \text{SNR}^{-1/2}$, which is quite slow. At 20 dB, $P_{\mathrm{out}} \sim 10^{-1}$ (10%).

At $r = 1$: $d^*(1) = 1$, so $P_{\mathrm{out}} \doteq \text{SNR}^{-1}$.

At 20 dB: $P_{\mathrm{out}} \sim 10^{-2}$ (1%), which is 10x more reliable. Reducing the multiplexing gain from 3/2 to 1 (a 33% rate reduction) improves reliability by a factor of 10 at 20 dB. $\blacksquare$

The rate scales as $R \sim \log_2(\text{SNR})$, giving $r = 1$. The diversity order is $d = 2n_r = 4$.

But wait: the optimal $d^*(1) = (2-1)^2 = 1$ for $2 \times 2$. The Alamouti code achieves $d = 4 > 1$!

The Alamouti code achieves $d = 4$ because it is a rate-$1$ code (1 symbol per channel use), so it sends at $R = \log_2(1 + \text{SNR})$. However, the DMT framework uses $R = r\log_2(\text{SNR})$, and

$$\frac{R}{\log_2(\text{SNR})} = \frac{\log_2(1+\text{SNR})}{\log_2(\text{SNR})} \to 1$$

In the strict DMT sense, $r = 1$, and the Alamouti code achieves $d = 2n_r = 4$, which is above the optimal curve $d^*(1) = 1$.

This apparent contradiction is resolved by noting that $d = 2n_r$ is the fixed-rate diversity (treating $R$ as fixed), while the DMT framework treats $R$ as growing. Under the DMT definition, the Alamouti code at $r = 1$ achieves $d = 1$, not 4.

The Golden code achieves $d^*(r) = (2-r)^2$ for all $r \in [0, 2]$, matching the optimal curve everywhere. At $r = 1$, the Golden code achieves $d = 1$ (same as Alamouti in the DMT sense) but also achieves $r = 2$ with $d = 0$ (Alamouti cannot reach $r = 2$ since its rate is capped at $\log_2(1 + \text{SNR})$). $\blacksquare$

$\mathbf{H} = \mathbf{a}_r\mathbf{a}_t^H$ is an outer product of two nonzero vectors (with probability 1). Any outer product $\mathbf{u}\mathbf{v}^{H}$ has rank 1 (all columns are scalar multiples of $\mathbf{u}$).

The single nonzero eigenvalue of $\mathbf{H}\mathbf{H}^{H}$ is

$$\lambda = \mathrm{tr}(\mathbf{H}\mathbf{H}^{H}) = \|\mathbf{a}_r\|^2 \|\mathbf{a}_t\|^2$$

With isotropic input:

$$C = \mathbb{E}\!\left[\log_2\!\left(1 + \frac{\text{SNR}}{n_t}\|\mathbf{a}_r\|^2\|\mathbf{a}_t\|^2\right)\right]$$

Since $\|\mathbf{a}_r\|^2 \sim \mathrm{Gamma}(n_r, 1)$ and $\|\mathbf{a}_t\|^2 \sim \mathrm{Gamma}(n_t, 1)$ independently, the product $\|\mathbf{a}_r\|^2\|\mathbf{a}_t\|^2$ has a known distribution (product of independent Gamma variables).

At high SNR:

$$C \approx \log_2(\text{SNR}) + \log_2\!\left(\frac{\mathbb{E}[\|\mathbf{a}_r\|^2\|\mathbf{a}_t\|^2]}{n_t}\right) = \log_2(\text{SNR}) + \log_2(n_r)$$

So $\mathrm{DoF} = 1$ regardless of $n_t, n_r$. The extra antennas provide only array gain ($\log_2(n_r)$ bits from receive array gain), not multiplexing gain. $\blacksquare$

The ergodic capacity with input covariance $\mathbf{Q}$ is

$$C(\mathbf{Q}) = \mathbb{E}\!\left[\log_2\det\!\left(\mathbf{I} + \frac{1}{\sigma^2}\mathbf{H}\mathbf{Q}\mathbf{H}^{H}\right)\right]$$

By the concavity of $\log\det$ and the structure of the Kronecker channel $\mathbf{H} = \mathbf{R}_{r}^{1/2}\mathbf{H}_{w}\mathbf{R}_{t}^{1/2}$, the transmitter should align $\mathbf{Q}$ with the eigenvectors of $\mathbf{R}_{t}$ (the transmit correlation).

Write $\mathbf{R}_{t} = \mathbf{V}_t \boldsymbol{\Lambda}_t \mathbf{V}_t^H$ and $\mathbf{Q} = \mathbf{V}_t \mathrm{diag}(q_1, \ldots, q_{n_t})\mathbf{V}_t^H$. The optimal powers $\{q_i\}$ satisfy a water-filling condition with effective channel gains that are functions of $\{\lambda_{t,i}\}$ and $\mathbf{R}_{r}$.

For the special case $\mathbf{R}_{r} = \mathbf{I}$:

$$C = \mathbb{E}\!\left[\sum_{i=1}^{n_t}\log_2\!\left(1 + \frac{q_i \lambda_{t,i}}{\sigma^2}\tilde{\lambda}_i\right)\right]$$

where $\tilde{\lambda}_i$ are eigenvalues of $\mathbf{H}_{w}\mathbf{H}_{w}^{H}$. The optimal $q_i$ follow from a statistical water-filling analogous to Section 15.3 but with averaged channels. $\blacksquare$

For $n_t = n_r = n$, the empirical spectral distribution of $\frac{1}{n}\mathbf{H}\mathbf{H}^{H}$ converges to the Marchenko-Pastur law with ratio $\beta = n_t/n_r = 1$.

The capacity is

$$C = n \cdot \frac{1}{n}\sum_{i=1}^{n}\log_2\!\left(1 + \frac{\text{SNR}}{n}\lambda_i\right) = n \cdot \frac{1}{n}\sum_{i=1}^{n}\log_2\!\left(1 + \text{SNR}\cdot\frac{\lambda_i}{n}\right)$$

As $n \to \infty$, $\frac{1}{n}\sum_i f(\lambda_i/n) \to \int f(x) dF_{\mathrm{MP}}(x)$ where $F_{\mathrm{MP}}$ is the Marchenko-Pastur CDF.

For $\beta = 1$, the Marchenko-Pastur density has mean $\mathbb{E}[\lambda_i/n] \to 1$. At the scale $\lambda_i/n$, the distribution concentrates around its mean (by the law of large numbers for empirical spectral measures), so

$$\frac{C}{n} \to \log_2(1 + \text{SNR} \cdot 1) = \log_2(1 + \text{SNR})$$

This confirms that $n$ MIMO antennas create $n$ parallel AWGN channels each with the same SNR, achieving a linear capacity gain. The per-antenna capacity equals the SISO AWGN capacity --- the ultimate efficiency of massive MIMO. $\blacksquare$