Exercises

$F^{\mathbf{A}}(\lambda) = \frac{1}{5}[\mathbf{1}_{\{\lambda \geq 1\}} + 2\cdot\mathbf{1}_{\{\lambda \geq 2\}} + \mathbf{1}_{\{\lambda \geq 3\}} + \mathbf{1}_{\{\lambda \geq 5\}}]$. Thus $F^{\mathbf{A}}(1.5) = 1/5$ and $F^{\mathbf{A}}(3) = 4/5$.

$\lambda_- = (1 - 1)^2 = 0$, $\lambda_+ = (1 + 1)^2 = 4$. Correct.

$\int_0^4 \frac{1}{2\pi}\sqrt{\frac{4-\lambda}{\lambda}}\, d\lambda$. Set $\lambda = 2 + 2\cos\theta$, $d\lambda = -2\sin\theta\, d\theta$, with $\theta$ from $\pi$ to $0$. $\sqrt{(4-\lambda)/\lambda} = \sqrt{(2-2\cos\theta)/(2+2\cos\theta)} = \sqrt{\frac{1-\cos\theta}{1+\cos\theta}} = |\tan(\theta/2)|$. The integral becomes $\frac{1}{2\pi}\int_0^\pi \tan(\theta/2) \cdot 2\sin\theta\, d\theta = \frac{1}{\pi}\int_0^\pi 2\sin^2(\theta/2)\, d\theta = 1$.

$\mathbb{E}[\frac{1}{m}\text{tr}(\frac{1}{m}\mathbf{H}^H\mathbf{H})] = \frac{1}{m^2}\sum_{i,j}\mathbb{E}[|H_{ij}|^2] = \frac{nm}{m^2} = \frac{n}{m} = \beta$. But the ESD of $\mathbf{W} = \frac{1}{m}\mathbf{H}^H\mathbf{H}$ is over the $m$ eigenvalues, so $\int \lambda\, dF^{\mathbf{W}} = \frac{1}{m}\text{tr}(\mathbf{W}) \to \beta$... Hmm.

Correction: for $\beta \leq 1$, all $m$ eigenvalues of $\frac{1}{m}\mathbf{H}^H\mathbf{H}$ are captured by the continuous MP part. The first moment of the ESD is $\frac{1}{m}\text{tr}(\frac{1}{m}\mathbf{H}^H\mathbf{H}) = n/m = \beta$.

So $m_1 = \beta$ for the MP distribution of $\frac{1}{m}\mathbf{H}^H\mathbf{H}$.

Note: some references normalize differently. With the convention in this chapter ($\mathbf{W} = \frac{1}{m}\mathbf{H}^H\mathbf{H}$, entries $\mathcal{CN}(0,1)$), the first moment is $\beta$.

$\frac{1}{m}\text{tr}(\mathbf{W}^2) = \frac{1}{m^3}\sum_{a,b,c,d} H_{ab}^* H_{ac} H_{cd}^* H_{cb}$.

The only nonzero pairings (for i.i.d. $\mathcal{CN}(0,1)$ entries) are: (i) $a=a, b=c, c=a, d=b$: contributes $nm/m^3 \cdot m = n/m = \beta$. (ii) $a=c, b=d$ (and distinct from (i)): contributes $n^2/m^2 = \beta^2$. Total: $m_2 = \beta + \beta^2$.

For $\beta = 1$: $m_2 = 2$. This matches the second moment of the MP density on $[0,4]$.

$m_F(z) = \int \frac{1}{\lambda - z}\, d\delta_a(\lambda) = \frac{1}{a - z}$. This is a simple pole at $z = a$ with residue $1$.

$m_F(z) = \frac{1}{2}\left[\log(\lambda - z)\right]_0^2 = \frac{1}{2}\log\frac{2 - z}{-z} = \frac{1}{2}\log\left(1 - \frac{2}{z}\right)$.

For $z = x + j\eta$ with $0 < x < 2$ and $\eta \to 0$: $\frac{2-z}{-z} = \frac{2-x-j\eta}{-x-j\eta}$. As $\eta \to 0^+$, the argument of the log transitions from positive to negative real part, picking up an imaginary part of $\pm \pi$. $\text{Im}\, m_F(x + j0^+) = -\pi/2$ for $x \in (0,2)$ and $0$ outside. Thus $f(x) = -\frac{1}{\pi}\cdot(-\pi/2) = 1/2$ on $(0,2)$, confirming the uniform density.

With $\beta = 1$ and $z = -1/\rho$: $(-1/\rho)m^2 + (-1/\rho)m + 1 = 0$. Multiply by $-\rho$: $m^2 + m - \rho = 0$.

$m = \frac{-1 + \sqrt{1 + 4\rho}}{2}$ (taking the root with $m > 0$ for $z < 0$).

This value of $m(-1/\rho)$ enters the ergodic capacity formula for $\beta = 1$: $\bar{C} = \log_2(1 + \rho - \frac{1}{4}(\sqrt{1+4\rho} - 1)^2/\rho \cdot \ldots)$. The exact closed-form capacity involves this quantity.

At $\lambda = \lambda_-$: $f_{\mathrm{MP}}(\lambda_-) \propto \sqrt{(\lambda_+ - \lambda_-)(\lambda_- - \lambda_-)} = 0$. At $\lambda = \lambda_+$: $f_{\mathrm{MP}}(\lambda_+) \propto \sqrt{(\lambda_+ - \lambda_+)(\lambda_+ - \lambda_-)} = 0$. The density vanishes as $\sqrt{\lambda - \lambda_-}$ near the left edge and as $\sqrt{\lambda_+ - \lambda}$ near the right edge. This square-root vanishing is universal — it holds for all Wigner-type and Wishart-type matrices.

For large $\rho$: $\bar{C} \approx \int_0^4 \log_2(\rho\lambda) f_{\mathrm{MP}}(\lambda;1)\, d\lambda = \log_2(\rho) + \int_0^4 \log_2(\lambda) f_{\mathrm{MP}}(\lambda;1)\, d\lambda$.

$\int_0^4 \ln(\lambda) f_{\mathrm{MP}}(\lambda;1)\, d\lambda = -1$ (this is a known result for the MP distribution with $\beta = 1$; it can be verified by substitution).

$\bar{C}/n_r \approx \log_2(\rho) - 1/\ln 2 \approx \log_2(\rho) - 1.443$ bits/s/Hz. The constant $1/\ln 2$ represents the capacity loss due to eigenvalue spread compared to having all eigenvalues equal to 1.

$\lambda_+ - \lambda_- = (1+\sqrt{\beta})^2 - (1-\sqrt{\beta})^2 = 4\sqrt{\beta} \to 0$ as $\beta \to 0$. Both endpoints converge to $1$.

Since the density is supported on an interval shrinking to $\{1\}$ and integrates to 1, $F_\beta \to \delta_1$ weakly as $\beta \to 0$.

When $n_t \gg n_r$ (massively more transmit antennas than users), all eigenvalues of $\frac{1}{n_t}\mathbf{H}^H\mathbf{H}$ concentrate near 1. This means all $n_r$ spatial channels have nearly the same gain — the channel becomes effectively orthogonal. This is the channel hardening effect of massive MIMO.

For $z = x + j\eta$ with $\eta > 0$ and $\lambda \in \mathbb{R}$: $|\lambda - z|^2 = (\lambda - x)^2 + \eta^2 \geq \eta^2$, so $|1/(\lambda - z)| \leq 1/\eta = 1/\text{Im}(z)$.

$|m_F(z)| = |\int \frac{1}{\lambda - z}\, dF(\lambda)| \leq \int \frac{1}{|\lambda - z|}\, dF(\lambda) \leq \frac{1}{\text{Im}(z)} \int dF(\lambda) = \frac{1}{\text{Im}(z)}$.

With $\boldsymbol{\Sigma}_{r} = \mathbf{I}_{n_r}$: $\tilde{\mathbf{T}} = \frac{1}{m}(\mathbf{I}_{n_r} + \rho\, \mathbf{T})^{-1}$. But $\mathbf{T} = (\mathbf{I}_m + \rho\, \boldsymbol{\Sigma}_{t} \tilde{T})^{-1}$ where $\tilde{\mathbf{T}} = \tilde{T}\, \mathbf{I}$ (scalar times identity since $\boldsymbol{\Sigma}_{r} = \mathbf{I}$).

$\tilde{T} = \frac{1}{m}\text{tr}\left(\mathbf{I}_{n_r} + \frac{\rho}{m}\text{tr}(\mathbf{I}_m + \rho\tilde{T}\, \boldsymbol{\Sigma}_{t})^{-1}\cdot \mathbf{I}_{n_r}\right)^{-1}$...

More precisely: $\tilde{T} = \frac{1}{n_r + \rho\, \text{tr}(\mathbf{I}_m + \rho\tilde{T}\, \boldsymbol{\Sigma}_{t})^{-1}/m \cdot n_r}$...

The key simplification is that $\tilde{T}$ satisfies a single scalar fixed-point equation $\tilde{T} = \frac{1}{m}\frac{1}{1 + \rho\delta(\tilde{T})}$ where $\delta(\tilde{T}) = \frac{1}{m}\text{tr}(\mathbf{I}_m + \rho\tilde{T}\, \boldsymbol{\Sigma}_{t})^{-1}$. This is a scalar equation (the matrix inversion is needed once, but the iteration is one-dimensional).

import numpy as np
n, m, N = 50, 200, 1000
eigs = []
for _ in range(N):
    H = (np.random.randn(n,m) + 1j*np.random.randn(n,m)) / np.sqrt(2)
    W = H.conj().T @ H / m
    eigs.extend(np.linalg.eigvalsh(W))

The histogram of the pooled $50000$ eigenvalues (only $n=50$ nonzero per realization) should closely match the MP density on $[1/4, 9/4]$, with $m-n = 150$ zero eigenvalues per realization forming a mass at the origin.

$\frac{d\bar{C}}{d\rho} = \frac{1}{\ln 2}\int_0^4 \frac{\lambda}{1+\rho\lambda} f_{\mathrm{MP}}(\lambda;1)\, d\lambda$.

$\int \frac{\lambda}{1+\rho\lambda}\, dF(\lambda) = \int \frac{1}{1+\rho\lambda}\, dF(\lambda) - \int \frac{1}{\rho(1+\rho\lambda)}\, dF(\lambda) \cdot 0$...

More directly: $\int \frac{\lambda}{1+\rho\lambda}\, dF = \frac{1}{\rho}(1 - \int \frac{1}{1+\rho\lambda}\, dF) = \frac{1}{\rho}(1 + \frac{1}{\rho}m_F(-1/\rho))$ using $\int \frac{1}{\lambda - z}\, dF = m_F(z)$ with $z = -1/\rho$.

Substituting $m_{F_1}(-1/\rho) = \frac{-1 + \sqrt{1 + 4\rho}}{2}$ (from Exercise 7) and integrating $\frac{d\bar{C}}{d\rho}$ from $0$ to the target $\rho$ gives the closed-form capacity.

$\frac{1}{m}\text{tr}(\mathbf{W}^k) = \frac{1}{m^{k+1}}\sum \prod H_{i_\ell j_\ell}^* H_{i_\ell j_{\ell+1}}$. The leading-order contributions come from pairings where each entry $H_{ij}$ appears exactly twice (once conjugated, once not). These are the non-crossing pair partitions.

For non-crossing pair partitions, each pair contributes $\mathbb{E}[|H_{ij}|^2] = 1$ (by the unit variance assumption). Crossing partitions and higher-order terms contribute factors like $\mathbb{E}[|H_{ij}|^4] - 2$ or $\mathbb{E}[H_{ij}^2]$, but these come with combinatorial prefactors of $O(m^{-1})$ and vanish in the limit.

Since only the first two moments of the entries matter, and both Gaussian and non-Gaussian i.i.d. entries with zero mean and unit variance give the same moments, the LSD is the same: the Marchenko-Pastur distribution. This is one of the most remarkable universality results in probability theory.

$m_1 = \beta = 1/4$, $m_2 = \beta + \beta^2 = 1/4 + 1/16 = 5/16$. $\text{Var} = 5/16 - (1/4)^2 = 5/16 - 1/16 = 4/16 = 1/4 = \beta$. The variance of the MP distribution equals $\beta$ — a clean result.

$(\mathbf{A} - z\mathbf{I})^{-1} = \mathbf{U}(\boldsymbol{\Lambda} - z\mathbf{I})^{-1}\mathbf{U}^H$. $\text{tr}(\mathbf{A} - z\mathbf{I})^{-1} = \text{tr}(\boldsymbol{\Lambda} - z\mathbf{I})^{-1} = \sum_{i=1}^n \frac{1}{\lambda_i - z}$. Dividing by $n$: $\frac{1}{n}\text{tr}(\mathbf{A} - z\mathbf{I})^{-1} = \frac{1}{n}\sum_i \frac{1}{\lambda_i - z} = m_{F^{\mathbf{A}}}(z)$.

$\frac{1}{n_r}\text{tr}(\mathbf{I} + \rho\mathbf{W}_r)^{-1} = \int \frac{1}{1 + \rho\lambda}\, dF^{\mathbf{W}_r}(\lambda)$ where $\mathbf{W}_r = \frac{1}{n_t}\mathbf{H}\mathbf{H}^H$.

As $n_r, n_t \to \infty$: $\text{MMSE} \to \int_{\lambda_-}^{\lambda_+} \frac{1}{1+\rho\lambda} f_{\mathrm{MP}}(\lambda;\beta)\, d\lambda = -m_{F_\beta}(-1/\rho)/\rho$.

1. Form $\boldsymbol{\Sigma}_{t}$ (Toeplitz). Set $\tilde{\mathbf{T}} = \frac{1}{n_t}\mathbf{I}$.
2. Iterate: $\mathbf{T} = (\mathbf{I} + \rho\, \boldsymbol{\Sigma}_{t} \tilde{T})^{-1}$, $\tilde{T} = \frac{1}{n_t(1 + \rho\, \frac{1}{n_t}\text{tr}(\mathbf{T}))}$.
3. Compute $\bar{V} = \frac{1}{n_r}\log\det(\mathbf{I} + \rho\tilde{T}\, \mathbf{I}_{n_r}) + \frac{1}{n_r}\frac{n_r}{n_t}\log\det(\mathbf{I} + \rho\, \boldsymbol{\Sigma}_{t}\tilde{T}) - \rho^2\tilde{T}\frac{1}{n_t}\text{tr}(\boldsymbol{\Sigma}_{t} \mathbf{T})$.

Capacity decreases with increasing $r$ (correlation reduces the effective degrees of freedom). At $r = 0$, the result matches the i.i.d. MP formula. At $r \to 1$, all transmit antennas are fully correlated — the rank-1 channel gives capacity $\approx \log_2(1 + n_t \rho)$.

Write $\mathbf{W} = \frac{1}{m}\sum_{i=1}^n \mathbf{h}_i \mathbf{h}_i^H$. Define $\mathbf{W}_{-i} = \mathbf{W} - \frac{1}{m}\mathbf{h}_i\mathbf{h}_i^H$ and $\mathbf{Q}_{-i} = (\mathbf{W}_{-i} - z\mathbf{I})^{-1}$.

$(\mathbf{W} - z\mathbf{I})^{-1} = \mathbf{Q}_{-i} - \frac{\frac{1}{m}\mathbf{Q}_{-i}\mathbf{h}_i\mathbf{h}_i^H\mathbf{Q}_{-i}}{1 + \frac{1}{m}\mathbf{h}_i^H\mathbf{Q}_{-i}\mathbf{h}_i}$. Taking the normalized trace and using $\frac{1}{m}\mathbf{h}_i^H\mathbf{Q}_{-i}\mathbf{h}_i \approx \frac{1}{m}\text{tr}\,\mathbf{Q}_{-i} \approx m(z)$ (by the LLN, since $\mathbf{h}_i$ is independent of $\mathbf{Q}_{-i}$).

Summing over $i = 1, \ldots, n$ and dividing by $m$: $m(z) \approx \frac{1}{m}\text{tr}\,\mathbf{Q}_{-1}(z) \cdot \frac{1}{1 + m(z)}$... after careful algebra, one arrives at $m(z) = \frac{-1}{z + \beta/(1 + m(z))} = \frac{1}{1 - \beta - z - \beta z m(z)}$.