Exercises

$H = -(0.5\log_2 0.5 + 0.25\log_2 0.25 + 0.125\log_2 0.125 + 0.125\log_2 0.125)$

$= -(0.5 \times (-1) + 0.25 \times (-2) + 0.125 \times (-3) + 0.125 \times (-3))$

$= -(- 0.5 - 0.5 - 0.375 - 0.375) = 1.75$ bits.

Note: $H = 1.75 < \log_2 4 = 2$ bits since $X$ is not uniformly distributed. $\blacksquare$

$H(0.1) = -0.1\log_2 0.1 - 0.9\log_2 0.9 = 0.332 + 0.137 = 0.469$ bits.

$C = 1 - 0.469 = 0.531$ bits/channel use.

$R_{\max} = C \times 10^6 = 0.531 \times 10^6 = 531$ kbps.

$C = 0.5$ requires $H(p) = 0.5$. Numerically, $p \approx 0.11$. $\blacksquare$

$\begin{bmatrix} 1 & 0 \\ q & 1-q \end{bmatrix}$$

$P(Y=0) = 1-\pi + \pi q = 1-\pi(1-q)$. $P(Y=1) = \pi(1-q)$.

$H(Y) = H(\pi(1-q))$ (binary entropy).

$H(Y|X) = \pi H(q)$.

$I = H(\pi(1-q)) - \pi H(q)$.

$I = H(0.5\pi) - \pi$ (since $H(0.5) = 1$).

Taking derivative and setting to zero: $\frac{dI}{d\pi} = 0.5 \log_2\frac{1-0.5\pi}{0.5\pi} - 1 = 0$

$\log_2\frac{1-0.5\pi}{0.5\pi} = 2$, so $\frac{1-0.5\pi}{0.5\pi} = 4$.

$1 = 2.5\pi$, $\pi^* = 0.4$.

$C = H(0.2) - 0.4 = 0.722 - 0.4 = 0.322$ bits. $\blacksquare$

By the chain rule: $I(X;Y,Z) = I + I(X;Z|Y)$

For the Markov chain $X \to Y \to Z$: $X$ and $Z$ are conditionally independent given $Y$, so $I(X;Z|Y) = 0$.

Therefore $I(X;Y,Z) = I$.

Also: $I(X;Y,Z) = I(X;Z) + I(X;Y|Z) \geq I(X;Z)$.

Combining: $I = I(X;Y,Z) \geq I(X;Z)$. $\blacksquare$

If $Z = f(Y)$, then knowing $Y$ determines $Z$, so $H(Z|Y) = 0$ and $I(Y;Z|X) = H(Z|X) - H(Z|X,Y) = H(Z|X) - H(Z|Y) \cdot ...$

More directly: $I(X;Y,Z) = I + I(X;Z|Y)$.

Since $Z = f(Y)$, $Z$ is determined by $Y$, so $H(Z|Y) = 0$ and $I(X;Z|Y) = H(Z|Y) - H(Z|X,Y) = 0$.

Therefore $I(X;Y,Z) = I$: knowing $Z$ in addition to $Y$ provides no extra information about $X$. $\blacksquare$

$\text{SNR} = 100$. $C = 200 \times 10^3 \times \log_2(101) = 200 \times 6.66 = 1.33$ Mbps.

$\text{SNR} = 10$. $C = 20 \times 10^6 \times \log_2(11) = 20 \times 3.46 = 69.2$ Mbps.

$\text{SNR} = 3.16$. $C = 100 \times 10^6 \times \log_2(4.16) = 100 \times 2.06 = 206$ Mbps. $\blacksquare$

$P_{\min} = 10^7 \times 10^{-20} \times 0.693 = 6.93 \times 10^{-14}$ W $= -131.6$ dBW.

$\eta = C/B = 2$. $\text{SNR} = 2^2 - 1 = 3$. $P = 3 \times 10^{-20} \times 5 \times 10^6 = 1.5 \times 10^{-13}$ W. Penalty: $10\log_{10}(1.5 \times 10^{-13}/6.93 \times 10^{-14}) = 3.35$ dB.

$\eta = 0.2$. $\text{SNR} = 2^{0.2} - 1 = 0.1487$. $P = 0.1487 \times 10^{-20} \times 5 \times 10^7 = 7.44 \times 10^{-14}$ W. Penalty: $10\log_{10}(7.44/6.93) = 0.31$ dB.

With 10x more bandwidth, the power penalty drops to only 0.31 dB above the infinite-bandwidth limit. $\blacksquare$

Let $u = P/(N_0 B)$. As $B \to \infty$, $u \to 0$:

$C = B\log_2(1+u) = \frac{P}{N_0 u}\log_2(1+u) = \frac{P}{N_0}\frac{\log_2(1+u)}{u}$

$\lim_{u \to 0} \frac{\log_2(1+u)}{u} = \frac{1}{\ln 2}$

$C \to \frac{P}{N_0 \ln 2}$.

$E_b = P/C$, so $E_b/N_0 = P/(C N_0)$.

At the infinite-bandwidth limit: $C = P/(N_0 \ln 2)$, so $E_b/N_0 = P/(P/(N_0 \ln 2) \cdot N_0) = \ln 2$.

In dB: $10\log_{10}(\ln 2) = 10\log_{10}(0.693) = -1.59$ dB.

$\text{SNR} = 2^\eta - 1$. For large $\eta$: $\text{SNR} \approx 2^\eta$.

Doubling $\eta$: $\text{SNR}' \approx 2^{2\eta} = (2^\eta)^2 \approx \text{SNR}^{2}$.

In dB: $\text{SNR}'_{\text{dB}} \approx 2 \times \text{SNR}_{\text{dB}}$.

More precisely, per unit increase in $\eta$: $\Delta\text{SNR} = 10\log_{10}(2) = 3.01$ dB. $\blacksquare$

$\bar{\gamma} = 1$ (0 dB): $C_{\text{AWGN}} = \log_2(2) = 1.00$ bits/s/Hz.

$\bar{\gamma} = 3.16$ (5 dB): $C_{\text{AWGN}} = \log_2(4.16) = 2.06$ bits/s/Hz.

$\bar{\gamma} = 10$ (10 dB): $C_{\text{AWGN}} = \log_2(11) = 3.46$ bits/s/Hz.

$\bar{\gamma} = 100$ (20 dB): $C_{\text{AWGN}} = \log_2(101) = 6.66$ bits/s/Hz.

Using $C_{\text{erg}} = e^{1/\bar{\gamma}} E_1(1/\bar{\gamma}) / \ln 2$:

$\bar{\gamma} = 1$: $C_{\text{erg}} \approx 0.61$ bits/s/Hz.

$\bar{\gamma} = 3.16$: $C_{\text{erg}} \approx 1.59$ bits/s/Hz.

$\bar{\gamma} = 10$: $C_{\text{erg}} \approx 2.94$ bits/s/Hz.

$\bar{\gamma} = 100$: $C_{\text{erg}} \approx 6.04$ bits/s/Hz.

The absolute loss grows slowly, but the relative loss decreases at high SNR. $\blacksquare$

$\bar{\gamma} = 10^{1.5} = 31.62$.

$\gamma_{0.01} = -31.62 \ln(0.99) = 31.62 \times 0.01005 = 0.318$.

$C_{0.01} = \log_2(1.318) = 0.399$ bits/s/Hz.

$\gamma_{0.10} = -31.62 \ln(0.90) = 31.62 \times 0.1054 = 3.33$.

$C_{0.10} = \log_2(4.33) = 2.11$ bits/s/Hz.

$C_{0.01} = 4$ requires $\gamma_{0.01} = 2^4 - 1 = 15$.

$\gamma_{0.01} = -\bar{\gamma}\ln(0.99) = 0.01005\bar{\gamma} = 15$.

$\bar{\gamma} = 15/0.01005 = 1493 = 31.7$ dB.

A very high average SNR is needed because the 1%-outage must support the rate even during deep fades. $\blacksquare$

$1/|h_k|^2$: $\{10, 0.5, 2.0, 0.667\}$.

Total power $= 4\bar{P} = 8$. Try all 4 active:

$4\mu - (10 + 0.5 + 2.0 + 0.667) = 8$. $4\mu = 21.167$, $\mu = 5.292$.

$P_1 = 5.292 - 10 = -4.71 < 0$. Turn off slot 1.

3 active: $3\mu - (0.5 + 2.0 + 0.667) = 8$. $3\mu = 11.167$, $\mu = 3.722$.

$P_1 = 0$, $P_2 = 3.222$, $P_3 = 1.722$, $P_4 = 3.056$. Sum $= 8.0$. Check.

$C = \frac{1}{4}\sum_k \log_2(1 + |h_k|^2 P_k)$

$= \frac{1}{4}[0 + \log_2(1+2\times3.222) + \log_2(1+0.5\times1.722) + \log_2(1+1.5\times3.056)]$

$= \frac{1}{4}[0 + 2.94 + 0.87 + 2.48] = 1.57$ bits/channel use.

$C_{\text{const}} = \frac{1}{4}\sum \log_2(1 + |h_k|^2 \times 2)$

$= \frac{1}{4}[\log_2(1.2) + \log_2(5) + \log_2(2) + \log_2(4)]$

$= \frac{1}{4}[0.26 + 2.32 + 1.0 + 2.0] = 1.40$ bits/channel use.

Water-filling gain: $1.57 - 1.40 = 0.17$ bits (12%). Transmitter is silent 25% of the time (slot 1). $\blacksquare$

$N_0/|H_k|^2$: $\{0.25, 0.5, 1.0, 2.0, 10.0, 2.0, 0.5, 0.333\}$.

Try all 8 active: $8\mu - 16.583 = 8$, $\mu = 3.073$. $P_5 = 3.073 - 10 < 0$. Remove sub-carrier 5.

7 active: $7\mu - 6.583 = 8$, $\mu = 2.083$. $P_4 = 2.083 - 2.0 = 0.083 > 0$. All remaining are positive.

$P = \{1.833, 1.583, 1.083, 0.083, 0, 0.083, 1.583, 1.750\}$.

$C = \sum_{k \neq 5} \log_2(1 + |H_k|^2 P_k)$

$= \log_2(8.33) + \log_2(4.17) + \log_2(2.08) + \log_2(1.04) + 0 + \log_2(1.04) + \log_2(4.17) + \log_2(6.25)$

$= 3.06 + 2.06 + 1.06 + 0.06 + 0 + 0.06 + 2.06 + 2.64 = 11.0$ bits/channel use.

One sub-carrier (12.5%) is turned off. $\blacksquare$

At high SNR, $P_k \gg N_0/g_k$ for all active sub-channels.

Water-filling: $P_k = \mu - N_0/g_k \approx \mu$ for all $k$. $\mu \approx P/K$ (equal allocation at high SNR!).

$C_{\text{WF}} \approx \sum_k \log_2(g_k P/K) = K\log_2(P/K) + \sum_k \log_2(g_k)$.

Equal power: $P_k = P/K$ for all $k$. $C_{\text{EP}} \approx \sum_k \log_2(g_k P/K) = K\log_2(P/K) + \sum_k \log_2(g_k)$.

They are identical at high SNR.

At high SNR, the water-filling gain is negligible, and equal power allocation across OFDM sub-carriers is near-optimal. This is why 5G NR does not mandate per-sub-carrier power adaptation but focuses instead on sub-band MCS adaptation. Water-filling matters most at low-to-moderate SNR where some sub-carriers are in deep fades. $\blacksquare$

$|h_1|^2 = 0.64 + 0.36 = 1.0$. $|h_2|^2 = 0.09 + 0.16 = 0.25$. $\|\mathbf{h}\|^2 = 1.25$.

$C = \log_2(1 + 1.25 \times 10) = \log_2(13.5) = 3.75$ bits/s/Hz.

With antenna 1 only: $C_1 = \log_2(1 + 1.0 \times 10) = \log_2(11) = 3.46$.

Array gain in SNR: $1.25/1.0 = 1.25 = 0.97$ dB.

For this particular realisation, antenna 2 contributes 0.29 bits/s/Hz additional capacity. $\blacksquare$

Total SNR $= 4 \times 10 = 40$ (16 dB). Per-antenna allocation: $\text{SNR}/N_t = 10$.

$C_{\text{no CSIT}} = E[\log_2(1 + \|\mathbf{h}\|^2 \times 10)]$

With $\|\mathbf{h}\|^2 \sim \chi^2_8/2$ (mean 4): $C_{\text{no CSIT}} \approx 5.1$ bits/s/Hz (numerical).

$C_{\text{CSIT}} = E[\log_2(1 + \|\mathbf{h}\|^2 \times 40)]$

$\approx 7.0$ bits/s/Hz (numerical).

$\Delta C = 7.0 - 5.1 = 1.9$ bits/s/Hz.

In SNR terms, beamforming provides array gain $N_t = 4 = 6$ dB. This translates to approximately $6/3 = 2$ bits/s/Hz at high SNR, consistent with the computed difference. $\blacksquare$

$\gamma_{\text{eff}} \sim \text{Gamma}(N_r, \bar{\gamma})$.

The 1% quantile of $\text{Gamma}(N_r, \bar{\gamma})$:

$N_r = 1$: $\gamma_{0.01} = -10\ln(0.99) = 0.1005$. $C_{0.01} = \log_2(1.1) = 0.137$ bits/s/Hz.

$N_r = 2$: $\gamma_{0.01} \approx 0.71$ (from Gamma CDF). $C_{0.01} = \log_2(1.71) = 0.774$ bits/s/Hz.

$N_r = 4$: $\gamma_{0.01} \approx 2.53$. $C_{0.01} = \log_2(3.53) = 1.82$ bits/s/Hz.

$N_r = 1$: $C_{\text{erg}} \approx 2.7$ bits/s/Hz.

$N_r = 2$: $C_{\text{erg}} \approx 4.0$ bits/s/Hz.

$N_r = 4$: $C_{\text{erg}} \approx 5.1$ bits/s/Hz.

The outage capacity improvement from $N_r = 1$ to $4$ is $1.82/0.137 = 13.3\times$, while the ergodic improvement is $5.1/2.7 = 1.9\times$.

Diversity gain primarily benefits the tail of the SNR distribution (deep fades), which directly affects outage capacity. Ergodic capacity averages over all fading states, so the tail improvement has less impact on the mean. $\blacksquare$

$(E_b/N_0)_{\text{Shannon}} = (2^2-1)/2 = 1.5 = 1.76$ dB. $\Gamma = 9.6 - 1.76 = 7.84$ dB.

$\eta = 1$: $(E_b/N_0)_{\text{Shannon}} = (2^1-1)/1 = 1 = 0$ dB. $\Gamma = 1.8 - 0 = 1.8$ dB.

$\Gamma = 2.1 - 0 = 2.1$ dB.

Summary: LDPC is 0.3 dB closer to capacity than polar in this scenario, but both are within 2.1 dB of the Shannon limit. $\blacksquare$

$\bar{\gamma} = 31.62$ (linear). For Rayleigh: $P(\gamma > \gamma_0) = e^{-\gamma_0/\bar{\gamma}}$.

$P(\gamma < -1 \text{ dB}) = P(\gamma < 0.794) = 1 - e^{-0.794/31.62} = 0.025$ $P(-1 \leq \gamma < 2) = e^{-0.794/31.62} - e^{-1.585/31.62} = 0.975 - 0.951 = 0.024$ $P(2 \leq \gamma < 5) = e^{-1.585/31.62} - e^{-3.162/31.62} = 0.951 - 0.905 = 0.046$ $P(5 \leq \gamma < 9) = 0.905 - 0.798 = 0.107$ $P(9 \leq \gamma < 13) = 0.798 - 0.611 = 0.187$ $P(13 \leq \gamma < 19) = 0.611 - 0.335 = 0.276$ $P(\gamma \geq 19) = 0.335$

$\bar{\eta} = 0 \times 0.025 + 0.67 \times 0.024 + 1.0 \times 0.046 + 1.5 \times 0.107 + 2.0 \times 0.187 + 3.0 \times 0.276 + 4.5 \times 0.335$

$= 0 + 0.016 + 0.046 + 0.161 + 0.374 + 0.828 + 1.508 = 2.93$ bits/s/Hz.

Compare with ergodic capacity $C_{\text{erg}} \approx 4.5$ bits/s/Hz. AMC efficiency: $2.93/4.5 = 65$%.

The gap is due to the coarse MCS table and the outage at low SNR. $\blacksquare$

Decompose: $h = \hat{h} + e$. The received signal is

$y = hx + w = \hat{h}x + ex + w$

The receiver treats $\hat{h}$ as the true channel. The "interference" $ex$ has power $\sigma_e^2 P$ and the noise is $N_0$.

$\text{SINR} = \frac{|\hat{h}|^2 P}{(\sigma_e^2 P + N_0)} = \frac{(|h|^2 - \sigma_e^2) \text{SNR}}{1 + \sigma_e^2 \text{SNR}}$

(approximately, using $E[|\hat{h}|^2] = E[|h|^2] - \sigma_e^2$).

$C_{\text{lower}} = E[\log_2(1 + \text{SINR})]$.

Perfect CSI: $C_{\text{perf}} \approx 6.0$ bits/s/Hz.

With $\sigma_e^2 = 0.01$: The effective SINR at the mean ($\gamma = 100$) is $(1-0.01) \times 100 / (1 + 0.01 \times 100) = 99/2 = 49.5$.

$C_{\text{imperfect}} \approx 5.4$ bits/s/Hz.

Loss $\approx 0.6$ bits/s/Hz.

At high SNR, the SINR saturates at $(1-\sigma_e^2)/\sigma_e^2$. The capacity ceiling is $\log_2(1 + (1-\sigma_e^2)/\sigma_e^2) = \log_2(1/\sigma_e^2)$.

For 1 bit loss at $\bar{\gamma} = 20$ dB ($C_{\text{perf}} = 6.0$): Need $C_{\text{imperfect}} \leq 5.0$, so $\log_2(1/\sigma_e^2) \approx 5$, $\sigma_e^2 \approx 1/32 = 0.031$. $\blacksquare$

$B_c = 1/(5 \times 10^{-6}) = 200$ kHz.

Sub-carrier spacing: $\Delta f = B/K = 10/64 = 156.25$ kHz.

Sub-carriers per coherence bandwidth: $B_c/\Delta f \approx 1.28$.

Number of independent groups: $B/B_c = 10\text{ MHz}/200\text{ kHz} = 50$.

So the 64 sub-carriers have approximately 50 independent fading realisations — high frequency selectivity.

With 50 independent fading groups and Rayleigh fading at $\bar{\gamma} = 15$ dB (31.62 linear):

Equal power: $C_{\text{EP}} \approx K \times E[\log_2(1 + |H_k|^2 \bar{\gamma})] \approx 64 \times 4.5 / (64/10) \approx 4.5$ bits/s/Hz per sub-carrier. Total: $C_{\text{EP}} = B \times 4.5 = 45$ Mbps.

Water-filling: at moderate SNR (15 dB), the water-filling gain over equal power is typically 5-10% for Rayleigh fading.

$C_{\text{WF}} \approx 1.07 \times C_{\text{EP}} = 48$ Mbps.

The gain is modest because at 15 dB SNR, most sub-carriers are well above the water-filling cutoff. $\blacksquare$

$B = 20$ MHz $\gg B_c = 500$ kHz. Highly frequency-selective. Number of independent frequency blocks: $B/B_c = 40$.

$T_{\text{slot}} = 0.5$ ms $< T_c = 5$ ms. Within one slot, the channel is approximately constant. Over 10 slots ($= T_c$), the channel changes independently.

Within frequency: 40 independent blocks per slot provide frequency diversity. Over time: need many coherence times.

For ergodic capacity, we need many independent fading realisations. Over 1 second: 200 coherence times in time $\times$ 40 in frequency $= 8000$ independent samples. Ergodic capacity is achievable with codewords spanning $\sim$100 ms.

$\bar{\gamma} = 10^{1.2} = 15.85$ (linear).

With $N_r = 4$ and MRC: effective $\bar{\gamma}_{\text{eff}} = 4 \times 15.85 = 63.4$ (18 dB).

$C_{\text{erg}} = E[\log_2(1 + \|\mathbf{h}\|^2 \times 15.85)]$

With $\|\mathbf{h}\|^2 \sim \text{Gamma}(4,1)$ (mean 4): $C_{\text{erg}} \approx 5.8$ bits/s/Hz.

Total: $5.8 \times 20 = 116$ Mbps.

$\gamma_{\text{eff}} = \|\mathbf{h}\|^2 \times 15.85$ where $\|\mathbf{h}\|^2 \sim \text{Gamma}(4,1)$.

1% quantile of Gamma(4,1): approximately 0.74.

$\gamma_{0.01} = 0.74 \times 15.85 = 11.73$.

$C_{0.01} = \log_2(12.73) = 3.67$ bits/s/Hz.

With frequency diversity (40 independent blocks), the effective outage capacity per slot is closer to the ergodic capacity: $C_{0.01}^{\text{freq}} \approx 5.0$ bits/s/Hz.

Total: $5.0 \times 20 = 100$ Mbps.

With gap $\Gamma = 2$ dB, the effective capacity is approximately

$C_{\text{AMC}} \approx E[\log_2(1 + \|\mathbf{h}\|^2 \bar{\gamma} / \Gamma)]$

$= E[\log_2(1 + \|\mathbf{h}\|^2 \times 15.85/1.585)]$

$= E[\log_2(1 + \|\mathbf{h}\|^2 \times 10)]$