Exercises

$C = \frac{1}{2}\log_2(1 + 5/51) = \frac{1}{2}\log_2(1.098) \approx 0.068$ bits.

$C = \frac{1}{2}\log_2(1 + 5/1) = \frac{1}{2}\log_2(6) \approx 1.29$ bits.

$C = \frac{1}{2}\log_2(1 + 5/1) \approx 1.29$ bits — same as DPC.

The gap between no CSI and DPC is enormous: $1.29 - 0.068 = 1.22$ bits. When $Q \gg P$, ignoring the state is catastrophic.

$\alpha^* = 10/(10+2) = 10/12 \approx 0.833$.

As $P \to \infty$: $\alpha^* \to 1$. The encoder fully correlates $U$ with $S$, effectively canceling the interference completely.

As $P \to 0$: $\alpha^* \to 0$. The encoder cannot afford to correlate with $S$ (the covering cost $I(U;S)$ would dominate the vanishing packing capacity), so it ignores the state.

$I(X;Y|S=0) = 1$ bit (noiseless channel, capacity 1). $I(X;Y|S=1) = 1 - \mathcal{H}_2(0.1) = 1 - 0.469 = 0.531$ bits (BSC).

$C = \max_{P_X} I(X;Y|S) = \frac{1}{2}(1 + 0.531) = 0.766$ bits.

Both are maximized by uniform $P_X(0) = P_X(1) = 1/2$.

If $U \perp S$: $I(U; S) = 0$, so $C = I(U; Y)$.

This is the capacity without any state knowledge at the encoder (the standard channel capacity with $S$ treated as additional noise). The Gel'fand-Pinsker formula generalizes the no-CSI result: allowing $U$ to depend on $S$ can only increase the rate.

$U = X + \alpha S$, $X \perp S$. $\text{Var}(U) = P + \alpha^2 Q$. $U|S = X + \alpha S|S \sim \mathcal{N}(\alpha S, P)$, so $\text{Var}(U|S) = P$.

$I(U; S) = \frac{1}{2}\log\frac{P + \alpha^2 Q}{P}$.

$Y = X + S + Z = (U - \alpha S) + S + Z = U + (1-\alpha)S + Z$.

Let $W = (1-\alpha)S + Z$, $\text{Var}(W) = (1-\alpha)^2 Q + N$. $\text{Cov}(U, W) = (1-\alpha)\alpha Q$. $\text{Var}(U|Y) = (P + \alpha^2 Q) - \frac{((1-\alpha)\alpha Q)^2 + 2(1-\alpha)\alpha Q P + P^2}{P + \alpha^2 Q + (1-\alpha)^2 Q + N}$.

After simplification: $I(U;Y) = \frac{1}{2}\log\frac{P + \alpha^2 Q + (1-\alpha)^2 Q + N}{(1-\alpha)^2 Q + N + P - \frac{P^2}{P+\alpha^2 Q}}$.

Actually, the cleaner approach: $Y = U + W$ where $\text{Cov}(U,W) = \alpha(1-\alpha)Q$. $I(U;Y) = \frac{1}{2}\log\frac{\text{Var}(U)\text{Var}(W) - \text{Cov}(U,W)^2}{\text{Var}(U)\text{Var}(W) - \text{Cov}(U,W)^2} \cdot \frac{\text{Var}(Y)}{\text{Var}(W)}$...

The simplest route: use $I(U;Y) = h(Y) - h(Y|U)$. $h(Y) = \frac{1}{2}\log(2\pi e(P + Q + N))$ (since $\text{Var}(Y) = P + Q + N$). $Y|U = (1-\alpha)S + Z + (U - \mathbb{E}[U|Y])$... This gets complicated.

At $\alpha^* = P/(P+N)$, a direct computation gives:

$I(U;Y) - I(U;S) = \frac{1}{2}\log(1 + P/N)$.

The key algebraic identity: with $\alpha^* = P/(P+N)$, $(1-\alpha^*)^2 Q + N = \frac{N^2}{(P+N)^2}Q + N = N\frac{N Q + (P+N)^2}{(P+N)^2}$ and $P + \alpha^{*2} Q = P + \frac{P^2 Q}{(P+N)^2}$. The $Q$-dependent terms cancel in the ratio, leaving $1 + P/N$.

The effective noise is $S \oplus Z \sim \text{Bern}(q * p)$ where $q * p = q(1-p) + (1-q)p$. $C = 1 - \mathcal{H}_2(q*p)$.

The decoder subtracts $S$: effective channel is BSC($p$). $C = 1 - \mathcal{H}_2(p)$.

Set $U = X \oplus S$ (the "clean" codeword). Then $Y = U \oplus Z$, so $I(U;Y) = 1 - \mathcal{H}_2(p)$. $I(U;S) = I(X \oplus S; S) = 0$ (since $X \perp S$ and $X \oplus S$ is independent of $S$ when $X$ is uniform).

$C = 1 - \mathcal{H}_2(p) - 0 = 1 - \mathcal{H}_2(p)$.

Same as decoder CSI! This is the binary analog of Costa's result: the additive state $S$ can be completely canceled.

$Y = (U - \alpha S) + S + Z = U + (1-\alpha)S + Z$. $U$ is independent of $(1-\alpha)S + Z$ (since $U \perp S, Z$). $I(U; Y) = \frac{1}{2}\log(1 + P_U/((1-\alpha)^2 Q + N))$ $= \frac{1}{2}\log(1 + (P - \alpha^2 Q)/((1-\alpha)^2 Q + N))$.

$C(\alpha) = \frac{1}{2}\log\frac{(1-\alpha^2) + (1-\alpha)^2 + 1}{(1-\alpha)^2 + 1} = \frac{1}{2}\log\frac{3 - 2\alpha}{2 - 2\alpha + \alpha^2}$.

Taking the derivative and setting to zero: $\alpha^* \approx 0.382$. $C(\alpha^*) \approx \frac{1}{2}\log(1.382) \approx 0.234$ bits.

Compare with DPC: $\frac{1}{2}\log(2) = 0.5$ bits. Causal CSI achieves only about half the DPC capacity in this example.

If $2^{nR'}$ codewords are drawn i.i.d. $\sim P_U^n$, and $S^n$ is drawn $\sim P_S^n$, the probability that at least one codeword is jointly typical with $S^n$ approaches 1 as $n \to \infty$ if and only if $R' > I(U; S)$.

The condition $R' > I(U; S)$ means we need enough codewords to "cover" the typical set of $S^n$. Since each codeword is jointly typical with $S^n$ with probability $\approx 2^{-nI(U;S)}$, we need $2^{nR'} \cdot 2^{-nI(U;S)} \gg 1$, i.e., $R' > I(U;S)$.

This covering cost represents the redundancy the encoder must spend to match the codeword to the state. It is subtracted from the packing capacity $I(U; Y)$ in the GP formula.

$I(U;S) = \frac{1}{2}\log(1 + \alpha^2 Q/P)$.

For $I(U;Y)$: $Y = U + (1-\alpha)S + Z$, and $\text{Cov}(U, (1-\alpha)S) = \alpha(1-\alpha)Q$.

I(U;Y) = \frac{1}{2}\log\frac{\text{Var}(U)\text{Var}(Y) - \text{Cov}(U,Y)^2 ... }

Actually, use $I(U;Y) = h(U) + h(Y) - h(U,Y)$ with the joint Gaussian entropy.

After careful computation, $f'(\alpha) = 0$ yields:

$\frac{Q[(1-\alpha)(P + \alpha^2 Q) - \alpha P]}{[(1-\alpha)^2 Q + N](P + \alpha^2 Q)\ln 2} = 0$.

The numerator: $(1-\alpha)(P + \alpha^2 Q) - \alpha P = P - 2\alpha P + \alpha^2 Q - \alpha^3 Q$. Setting to zero and solving (noting this factors): $\alpha(P+N) = P$, giving $\alpha^* = P/(P+N)$.

The optimal $\alpha^* = P/(P+N)$ depends only on $P$ and $N$, not on the interference power $Q$. This is remarkable: no matter how strong the interference, the same coding strategy is optimal. The MMSE coefficient $P/(P+N)$ for the interference-free channel is also the optimal DPC parameter.

$Y = X + S + Z$ with $\text{Cov}(S, Z) = \rho\sqrt{QN}$. $\text{Var}(Y|X) = Q + N + 2\rho\sqrt{QN}$.

$I(U; S) = \frac{1}{2}\log(1 + \alpha^2 Q/P)$ (unchanged, since $X \perp S$).

$Y = U + (1-\alpha)S + Z$. Now $(1-\alpha)S$ and $Z$ are correlated: $\text{Cov}((1-\alpha)S, Z) = (1-\alpha)\rho\sqrt{QN}$. $\text{Var}((1-\alpha)S + Z) = (1-\alpha)^2 Q + N + 2(1-\alpha)\rho\sqrt{QN}$.

The optimal $\alpha^*$ now depends on $\rho$, but the resulting capacity is: $C = \frac{1}{2}\log(1 + P/\text{Var}(Z|S))$.

Since $\text{Var}(Z|S) = N(1 - \rho^2)$... wait, this would change the capacity.

Actually, the correct analysis: the capacity is $C = \frac{1}{2}\log(1 + P/N)$ because the DPC encoder can also account for the correlation. The interference $S$ is known, and the effective noise is $Z$ with variance $N$ (the correlation with $S$ does not add noise once $S$ is known). The formal proof optimizes $\alpha$ over the generalized expressions.

Allocate power $\alpha P$ to user 2. User 2's signal $X_2$ is encoded with a standard Gaussian codebook.

User 2's rate: $R_{2} = \frac{1}{2}\log(1 + \alpha P/((1-\alpha)P + N_2))$ if user 2 treats user 1's signal as noise, or $R_{2} = \frac{1}{2}\log(1 + \alpha P/N_2)$ if user 1 is encoded via DPC and does not interfere.

The base station knows $X_2$ (it generated it). Encode user 1's message with power $(1-\alpha)P$ using DPC to code around $X_2$.

By Costa's theorem, user 1's rate: $R_{1} = \frac{1}{2}\log(1 + (1-\alpha)P/N_1)$.

The interference from $X_2$ is completely eliminated!

Varying $\alpha \in [0,1]$ traces a rate region that includes the corner points $(\frac{1}{2}\log(1+P/N_1), 0)$ and $(0, \frac{1}{2}\log(1+P/N_2))$. This matches the capacity region of the degraded Gaussian BC (Bergmans, 1973).

$nR = H(M) \leq I(M; Y^n) + n\epsilon_n$ $= \sum_{i=1}^n [I(M; Y_i | Y^{i-1}) ] + n\epsilon_n$.

$I(M; Y_i | Y^{i-1}) = I(M, S^{i-1}, S_{i+1}^n; Y_i | Y^{i-1}) - I(S^{i-1}, S_{i+1}^n; Y_i | Y^{i-1}, M)$.

The second term can be bounded using the Csisz'ar sum identity and the memoryless property.

Defining $U_i = (M, S^{i-1}, S_{i+1}^n, Y^{i-1})$ and using the memoryless property:

$I(U_i; Y_i | S_i) = I(M, Y^{i-1}, S^{i-1}, S_{i+1}^n; Y_i | S_i)$ and $I(U_i; S_i) = I(M, Y^{i-1}, S^{i-1}, S_{i+1}^n; S_i)$.

The Csisz'ar sum identity gives: $\sum_i I(U_i; S_i) = \sum_i I(U_i; S_i)$ (telescoping).

After careful bookkeeping: $nR \leq \sum_i [I(U_i; Y_i) - I(U_i; S_i)] + n\epsilon_n$. Standard time-sharing completes the converse.

$C_{\text{CSIR}} = \frac{1}{2}[\frac{1}{2}\log(1 + 1 \cdot 5) + \frac{1}{2}\log(1 + 4 \cdot 5)]$ $= \frac{1}{4}\log(6) + \frac{1}{4}\log(21) \approx 0.646 + 1.089 = 1.735$ bits.

Water-fill over the two states. Let $P_1, P_2$ be the powers. $P_1 + P_2 = 2P = 10$ (two channel uses on average). $\nu - 1/1 = P_1$, $\nu - 1/4 = P_2$. $2\nu - 1.25 = 10$, $\nu = 5.625$. $P_1 = 4.625$, $P_2 = 5.375$.

$C_{\text{CSIT}} = \frac{1}{2}[\frac{1}{2}\log(1+4.625) + \frac{1}{2}\log(1+4 \cdot 5.375)]$ $= \frac{1}{4}\log(5.625) + \frac{1}{4}\log(22.5) \approx 0.622 + 1.124 = 1.746$ bits.

With fixed power $P = 5$: $C_{\text{no CSI}} = \max_{P_X} \frac{1}{2}I(X; HX + Z)$. With Gaussian $X \sim \mathcal{N}(0, 5)$: $I(X; Y) < C_{\text{CSIR}}$ because the decoder cannot separate the signal from the fading without knowing $H$.

$C = \frac{1}{2}\log_2(1 + P/N) = \frac{1}{2}\log_2(2) = 0.5$ bits. This holds for any $Q \geq 0$.

$E_b/N_0 = \text{SNR}/\eta = 1/0.5 = 2$ (linear) = $10\log_{10}(2) \approx 3.01$ dB. This is $3.01 - (-1.59) = 4.6$ dB above the Shannon limit.

User 1 knows $X_2 = S$ non-causally. By Costa's theorem: $R_{1} = \frac{1}{2}\log(1 + P_1/N_1)$. The interference from user 2 is completely eliminated.

User 2's channel is $Y_2 = X_2 + X_1 + Z_{2}$. User 2 does not know $X_1$, so it treats it as noise: $R_{2} = \frac{1}{2}\log(1 + P_2/(P_1 + N_2))$.

The cognitive user achieves its interference-free rate, but the non-cognitive user suffers the full interference penalty. This asymmetry motivates cooperative or cognitive radio designs where one transmitter has knowledge of the other's signal.