Exercises

$L_E = L_D - L_A - L_{ch} = 4.2 - 1.0 - 1.8 = 1.4$.

The peer decoder already knows $L_{ch}$ for the systematic bit and supplied $L_A$ itself. Reusing either amounts to positive feedback: the same evidence is counted twice, LLR magnitudes blow up, and the decoder becomes overconfident about wrong decisions. Only $L_E$ represents new information learned through the code constraints.

Systematic: $K$. Parity 1: $K$. Parity 2 punctured to $K/2$. Total: $K + K + K/2 = 5K/2$.

Rate $R = K / (5K/2) = 2/5$. Systematic fraction is $K / (5K/2) = 2/5$ of transmitted bits.

$L | X = \pm 1 \sim \mathcal{N}(\pm \sigma^2/2, \sigma^2)$. As $\sigma \to 0$ both distributions concentrate at $0$, so $L$ carries no information about $X$: $I(X; L) = 0$.

The two conditional distributions separate (means $\pm \sigma^2/2$ grow faster than the standard deviation $\sigma$), so $L$ determines $X$: $I(X; L) = H(X) = 1$ bit.

$J(\sigma) = 0$ means a-priori LLRs are useless; the SISO block decodes from channel data alone. $J(\sigma) = 1$ means a-priori LLRs are perfect; nothing new can be learned and the iteration terminates.

A fixed point of the iteration satisfies $I = T(T(I))$, i.e., $T(I) = T^{-1}(I)$. Graphically, these are crossings of $T$ with its mirror. Between two crossings the trajectory monotonically moves toward the upper one and is trapped below the upper fixed point.

The curves cross at some $I^\star < 1$. The trajectory climbs stairwise from $(0,0)$, squeezes into the narrowing tunnel, and stalls at $I^\star$. Residual BER is determined by $I^\star$ and is nonzero.

At the cliff the two curves touch tangentially at $I^\star$: an extremely slow passage through the pinch point, so convergence takes many iterations. Just above, the tunnel is open and the trajectory reaches $I = 1$ within a handful of iterations, producing the steep BER drop known as the waterfall.

For a rate-1/2 RSC, $L_D[k] \approx L_A[k] + L_{ch}^{\text{sys}}[k] + L_E[k]$, where $L_E[k]$ is the information extracted from the parity constraints via forward-backward propagation.

The peer decoder (via the interleaver) already supplied $L_A[k]$ and will directly use $L_{ch}^{\text{sys}}[k]$ for its own systematic bit. Hence the extrinsic output is $L_E[k] = L_D[k] - L_A[k] - L_{ch}^{\text{sys}}[k]$.

$L_{ch}^{(1)}$ (parity of encoder 1) is information that only this decoder owns; the peer cannot double-count it. Hence only the systematic channel LLR is subtracted.

$\bar{x} = (+1)\sigma(L) + (-1)(1 - \sigma(L)) = 2\sigma(L) - 1 = \tanh(L/2)$.

Since $x^2 = 1$ deterministically, $\text{Var}(x|L) = \mathbb{E}[x^2|L] - \bar{x}^2 = 1 - \tanh^2(L/2)$.

$L = 0$ gives $\bar{x} = 0, v = 1$ (no decoder knowledge). $|L| \to \infty$ gives $\bar{x} \to \pm 1, v \to 0$ (hard decision).

Form $\tilde{y}_k = y_k - 0.6\bar{x}_{k-1}$. The residual model is $\tilde{y}_k = x_k + 0.6(x_{k-1} - \bar{x}_{k-1}) + w_k$ whose interference term has mean zero and variance $0.36 v_{k-1}$.

For a scalar estimator $\hat{x}_k = g \tilde{y}_k$, the LMMSE gain is $g = \frac{\text{Cov}(x_k, \tilde{y}_k)}{\text{Var}(\tilde{y}_k)} = \frac{1}{1 + 0.36 v_{k-1} + \sigma^2}.$

The output SINR is $\text{SINR} = \frac{1}{0.36 v_{k-1} + \sigma^2}.$ As decoder feedback sharpens ($v_{k-1} \to 0$) the SINR grows toward $1/\sigma^2$ — the matched-filter bound.

For two repetitions of bit $x$ producing channel LLRs $L_{ch,1}, L_{ch,2}$, the extrinsic LLR on the first copy is $L_E = L_A + L_{ch,2}$.

Under symmetric-Gaussian consistency, variance is additive: $\sigma_E^2 = \sigma_A^2 + \sigma_{ch}^2$ where $\sigma_A = J^{-1}(I_A)$ and $\sigma_{ch} = J^{-1}(I_{ch})$. For the bound from both repetitions $\sigma_E^2 = \sigma_A^2 + 2\sigma_{ch}^2$.

$I_E = J(\sigma_E) = J\!\left(\sqrt{J^{-1}(I_A)^2 + 2 J^{-1}(I_{ch})^2}\right).$ Repetition codes have strictly increasing EXIT curves.

$I_E^{\text{eq}} = 0.3 + 0.6(0) = 0.30$. Then $I_E^{\text{dec}} = 0.2 + 1.4(0.30) - 0.6(0.30)^2 = 0.2 + 0.42 - 0.054 = 0.566$.

$I_E^{\text{eq}} = 0.3 + 0.6(0.566) = 0.640$. Then $I_E^{\text{dec}} = 0.2 + 1.4(0.640) - 0.6(0.640)^2 = 0.2 + 0.896 - 0.246 = 0.850$.

$I_E^{\text{eq}} = 0.3 + 0.6(0.850) = 0.810$. Then $I_E^{\text{dec}} = 0.2 + 1.4(0.810) - 0.6(0.810)^2 = 0.2 + 1.134 - 0.394 = 0.940$. Trajectory monotonically approaches 1. The tunnel is open and the loop converges.

With $v_k = 0$ for all interfering symbols, the soft-IC residual is $\tilde{y}_k = x_k + w_k$: a genuine AWGN scalar observation for the current symbol. The ISI taps have been perfectly subtracted.

The LMMSE filter reduces to the identity and passes per-symbol LLRs of AWGN quality to the decoder.

The decoder now operates as if on AWGN without ISI, so its BER is the AWGN matched-filter bound of the outer code, independent of the ISI response (assuming nonzero energy).

The matched-filter output $\mathbf{h}_1^H \mathbf{y}$ has signal power $\|\mathbf{h}_1\|^4 \approx N_r^2$ (deterministic limit) and noise power $\|\mathbf{h}_1\|^2 \sigma^2 \approx N_r \sigma^2$.

Each of the $N_t - 1$ cancelled interferers contributes residual variance $|\mathbf{h}_1^H \mathbf{h}_j|^2 \cdot v$ which on average equals $\|\mathbf{h}_1\|^2 \cdot v \approx N_r v$ per interferer. Total interference power $\approx (N_t - 1) N_r v$.

$\text{SINR}_1 = \frac{N_r^2}{N_r \sigma^2 + (N_t - 1) N_r v} = \frac{N_r}{\sigma^2 + (N_t - 1) v}.$ $v = 1$ (no feedback): $\text{SINR} = N_r / (\sigma^2 + N_t - 1)$ — the naive MF SINR. $v = 0$ (perfect feedback): $\text{SINR} = N_r / \sigma^2$ — the single-user MFB.

$\Pr(x = +1 | y) = \frac{e^{L/2}}{e^{L/2} + e^{-L/2}} = \sigma(L)$.

$m = \mathbb{E}[x] = \tanh(L/2)$.

$s^2 = \text{Var}(x) = 1 - \tanh^2(L/2) = \text{sech}^2(L/2)$. The EP Gaussian has these moments; its natural parameters are then passed as an effective Gaussian prior to neighbouring factors.

LMMSE-PIC always uses $\mathcal{N}(0,1)$ for the residual symbol; this ignores decoder feedback about specific constellation points. EP's $\mathcal{N}(m, s^2)$ incorporates the posterior discrete constellation marginals and tightens with iteration.

As feedback sharpens, EP's $s^2$ shrinks toward zero and $m$ concentrates on the true symbol; LMMSE variance stays fixed at $1$. EP's filter is thus closer to matched filtering earlier.

High-order QAM with moderate-to-large MIMO dimensions: LMMSE-PIC wastes prior information because $\mathcal{N}(0,1)$ is far from the actual discrete QAM posterior, while EP tracks the full posterior mean and variance.

The cavity $q^{\setminus a}$ represents what the other factors collectively believe about $x$ without factor $a$'s contribution. Tilting by $f_a$ then incorporates $a$'s exact constraint once.

Without division by $\tilde{f}_a$, the tilted distribution contains $\tilde{f}_a \cdot f_a$, which double-counts factor $a$. The new message includes a copy of the previous message — the EP analogue of feeding the a-posteriori LLR back into the peer decoder.

The cavity is exactly the extrinsic information of the turbo principle: the belief minus the contribution of the factor that will consume the new message. Both constructs enforce that only new evidence is propagated.

EP seeks moment-matching fixed points of a nonlinear map. In ill-conditioned regimes (e.g., strongly non-Gaussian posteriors) consecutive proposals can leapfrog the fixed point, producing oscillation.

Damping mixes the new proposal with the old, reducing the effective update magnitude. This is analogous to gradient descent with a learning-rate $\alpha$ on a log-scale.

Small $\alpha$ (e.g., $0.3$) is robust but slow; $\alpha$ close to $1$ is fast but risks oscillation. Adaptive damping based on change-in-belief is common in practice.

A long random interleaver spreads low-weight input patterns across the two component codes; the joint codeword distance is much larger than either component's minimum distance. Short interleavers produce many low-weight codewords, causing high error floors.

EXIT analysis assumes i.i.d. symmetric-Gaussian extrinsic LLRs. This holds asymptotically as $K \to \infty$. At $K = 100$ the Gaussian approximation fails and the waterfall threshold is much worse than predicted.

Long interleavers lower both the waterfall (convergence threshold) and the error floor. The 5G-NR data channel uses LDPC codes with effective block sizes in the thousands for precisely this reason.

Under the BEC or the Gaussian approximation, a code of rate $R$ has decoder EXIT area $A = 1 - R$ (for the EXIT under channel capacity matching).

For the tunnel to stay open, the outer decoder curve (when mirrored) must fit beneath the inner curve. This requires the outer area to satisfy $1 - R_{\text{outer}} \leq A_{\text{in}}$, i.e., $R_{\text{outer}} \geq 1 - A_{\text{in}}$.

The inner EXIT area determines the maximal achievable outer rate. Matching $A_{\text{in}}$ to $1 - R_{\text{outer}}$ by curve shaping (e.g., irregular LDPC degree distributions) approaches channel capacity.

$\tilde{y}_k = x_k + w_k$, since $0.8 \bar{x}_{k-1} = 0.8 x_{k-1}$ exactly. The equalizer output equals the matched filter output on AWGN with unit energy.

$\text{SINR} = 1/\sigma^2 = 1/0.25 = 4$ (or $6$ dB).

This equals the MFB of unit-energy AWGN. The ISI tap at $0.8$ is irrelevant under perfect feedback — it contributes no interference and no energy loss. The additional tap energy could be used if matched filtering across the ISI span were employed (combining tap $0.8$'s contribution from the next sample), but soft-IC without matched filtering discards it.