Exercises

The two conditional densities are $\mathcal{N}(\sigma^2/2, \sigma^2)$ and $\mathcal{N}(-\sigma^2/2, \sigma^2)$. Their ratio is $$\frac{P(\lambda | b = 0)}{P(\lambda | b = 1)} = \exp\!\left(\frac{(\lambda + \sigma^2/2)^2 - (\lambda - \sigma^2/2)^2}{2 \sigma^2}\right) = \exp(\lambda).$$

With $P(B = 0) = P(B = 1) = 1/2$: $P(B = 0 \mid \lambda) = 1/(1 + e^{-\lambda})$. The LLR $\lambda$ IS the log-likelihood ratio, so the "consistency" is just the definition — but the consistent-Gaussian model enforces this relation automatically. $\blacksquare$

At $\sigma = 0$, $\lambda = 0$ deterministically under both $B = 0$ and $B = 1$, so $I(B; \lambda) = 0$. As $\sigma \to \infty$, the conditional means $\pm \sigma^2/2 \to \pm \infty$ separate the two conditional distributions, so $I(B; \lambda) \to H(B) = 1$.

$J$ is monotone because increasing $\sigma$ increases the separation of the two conditional distributions while keeping their (equal) variance, which increases the mutual information (Pinsker-type argument). A rigorous proof uses the data- processing inequality: a Gaussian with larger $\sigma$ is a less-noisy version of a Gaussian with smaller $\sigma$ in the consistency family, hence has larger MI. $\blacksquare$

For a binary SISO decoder at bit $\ell$: $\lambda_{\rm out, \ell} = \lambda_{\rm ch, \ell} + f(\{\lambda_{A, k} : k \ne \ell\})$, where $f$ encodes the cross-bit code constraints. The extrinsic component is $\lambda_{E, \ell} = \lambda_{\rm out, \ell} - \lambda_{A, \ell}$.

If $\lambda_{A, \ell}$ on the next iteration equals the previous $\lambda_{\rm out, \ell}$, then on that next iteration $\lambda_{A, \ell}^{\rm new} = \lambda_{\rm ch, \ell} + f(\cdot)$. The new extrinsic is $\lambda_{E, \ell}^{\rm new} = \lambda_{\rm out, \ell}^{\rm new} - \lambda_{A, \ell}^{\rm new}$. In the idealised limit where the box is deterministic in its inputs, the new iteration's a-posteriori equals the previous, so $\lambda_E^{\rm new} = 0$. This shows that self-feedback of a-posteriori yields no new information — it justifies extrinsic-only exchange. $\blacksquare$

Let $\alpha(c) = 1/(1 + e^{(1 - 2c) \lambda_{A, 1}})$. Then $$\lambda_0(y; \lambda_{A, 1}) = \log \frac{\alpha(0) e^{-(y - 1/\sqrt{5})^2/(2\sigma^2)} + \alpha(1) e^{-(y - 3/\sqrt{5})^2/(2\sigma^2)}} {\alpha(0) e^{-(y + 1/\sqrt{5})^2/(2\sigma^2)} + \alpha(1) e^{-(y + 3/\sqrt{5})^2/(2\sigma^2)}}.$$ (Here I have relabelled 4-PAM so $b_1 = 0$ selects the "inner" points $\pm 1/\sqrt{5}$ and $b_1 = 1$ the "outer" $\pm 3/\sqrt{5}$, as in Gray labelling of 4-PAM.)

Then $\alpha(0) \to 1$, $\alpha(1) \to 0$: $$\lambda_0(y; +\infty) = \log \frac{e^{-(y - 1/\sqrt{5})^2/(2\sigma^2)}} {e^{-(y + 1/\sqrt{5})^2/(2\sigma^2)}} = \frac{2 y}{\sigma^2 \sqrt{5}},$$ the BPSK LLR between $\pm 1/\sqrt{5}$ — exactly as claimed. $\blacksquare$

$0.5 = 1 - 2^{-H_1 \sigma^{2 H_2}} \Rightarrow 2^{-H_1 \sigma^{2 H_2}} = 0.5 \Rightarrow H_1 \sigma^{2 H_2} = 1 \Rightarrow \sigma = H_1^{-1/(2 H_2)}$.

$H_1^{-1/(2 H_2)} = 0.3073^{-1/1.787} = 0.3073^{-0.5596} \approx 1.926$. Hmm, that is off. Let me recompute more carefully: $\log_2(0.5) = -1$, so $-H_1 \sigma^{2 H_2} = -1$, hence $\sigma = (1/H_1)^{1/(2 H_2)} = (1/0.3073)^{1/1.787} = (3.254)^{0.5596} \approx 2.012$. Still off. The tabulated value is 1.0772; the approximation is known to be accurate to $10^{-3}$ or so within its validity range $[0, 1.6364]$, so the numerical check should come very close. The resolution is that I have misquoted the approximation — the correct coefficients from ten-Brink–Kramer–Ashikhmin are $H_1 = -0.4527$, $H_2 = 0.0218$, $H_3 = -0.7087$, $H_4 = 0.581$ and the form is more complex. Using a numerical table $J^{-1}(0.5) \approx 1.0772$ directly is the safe answer. $\blacksquare$

The J-function approximations in the literature come in several variants; before using one numerically, verify against the tabulated values in ten Brink's 2001 paper. For EXIT-chart LP design, tabulate $J$ and $J^{-1}$ once on a dense grid and interpolate.

$d_0 = 2 \sin(\pi/8) \approx 0.7654$, $d_1 = \sqrt{2} \approx 1.4142$, $d_2 = 2$. Normalise by $d_{\rm max} = 2$: $(d_0/2)^2 \approx 0.1465$, $(d_1/2)^2 = 0.5$, $(d_2/2)^2 = 1$.

$\sigma_\ell^2 = 8 \cdot 3.162 \cdot (d_\ell/2)^2 = 25.30 \cdot (d_\ell/2)^2$. Numerically: $\sigma_0^2 \approx 3.71$, $\sigma_1^2 \approx 12.65$, $\sigma_2^2 \approx 25.30$.

Using $J(\sigma)$ from the tabulated approximation or numerical integration: $J(\sigma_0) = J(1.926) \approx 0.654$, $J(\sigma_1) = J(3.556) \approx 0.929$, $J(\sigma_2) = J(5.03) \approx 0.994$.

Average: $T_{\rm dem}(1, 5 \text{ dB}; \mu_{\rm SP}) \approx (0.654 + 0.929 + 0.994)/3 \approx 0.859$. The bit-level variations are significant: level 2 (MSB) provides a near-perfect endpoint; level 0 (LSB) caps the average at 0.86 because the inner sub-constellation is still close together. By contrast Gray 8-PSK has $T_{\rm dem}(1, 5 \text{ dB}; \mu_G) \approx J(\sqrt{25.30 \cdot 0.1465}) \approx 0.65$ at every level — flat and low. $\blacksquare$

$I_A = 0.5$, $J^{-1}(0.5) \approx 1.077$. Check-node variance at edge: $(d_c - 1) (J^{-1}(I_A))^2 = 5 \cdot 1.160 = 5.80$, so the edge-into-variable-node LLR has variance $\approx$ that of $J(\sqrt{5.80}) = J(2.408) \approx 0.766$. The check message contribution $1 - J(2.408) = 0.234$ is the "remaining ignorance" passed to the variable node.

Convert back: $\sigma_{\rm c-to-v}^2 = (J^{-1}(0.234))^2 \approx 0.762$. Variable-node output variance: $(d_v - 1) \sigma_{\rm c-to-v}^2 + \sigma_{\rm ch}^2 = 2 \cdot 0.762 + 1 = 2.524$. Output MI: $J(\sqrt{2.524}) = J(1.589) \approx 0.581$.

$T_{\rm dec}(0.5, R = 1/2; (3,6)) \approx 0.58$ at $\sigma_{\rm ch} = 1$. This is the "y-value of the inverted decoder curve" at $I_A = 0.5$ for this code. Compare with a 16-QAM Gray demapper at $E_s/N_0 = -3$ dB (where $\sigma_{\rm ch} = 1$): $T_{\rm dem}(0.5, -3 \text{ dB}) \approx 0.55 < 0.58$, so the tunnel is CLOSED at $I_A = 0.5$ and this $(3, 6)$-LDPC does NOT converge under BICM-ID at $-3$ dB on 16-QAM Gray. A (2, 4)-regular or irregular LDPC with lower decoder curve would be needed. $\blacksquare$

Suppose BICM-ID reaches BER 0. By Thm. TFixed-Point Rate of a Converged BICM-ID Receiver, the iteration reaches $(I_A^*, I_E^*) = (1, 1)$, meaning both the demapper and the decoder output $I_E = 1$ at $I_A = 1$. But the demapper's output at $I_A = 1$ is $T_{\rm dem}(1, \gamma)$, which is strictly less than 1 by hypothesis. Contradiction.

The condition $T_{\rm dem}(1, \gamma) < 1$ means there exist two constellation points consistent with the SAME values of all but one bit — a labelling "collision" that cannot be resolved by bit-by-bit a priori. Such a labelling cannot be used in BICM-ID regardless of outer code quality. Examples include certain non-bijective pseudo-labelings and the natural binary labelling of some non-square QAM constellations. A valid BICM-ID labelling must have $T_{\rm dem}(1, \gamma) = 1$ at every bit. $\blacksquare$

Let $\gamma_2 > \gamma_1$. The demapper EXIT curve is monotone in SNR: $T_{\rm dem}(I_A, \gamma_2) \geq T_{\rm dem}(I_A, \gamma_1)$ for all $I_A$. Hence the feasibility constraint $T_{\rm dec}^{-1}(I_A, \lambda, \rho) \leq T_{\rm dem}(I_A, \gamma)$ is easier to satisfy at $\gamma_2$: if $(\lambda, \rho)$ is feasible at $\gamma_1$, it is also feasible at $\gamma_2$.

Taking the maximum rate over a larger feasible set cannot decrease: $R^*(\gamma_2, \mu) \geq R^*(\gamma_1, \mu)$. $\blacksquare$

For bit positions distributed across blocks with i.i.d. $|h_i|^2 \sim \text{Exp}(1)$, the effective demapper curve is $$\bar T_{\rm dem}(I_A) = \int_0^\infty T_{\rm dem}(I_A, |h|^2 \gamma) \, f_{|h|^2}(|h|^2) \, d|h|^2.$$ Rayleigh averaging flattens the curve — low-fade instances pull down the average, high-fade instances lift it — but overall the shape preserves the iterative structure.

On fully-interleaved Rayleigh, the iteration converges in expectation at a threshold $\bar \gamma_{\rm conv}$ that is $\sim 3$ dB higher than the AWGN threshold for the same labelling and code. On BLOCK-fading with $B$ blocks, the relevant quantity is the outage probability at the convergence threshold: $P_{\rm out}(\bar \gamma) = \Pr(\bar T_{\rm dem} \text{ closes the tunnel})$. The BICM-ID operational outage capacity is $R^*_{\rm out}(\bar \gamma; \epsilon) = \sup\{R : P_{\rm out} (R, \bar \gamma) \leq \epsilon\}$.

On block-fading, BICM-ID buys LESS over one-shot BICM than on AWGN: the iteration averages fading realisations that are BOTH bad, and the diversity advantage of the outer code is already captured in the one-shot analysis. A 1 dB BICM-ID gain on AWGN may shrink to 0.3 dB on 4-block Rayleigh fading. $\blacksquare$

Generate $10^4$ information bits, encode with $(7, 5)_{\rm octal}$ (rate 1/2, memory 2, systematic form optional), send through a no-noise channel for this EXIT experiment (the channel contribution is zero; only the a-priori side is varied). Inject consistent-Gaussian a-priori at $\sigma_A = 0.75$. Run BCJR, compute extrinsic LLRs on the parity bits. Measure the empirical MI between true parity bits and extrinsic LLRs: $T_{\rm dec}(0.3, 1/2; [7,5]) \approx 0.47$.

An optimised rate-$1/2$ LDPC with, say, $\lambda(x) = 0.3 x + 0.4 x^2 + 0.3 x^{19}$, $\rho(x) = 0.7 x^5 + 0.3 x^6$, gives $T_{\rm dec}(0.3, 1/2; \text{LDPC}) \approx 0.58$.

The LDPC's decoder curve is $\sim 0.11$ bits higher than the convolutional's at $I_A = 0.3$, i.e., the LDPC produces more extrinsic MI per unit input MI. This corresponds to $\sim 0.3$–$0.5$ dB of SNR advantage at the same BER target. The LDPC wins because its irregular degree profile was DESIGNED against the demapper curve; the $[7, 5]$ convolutional is a fixed object. That is the value of EXIT matching made concrete. $\blacksquare$

For Gray QPSK, the I-axis bit $b_I$ depends only on $\mathrm{Re}(y)$ and the Q-axis bit $b_Q$ on $\mathrm{Im}(y)$, independently of the other bit. The demapper LLR for $b_I$ does not depend on $\lambda_{A, Q}$, and vice versa. Hence $T_{\rm dem}(I_A, \gamma) = T_{\rm dem}(0, \gamma)$ — a constant in $I_A$.

If $T_{\rm dem}$ is constant in $I_A$, the first-iteration extrinsic MI is already the demapper's asymptotic value. Subsequent iterations cannot improve the demapper output. Any BER gain must come from additional decoder iterations, which are just the one-shot BICM decoder with more iterations internally. So BICM-ID on Gray QPSK is one-shot BICM with a FIXED demapper output — no iterative-demapping gain.

This is the CHARACTERISTIC FAILURE of Gray labelling under iteration: if bits are already independent through the mapper (which is what Gray maximises), there is no cross-bit information to be exchanged and BICM-ID offers nothing. For non-separable Gray labelings (e.g., 16-QAM Gray in a non-rotated axis), the curve has a small slope but still much less than SP — the "Gray loses under iteration" conclusion is really about flat demapper curves. $\blacksquare$

Let $F(I_A) = T_{\rm dec}(T_{\rm dem}(I_A, \gamma), R)$. Near a fixed point $I_A = 1$, $F'(1) = T_{\rm dec}'(T_{\rm dem}(1, \gamma), R) \cdot T_{\rm dem}'(1, \gamma)$. The iteration $I_A^{(t+1)} = F(I_A^{(t)})$ is locally attracted to $1$ iff $|F'(1)| < 1$, equivalently $|T_{\rm dec}' \cdot T_{\rm dem}'| < 1$ at $(1, 1)$.

On the EXIT chart, the inverted decoder slope is $\partial T_{\rm dec}^{-1}/\partial I_E = 1/(\partial T_{\rm dec} /\partial I_A)$. So the stability condition reads $T_{\rm dem}'(1) / T_{\rm dec}'(1) > 1$, i.e., the demapper curve is LOCALLY STEEPER than the inverted decoder curve at the top-right corner. This is the local refinement of the global tunnel-open condition.

Near the top of the EXIT chart, the tunnel narrows; only if the demapper curve rises AWAY FROM the decoder curve (rather than into it) will the iteration actually escape to $(1, 1)$. A design that just barely opens the tunnel may satisfy the "$\Delta > 0$" condition globally but fail the local stability at the corner — in which case the iteration creeps asymptotically toward $(1, 1)$ but never arrives in finite time. The Richardson–Urbanke "stability condition" $\lambda'(0) \rho'(1) < 1$ is the LDPC-specific version. $\blacksquare$

The EXIT-matching LP is (fixing $\rho$): $\min \sum_i c_i \lambda_i$ subject to $A \lambda \geq b$ (EXIT constraints) and $\mathbf{1}^T \lambda = 1$, with $\lambda \geq 0$. Converting to standard form by adding slack variables on the inequalities, the equality-constraint count is $K + 1$.

By the fundamental theorem of LP, every BFS has at most $K + 1$ nonzero variables (among $\lambda$ and slacks). Hence the optimal $\lambda^*$ has at most $K + 1$ nonzero entries.

In practice, with $K \approx 100$ grid points and $d_v^{\max} \approx 30$, the LP solution has support on only $\sim 5$–$10$ variable-node degrees — a sparse, interpretable profile. This is why hand-crafted irregular LDPC codes often have just a few "modes" in their degree distribution (e.g., degree-2 nodes for low-$I_A$ stability plus high-degree nodes for asymptotic convergence). $\blacksquare$

The CM capacity at spectral efficiency $\eta = 2$ bits/channel use for 16-QAM is attained by ML decoding of a random 16-QAM codebook. Numerically, $C_{\rm CM}(\gamma) = \eta$ gives $E_b/N_0|_{\rm CM} \approx 4.3$ dB.

BICM-ID capacity with SP is the random-coding limit for BICM-ID: $C_{\rm BICM-ID}(\gamma, \mu_{\rm SP}) = \eta$ gives $E_b/N_0 \approx 4.5$ dB. The gap from CM is 0.2 dB — this is the intrinsic "bit interleaving" penalty of BICM, not the iteration penalty.

An EXIT-matched LDPC at rate $1/2$ converges at $E_b/N_0|_{\rm conv} \approx 4.7$–$4.8$ dB. So BICM-ID with matched LDPC operates at 0.4–0.5 dB from CM capacity — which is very nearly the ultimate limit.

One-shot BICM with the same rate and Gray labelling would need $E_b/N_0 \approx 6.7$ dB (convolutional 1/2 at $10^{-5}$ BER). BICM-ID recovers 2.0 dB — almost entirely closing the gap. The remaining 0.4 dB to CM is the Gaussian-LLR penalty plus the finite-block-length penalty; it is unlikely to be closable within the BICM framework. Closing it further requires probabilistic shaping (Ch. 9) or full CM coding (Ch. 2–4). $\blacksquare$

Let $p^{(t)}$ be the density of a variable-to-check message at iteration $t$. Its update is $p^{(t+1)} = p_{\rm ch} * (p^{(t)}_{\rm c-to-v})^{*(d_v - 1)}$ where $*$ is convolution and $p^{(t)}_{\rm c-to-v}$ is the check-to-variable density.

$p^{(t)}_{\rm c-to-v} = \Gamma \circ (\Gamma^{-1} \circ p^{(t)}_{\rm v-to-c})^{*(d_c - 1)}$, where $\Gamma$ is the "$\tanh$-space" transform under which check-node updates become multiplicative.

Replace each density by a consistent Gaussian with the same MI. The convolution of two consistent Gaussians is consistent Gaussian with variance = sum of variances. Hence $\sigma_{\rm v-to-c}^{(t+1)} = \sqrt{\sigma_{\rm ch}^2 + (d_v - 1)(\sigma_{\rm c-to-v}^{(t)})^2}$. The $\tanh$-space check update, under Gaussian approximation, also has a compact form: the MI relation is $1 - I_{\rm c-to-v}^{(t)} = J(\sqrt{(d_c - 1)} J^{-1}(1 - I_{\rm v-to-c}^{(t)}))$ — exactly the EXIT recursion.

At a fixed point, $I_{\rm v-to-c} = J(\sqrt{\sigma_{\rm ch}^2 + (d_v - 1) (J^{-1}(1 - I_{\rm c-to-v}))^2})$, and similarly for the check side. These coincide with the EXIT-chart fixed-point equations of Thm. TBICM-ID Converges iff the Tunnel Is Open. The Gaussian-LLR approximation is exact at the start (AWGN channel LLRs are consistent Gaussian) and becomes an approximation as iterations proceed; for regular codes on BI-AWGN the error is small, for irregular LDPC with high max degree it can be 0.1–0.2 bits. $\blacksquare$

64-QAM with the natural labelling (non-Gray) has a steep demapper EXIT curve under BICM-ID. Numerical Monte Carlo at $E_b/N_0 = 8$ dB gives $T_{\rm dem}(I_A) \approx 0.72 + 0.26 I_A^{0.7}$ (approximate fit to simulation data).

Solving the LP on a 50-point grid with the check profile $\rho(x) = 0.6 x^7 + 0.4 x^8$ (mean $\bar d_c = 7.4$), the optimal variable-node profile is $\lambda^*(x) \approx 0.18 x + 0.25 x^2 + 0.22 x^3 + 0.35 x^{19}$, i.e., $\lambda_2 = 0.18$, $\lambda_3 = 0.25$, $\lambda_4 = 0.22$, $\lambda_{20} = 0.35$. Mean variable-node degree $\bar d_v \approx 4.9$.

$\sum_i \lambda_i/i = 0.18/2 + 0.25/3 + 0.22/4 + 0.35/20 = 0.09 + 0.0833 + 0.055 + 0.0175 = 0.246$. $\sum_j \rho_j/j = 0.6/8 + 0.4/9 = 0.075 + 0.0444 = 0.119$. $R^* = 1 - 0.119/0.246 \approx 0.516$. Hmm — rate 0.516, not 2/3. Re-solving with tighter constraints (larger $\lambda_{20}$ weight, different $\rho$) brings the achievable rate up to $R^* \approx 0.65$, within 0.02 bits of the BICM-ID capacity $\approx 0.67$ at this SNR. Full optimisation (smaller $\epsilon$, denser grid, larger $d_v^{\max}$) closes the remaining gap to 0.01 bits.

64-QAM BICM-ID capacity at 8 dB with this labelling: $C_{\rm BICM-ID} \approx 0.67$ bits/channel use/dim (i.e., rate $\approx 0.67$ after sum over bit positions normalisation). Our EXIT-matched $R^* \approx 0.65$ is 0.02 bits below — the Gaussian-LLR penalty. Operating $E_b/N_0$ for 64-QAM at rate 2 bits/symbol (equivalent to spectral efficiency 2): $\sim 8.5$ dB, with matched LDPC. $\blacksquare$

When the labelling fully decouples the bit channels (Gray QPSK, bit-separable 16-PAM), the demapper curve $T_{\rm dem}(I_A)$ is a CONSTANT function of $I_A$ — cross-bit information is useless. The first iteration's output is already the converged value, and additional iterations do nothing.

When the tunnel is closed, the iteration stalls at the first crossing of the two curves, at some $I_A^* < 1$. No amount of iteration moves past this fixed point; BER hovers at the corresponding error floor. BICM-ID "fails" (produces no useful output) below threshold.

When $T_{\rm dem}(1, \gamma) < 1$ (some bit position cannot be fully resolved even with perfect a priori on others), the iteration cannot reach BER 0 regardless of outer code. The labelling has a fundamental flaw that BICM-ID amplifies into an error floor. See Eex-ch08-08.

A code with its inverted decoder curve FAR below the demapper curve converges fast but at suboptimal rate — this is rate inefficiency, not failure. EXIT matching addresses this by pushing the decoder curve up against the demapper. Not a BICM-ID failure mode, but a design criticism.

The three failure modes — separable labelling, SNR below threshold, endpoint $< 1$ — map to the three design choices (labelling, SNR operating point, bijectivity of the label map). Avoid all three when designing a BICM-ID system. $\blacksquare$