Ferkans — Interactive Telecom Tutor

From LLR Densities to Mutual-Information Scalars

The analysis of BICM-ID in its most general form is a dynamical system on a space of LLR distributions: at each iteration the demapper maps an input LLR density to an output LLR density, and the decoder does the same. That is the "density evolution" analysis used for LDPC codes, and while it is exact, it lives in an infinite- dimensional space that is hard to visualise, slow to compute, and painful to optimise over.

The EXIT chart, introduced by Stephan ten Brink in 2001, is the projection of density evolution onto a SINGLE scalar per iteration — the mutual information between the true bit and its LLR. The tracking is no longer exact, but it preserves enough information for design purposes whenever the LLR density is approximately Gaussian. Okay so what we get is this: two curves on the unit square, a staircase trajectory that walks between them, and a "tunnel" whose width is the SNR margin. You can read convergence off the picture with a ruler. That is why the EXIT chart took over the iterative- coding literature within a year of its publication — it turned a research problem into a design chart.

The point is that iterative BICM-ID is just alternating projections between two nonlinear maps — the demapper and the decoder. EXIT charts are the dynamical-systems view: we watch the orbit and ask whether it escapes to $(1, 1)$ or gets trapped.

Definition:
Consistent-Gaussian LLR Assumption

An LLR random variable $\lambda$ conditioned on a bit $B \in \{0, 1\}$ is said to be consistent Gaussian if $\lambda \mid B = b \;\sim\; \mathcal{N}\!\left((1 - 2b) \cdot \tfrac{\sigma^2}{2}, \; \sigma^2\right),$ i.e., Gaussian with mean $\pm \sigma^2/2$ (positive for $b = 0$ , negative for $b = 1$ ) and variance $\sigma^2$ . The parameter $\sigma$ (standard deviation of $\lambda$ ) is the sole summary statistic: mean and variance are locked together by the "consistency" relation $\mathrm{mean} = \mathrm{var}/2$ . Equivalently, the likelihood ratio $e^{\lambda}$ is a log-normal with the correct Bayes property $P(B = 0 \mid \lambda) = 1/(1 + e^{-\lambda})$ .

The consistent-Gaussian model is EXACT for the BI-AWGN channel: if $y = (1 - 2b)\sqrt{E_s} + w$ with $w \sim \mathcal{N}(0, N_0/2)$ , then $\lambda = (4\sqrt{E_s}/N_0) y$ is consistent Gaussian with $\sigma^2 = 8 E_s/N_0$ . For other channels — non-Gaussian noise, higher-order modulation, LDPC messages after a few iterations — the Gaussian model is an APPROXIMATION that turns out to be surprisingly accurate for mutual-information purposes, even when the exact density is markedly non-Gaussian.

,

Definition:
The J-Function

The J-function is the mutual information between a uniformly distributed binary input $B \in \{0, 1\}$ and a consistent-Gaussian LLR $\lambda$ of standard deviation $\sigma$ : $J(\sigma) \triangleq I(B; \lambda) = 1 - \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi \sigma^2}} \exp\!\left(-\frac{(\xi - \sigma^2/2)^2}{2\sigma^2}\right) \log_2\!\left(1 + e^{-\xi}\right) d\xi.$ It is a monotonically increasing function $J : [0, \infty) \to [0, 1]$ with $J(0) = 0$ and $\lim_{\sigma \to \infty} J(\sigma) = 1$ . The inverse $J^{-1} : [0, 1] \to [0, \infty)$ is equally important: given a target mutual information $I \in [0, 1]$ , $J^{-1}(I)$ is the LLR standard deviation needed to achieve it.

Several accurate closed-form approximations to $J(\sigma)$ are tabulated in [?ten-brink-kramer-ashikhmin-2004], including a two-piece fit of the form $J(\sigma) \approx 1 - 2^{-H_1 \sigma^{2 H_2}}$ for small $\sigma$ and $J(\sigma) \approx 1 - e^{-H_3 \sigma + H_4}$ for large $\sigma$ . Any numerical implementation of EXIT analysis should tabulate $J$ and $J^{-1}$ once and interpolate.

,

Example: Computing $J(1)$ and $J(2)$

Compute $J(\sigma)$ for $\sigma = 1$ and $\sigma = 2$ to 4-digit accuracy using the defining integral or a tabulated approximation. Interpret the values.

Solution

Direct numerical integration

A 256-point Gauss–Hermite quadrature applied to $J(\sigma) = 1 - \mathbb{E}[\log_2(1 + e^{-\Lambda})], \quad \Lambda \sim \mathcal{N}(\sigma^2/2, \sigma^2),$ gives $J(1) \approx 0.2891$ and $J(2) \approx 0.6727$ . A spot-check against the tabulated values in ten Brink's 2001 Table I matches to four decimal places.

Interpretation

An LLR standard deviation $\sigma = 1$ carries only 0.29 bits of information about a uniform binary input — the bit is mostly undetermined. Doubling to $\sigma = 2$ more than doubles the MI to 0.67 bits. The transition from $\sigma \approx 1$ to $\sigma \approx 3$ covers almost the full MI range $[0.25, 0.96]$ : EXIT curves live in a narrow LLR range, which is why the Gaussian-LLR assumption is especially accurate in the regime where it matters.

Translating to SNR for BI-AWGN

For BI-AWGN, $\sigma^2 = 8 E_s/N_0$ , so $\sigma = 1$ corresponds to $E_s/N_0 = 1/8 = -9.0$ dB and $\sigma = 2$ to $E_s/N_0 = 0.5 = -3.0$ dB. These are the SNR values at which the BI-AWGN channel carries $J(\sigma)$ bits of MI per use. $\blacksquare$

Definition:
Demapper EXIT Curve $T_{\mathrm{dem}}$

Let the demapper with a priori take independent consistent-Gaussian a-priori LLRs $\lambda_A$ with variance $\sigma_A^2$ satisfying $J(\sigma_A) = I_A$ , and channel observations at a given SNR. Assume the labelling $\mu$ is symmetric (bit-channel invariance under the label group). The demapper's extrinsic output $\lambda_E$ is modelled as consistent Gaussian with variance $\sigma_E^2$ , and the demapper EXIT curve is the map $T_{\mathrm{dem}}(I_A, \mathrm{SNR}) \triangleq I(B; \lambda_E) = J(\sigma_E), \qquad \sigma_E^2 = f_{\rm dem}(\sigma_A^2, \mathrm{SNR}, \mu),$ where $f_{\rm dem}$ is determined by the labelling, the constellation, and the channel. In practice $T_{\rm dem}$ is computed by Monte Carlo over the demapper, fitting the output LLR to a consistent Gaussian by MI matching.

Three landmarks identify a demapper EXIT curve at a glance. At $I_A = 0$ the curve starts at the one-shot BICM bit-channel MI: $T_{\mathrm{dem}}(0, \mathrm{SNR}) = I(B; \lambda_{\rm ch})$ , the Ch. 5 bit-channel capacity. At $I_A = 1$ the curve ends at 1 if the demapper-with-perfect-a-priori can identify the bit uniquely (which depends on the labelling, s04), otherwise at a value less than 1. Between the two endpoints, the steepness of the curve is set by how much the a priori on other bits SHARPENS the bit-0 LLR — the crucial quantity that distinguishes SP from Gray (s04).

,

Theorem: Demapper EXIT Is a Function of $(I_A, \mathrm{SNR})$ Alone

Under the consistent-Gaussian LLR assumption for the a-priori input and the symmetry of the labelling, the demapper extrinsic mutual information $I_E$ depends on the a-priori LLR distribution only through its mutual information $I_A = J(\sigma_A)$ . Equivalently, $T_{\mathrm{dem}}(I_A, \mathrm{SNR})$ is well-defined as a function of two scalars: a-priori MI and channel SNR. The same conclusion holds for the decoder EXIT $T_{\mathrm{dec}}(I_A, R)$ : under the consistent- Gaussian model, the decoder extrinsic MI depends on the input LLR distribution only through $I_A$ and the code parameters (rate, degree distribution).

Within the consistent-Gaussian family, a single parameter — variance, equivalently MI — describes the entire distribution. So any functional of the distribution (including the demapper's extrinsic output) can be parametrised by a single scalar. This is what allows us to plot $I_E$ against $I_A$ instead of tracking full densities.

Show Hint

Start from the fact that a consistent-Gaussian family is one-parameter: mean and variance are locked by mean = var/2, so the distribution is determined by $\sigma$ alone.

The invertible map $J : \sigma \to I_A$ lets us use $I_A$ as the parameter.

The demapper's extrinsic output, being a deterministic function of its inputs (observations + a priori), has an output distribution that depends on the a-priori distribution only through $\sigma_A$ , hence through $I_A$ .

Proof

Step 1: One-parameter family

A consistent-Gaussian LLR is fully characterised by $\sigma$ : both mean and variance are functions of $\sigma$ . The family of such distributions is therefore a one-parameter family parametrised by $\sigma \in [0, \infty)$ , or equivalently by $I_A = J(\sigma) \in [0, 1]$ (since $J$ is a bijection).

Step 2: Extrinsic output distribution

The demapper output $\lambda_E$ is a deterministic function of the channel observation $y$ , the a-priori LLR vector $\lambda_A$ , and the constellation. Conditional on the transmitted bit $B_0 = b_0$ , the distribution of $\lambda_E$ is determined by the joint distribution of $(y, \lambda_A)$ — and under the consistent-Gaussian assumption, the distribution of $\lambda_A$ is determined by $\sigma_A$ , hence by $I_A$ .

Step 3: MI summarises what matters

We now project the output distribution onto its mutual information with the true bit: $I_E = I(B_0; \lambda_E)$ . Since the joint distribution of $(B_0, \lambda_E)$ is determined by $(I_A, \mathrm{SNR})$ , so is $I_E$ . Hence $T_{\mathrm{dem}}(I_A, \mathrm{SNR})$ is a well-defined scalar function.

Step 4: Decoder case

The same argument applies to the decoder: under the consistent- Gaussian assumption for both channel and a-priori LLRs at the decoder input, the decoder's extrinsic output has a distribution determined by $I_A$ and the code. MI projection gives $T_{\mathrm{dec}}(I_A, R)$ . $\blacksquare$

,

Definition:
Decoder EXIT Curve $T_{\mathrm{dec}}$

Let the SISO decoder take a-priori LLRs $\lambda_A$ with MI $I_A$ (assumed consistent Gaussian) and produce extrinsic LLRs $\lambda_E$ . The decoder EXIT curve is $T_{\mathrm{dec}}(I_A, R) \triangleq I(B; \lambda_E),$ where $R$ is the code rate (and, for LDPC, the degree profile). For an $(d_v, d_c)$ -regular LDPC code, the curve factorises cleanly into check-node and variable-node sub-curves; for convolutional codes it is computed by running the BCJR algorithm with consistent-Gaussian a-priori noise. On the EXIT chart the decoder curve is plotted with the ROLE OF $I_A$ AND $I_E$ SWAPPED: its INVERSE $I_A = T_{\mathrm{dec}}^{-1}(I_E, R)$ is drawn on the same axes as the demapper curve, so both maps share the $(I_A, I_E)$ coordinate system.

The "swap" convention matters: when following the iteration on the chart, the demapper output $I_E$ becomes the decoder input $I_A$ (after deinterleaving, which preserves MI), so the natural reading direction alternates horizontal — vertical — horizontal between the two curves. The staircase trajectory is what falls out.

,

EXIT Chart: Demapper, Decoder, and Iteration Trajectory

The EXIT chart for 16-QAM BICM-ID. The blue curve is the demapper EXIT $T_{\mathrm{dem}}(I_A, \mathrm{SNR})$ for the selected labelling and SNR. The red curve is the INVERTED decoder EXIT $T_{\mathrm{dec}}^{-1}(I_A, R)$ for a regular rate- $R$ LDPC (or, equivalently, a convolutional code of the same rate), plotted on the same $(I_A, I_E)$ axes. The green staircase is the iterative trajectory starting from $(0, 0)$ : each horizontal step is one demapper pass (moves $I_A \to T_{\mathrm{dem}}(I_A)$ ), each vertical step is one decoder pass (moves along the inverted decoder curve). The trajectory converges to $(1, 1)$ if the tunnel is open, i.e., the demapper curve lies strictly above the inverted decoder curve. Try SNR = 2 dB with SP and watch the tunnel close.

Parameters

SNR [dB]3

Code rate

R

0.5

Bit labelling

Pattern: EXIT Curves Are Transfer Functions

Thinking of $T_{\mathrm{dem}}$ and $T_{\mathrm{dec}}$ as transfer functions in the sense of linear-system theory is productive even though both maps are nonlinear. The iteration $I_A^{(t+1)} = T_{\mathrm{dec}}(T_{\mathrm{dem}}(I_A^{(t)}, \mathrm{SNR}), R)$ is the feedback loop; its fixed points are where $I_A = T_{\mathrm{dec}}(T_{\mathrm{dem}}(I_A, \mathrm{SNR}), R)$ ; stability of a fixed point is governed by the local slopes of the two curves. When we talk about "matched" codes in s05, the right analogy is loop shaping: arrange the decoder curve to mirror the demapper curve, from just below, so the closed-loop gain is slightly greater than 1 everywhere on $[0, 1)$ and the trajectory is pulled up to $(1, 1)$ .

Common Mistake: The Gaussian-LLR Assumption Is Not Always Accurate

Mistake:

EXIT charts assume all LLR densities along the iteration are consistent Gaussian. The chart looks clean and gives sharp convergence thresholds. It is tempting to report those thresholds as if they were operating SNRs for finite-block-length implementations.

Correction:

The consistent-Gaussian approximation is excellent for regular LDPC decoders at moderate iterations on BI-AWGN, and good enough for higher-order modulation DEMAPPERS that average over multiple symbols. It fails in three regimes: (i) very low SNR, where demapper LLRs become heavy-tailed non-Gaussian; (ii) short block lengths (< 1000 bits), where the interleaver does not fully decorrelate and the i.i.d. a-priori assumption breaks; (iii) early iterations of irregular LDPC codes with high-degree variable nodes, where the output distribution is visibly bimodal. When in doubt, validate the EXIT-chart prediction with a Monte Carlo BER simulation at 2–3 design SNRs before committing to the design. Density evolution is the expensive but exact alternative.

Historical Note: ten Brink's EXIT Chart (2001)

2001

Stephan ten Brink introduced the EXIT chart in 2001 with a paper in IEEE Transactions on Communications originally titled "Convergence behavior of iteratively decoded parallel concatenated codes." The context was turbo-code analysis — two parallel soft decoders exchanging extrinsic information — and the paper's central innovation was the realisation that the full LLR-density dynamics could be replaced by a one-dimensional mutual-information dynamics with negligible loss of predictive power. Within a year the methodology had been ported to BICM-ID (by ten Brink himself and by Li and Ritcey), to iterative equalisation (Tüchler et al.), and to LDPC decoding (ten Brink, Kramer, Ashikhmin). By the mid-2000s EXIT charts were a standard design tool in industrial coding labs. The underlying Gaussian-LLR approximation has been rigorously justified for binary erasure channels [?ashikhmin-kramer-ten-brink-2004] and remains an excellent engineering approximation on more general channels.

Historical Note: Li and Ritcey: BICM-ID Before EXIT Charts

1997–1999

Four years before ten Brink's EXIT-chart paper, Xiaodong Li and James Ritcey of the University of Washington noticed that the iterative- decoding principle developed by Berrou, Glavieux, and Thitimajshima for turbo codes could be applied to BICM: let the decoder's extrinsic output serve as a priori for the demapper, and iterate. Their 1997 IEEE Communications Letters paper showed, by Monte Carlo simulation, that on Rayleigh fading BICM-ID could close 2–3 dB of the one-shot BICM gap — and that set-partition labelling was needed to realise that gain, overturning the Gray-is-best conclusion for one-shot BICM (Ch. 6). They lacked a theoretical tool for predicting the convergence threshold, however, and their 1999 follow-up paper used ad-hoc fixed-point analysis. ten Brink's EXIT chart supplied the missing theoretical tool and, in retrospect, explains exactly WHY SP's steeper demapper curve is the right choice under iteration.

,

Quick Check

What does the J-function $J(\sigma)$ give?

The BER of BPSK over AWGN at SNR $\sigma^2$ .

The mutual information between a binary input and a consistent-Gaussian LLR of standard deviation $\sigma$ .

The capacity of 16-QAM at SNR $\sigma$ .

The rate of a BICM system at SNR $\sigma$ with Gray labelling.

Correction:

The mutual information between a binary input and a consistent-Gaussian LLR of standard deviation

\sigma

.

Correct. $J(\sigma) = I(B; \Lambda)$ where $\Lambda \sim \mathcal{N}(\pm \sigma^2/2, \sigma^2)$ conditional on $B$ . It takes values in $[0, 1]$ with $J(0) = 0$ and $J(\infty) = 1$ .

J-function

The mutual information between a uniformly distributed binary input $B \in \{0, 1\}$ and a consistent-Gaussian LLR $\Lambda \sim \mathcal{N}((1 - 2B)\sigma^2/2, \sigma^2)$ . A bijection $J : [0, \infty) \to [0, 1]$ with $J(0) = 0$ , $J(\infty) = 1$ . The J-function and its inverse are the bridge between the LLR-variance and the mutual- information views of a bit channel, and are the numerical workhorse of every EXIT-chart computation.

EXIT Chart (Extrinsic Information Transfer Chart)

A two-dimensional diagram on the unit square $(I_A, I_E) \in [0, 1]^2$ showing the demapper and decoder transfer curves of an iterative receiver. The demapper curve is $I_E = T_{\rm dem}(I_A, \mathrm{SNR})$ ; the decoder curve is plotted with swapped axes, $I_A = T_{\rm dec}^{-1}(I_E, R)$ . The iteration follows a staircase between the two curves; convergence to $(1, 1)$ occurs iff the tunnel between them is open.

Key Takeaway

EXIT charts compress density evolution into a 1D dynamical system. Under the consistent-Gaussian LLR assumption, each soft-in/soft-out box is a scalar map from a-priori MI to extrinsic MI. The demapper map $T_{\mathrm{dem}}(\cdot, \mathrm{SNR})$ and decoder map $T_{\mathrm{dec}}(\cdot, R)$ jointly define BICM-ID's iteration, and the J-function converts between MI and LLR variance so the maps can be implemented with a single tabulated function. Everything interesting — convergence, thresholds, code design — becomes a geometric property of two curves on the unit square.

EXIT Chart Fundamentals