Ferkans — Interactive Telecom Tutor

Beyond Shannon: When Blocks Are Short

Shannon's channel coding theorem guarantees reliable communication at rates below capacity — in the limit of infinite blocklength. But URLLC in 5G NR (Chapter 24) uses block lengths of $n = 100$ -- $500$ symbols with error probability targets of $\epsilon = 10^{-5}$ . At these parameters, the achievable rate is significantly below Shannon capacity. The finite blocklength framework, developed by Polyanskiy, Poor, and Verdu (2010), quantifies this gap precisely: the maximum rate at blocklength $n$ and error probability $\epsilon$ is $C - \sqrt{V/n} \, Q^{-1}(\epsilon) + O(\log n / n)$ , where $V$ is the channel dispersion — a new fundamental channel parameter that has no analogue in the infinite-blocklength theory.

Definition:
Finite Blocklength Channel Coding

An $(M, n, \epsilon)$ code for a channel $p(y|x)$ consists of:

An encoder $f: \{1, \ldots, M\} \to \mathcal{X}^n$
A decoder $g: \mathcal{Y}^n \to \{1, \ldots, M\}$
Average error probability $P_e = \frac{1}{M}\sum_{m=1}^M \Pr[g(Y^n) \neq m \mid X^n = f(m)] \leq \epsilon$

The maximum coding rate at blocklength $n$ and error probability $\epsilon$ is:

$R^*(n, \epsilon) = \frac{\log_2 M^*(n, \epsilon)}{n}$

where $M^*(n, \epsilon)$ is the maximum number of messages achievable with block error probability $\leq \epsilon$ .

Key properties:

$R^*(n, \epsilon) < C$ for all finite $n$ and $\epsilon < 1/2$ .
$\lim_{n \to \infty} R^*(n, \epsilon) = C$ for any $\epsilon \in (0, 1)$ (Shannon's theorem).
The rate of convergence to $C$ is governed by the channel dispersion.

In the infinite-blocklength regime, the error probability drops to zero exponentially fast for $R < C$ (error exponent theory). In the finite-blocklength regime, we fix $\epsilon > 0$ and ask how close $R$ can be to $C$ — a complementary perspective.

Definition:
Channel Dispersion

The channel dispersion of a channel $p(y|x)$ at the capacity-achieving input distribution $p^*(x)$ is:

$V = \mathrm{Var}_{p^*(x)p(y|x)}\!\left[\log\frac{p(Y|X)}{p^*(Y)}\right]$

where $i(X;Y) = \log\frac{p(Y|X)}{p^*(Y)}$ is the information density — a random variable whose mean is the mutual information $I = C$ .

For the AWGN channel with SNR $= P/N$ :

$V = \frac{1}{2}\left(1 - \frac{1}{(1 + \text{SNR})^2}\right) \cdot \left(\log_2 e\right)^2 \quad \text{(bits}^2\text{)}$

Properties:

$V > 0$ for all non-trivial channels (unless the channel is deterministic or completely noisy).
$V \to 1/2 \cdot (\log_2 e)^2$ as $\text{SNR} \to \infty$ .
$V \to 0$ as $\text{SNR} \to 0$ .
Higher dispersion means more variability in the information density, requiring longer blocks for reliable communication.

The channel dispersion plays the role of variance in the CLT applied to the information density. Just as the CLT governs the convergence of sample means, the channel dispersion governs the convergence of the empirical mutual information to its mean.

Theorem: Finite Blocklength Normal Approximation

For a stationary memoryless channel with capacity $C > 0$ and dispersion $V > 0$ , the maximum coding rate at blocklength $n$ and error probability $\epsilon \in (0, 1)$ satisfies:

$R^*(n, \epsilon) = C - \sqrt{\frac{V}{n}} \, Q^{-1}(\epsilon) + \frac{\log_2 n}{2n} + O\!\left(\frac{1}{n}\right)$

where $Q^{-1}(\cdot)$ is the inverse of the Gaussian Q-function $Q(x) = \int_x^{\infty} \frac{1}{\sqrt{2\pi}} e^{-t^2/2} dt$ .

Key consequences:

The rate penalty below capacity is $\sqrt{V/n} \, Q^{-1}(\epsilon)$ .
For $\epsilon = 10^{-5}$ : $Q^{-1}(\epsilon) = 4.265$ .
For $\epsilon = 10^{-1}$ : $Q^{-1}(\epsilon) = 1.282$ .
Tightening reliability from $\epsilon = 10^{-1}$ to $10^{-5}$ costs $\approx 3\sqrt{V/n}$ bits/c.u. — a significant penalty at short blocklengths.

Think of each channel use as generating a random "information unit" $i(X_i; Y_i)$ with mean $C$ and variance $V$ . After $n$ uses, the total information is $\sum_i i(X_i; Y_i) \approx nC + \sqrt{nV} \, Z$ where $Z \sim \mathcal{N}(0,1)$ by the CLT. Reliable decoding requires the total information to exceed $nR$ , which happens with probability $\approx Q((nR - nC)/\sqrt{nV})$ . Setting this to $1 - \epsilon$ and solving for $R$ gives the normal approximation.

Proof

Achievability (random coding)

Generate $M$ codewords i.i.d. from the capacity-achieving distribution $p^*(x)$ . Use threshold decoding: declare $\hat{m} = m$ if $\sum_{i=1}^n i(x_i(m); y_i) > n\gamma$ for some threshold $\gamma$ , and $\hat{m} = m$ is the unique such $m$ .

Error analysis: Two error events:

Correct codeword falls below threshold (missed detection): $\Pr\left[\sum_i i(X_i; Y_i) < n\gamma\right]$
Wrong codeword exceeds threshold (false alarm): $\Pr\left[\sum_i i(X_i'; Y_i) > n\gamma\right]$ for $X_i' \perp Y_i$ .

By the Berry-Esseen CLT, the first probability is: $\Pr\left[\frac{\sum_i i(X_i; Y_i) - nC}{\sqrt{nV}} < \frac{n\gamma - nC}{\sqrt{nV}}\right] \approx \Phi\!\left(\frac{\sqrt{n}(\gamma - C)}{\sqrt{V}}\right)$

The second probability is bounded by $e^{-n\gamma}$ (for the i.i.d. wrong codeword). Optimising $\gamma$ and $M = 2^{nR}$ gives the achievability bound.

Converse (meta-converse)

The converse uses the meta-converse (Polyanskiy, Poor, Verdu, 2010). For any $(M, n, \epsilon)$ code:

$M \leq \frac{1}{\sup_{\gamma} \frac{Q_X^n \times W^n[\imath > \gamma] - \epsilon}{e^{\gamma}}}$

where $\imath = \sum_i \log\frac{p(y_i|x_i)}{q(y_i)}$ is the information density and $q$ is an auxiliary output distribution.

Choosing $q = p^*(y)$ and applying the Berry-Esseen theorem: $\log_2 M \leq nC - \sqrt{nV} \, Q^{-1}(\epsilon) + \frac{1}{2}\log_2 n + O(1)$

Matching the achievability bound to $O(1)$ terms. $\blacksquare$

Definition:
URLLC Design Implications

The finite blocklength framework has direct implications for URLLC system design:

1. Rate penalty quantification: For AWGN at SNR = 5 dB ( $C = 2.06$ bits/c.u., $V = 1.96$ bits $^2$ ):

$n$	$\epsilon = 10^{-1}$	$\epsilon = 10^{-3}$	$\epsilon = 10^{-5}$
100	1.88	1.65	1.47
200	1.96	1.82	1.72
500	2.02	1.94	1.88

2. Blocklength-reliability-rate trade-off: Tightening $\epsilon$ from $10^{-1}$ to $10^{-5}$ at $n = 100$ costs $0.41$ bits/c.u. ( $22$ % of capacity) — equivalent to $\sim$ 3 dB SNR penalty.

3. Design guidelines for URLLC:

Use conservative MCS (lower rate) to absorb the finite blocklength penalty.
HARQ retransmissions reduce the effective $\epsilon$ per transmission, allowing higher MCS.
Frequency diversity (wider bandwidth) reduces the effective channel dispersion.
Pilot overhead at short blocklengths is significant: with $n_p$ pilots, effective blocklength is $n - n_p$ .

The finite blocklength theory bridges information theory and system design: it provides the fundamental limits that URLLC system designers must account for. The 1 ms latency and $10^{-5}$ BLER targets of 5G URLLC operate firmly in the finite blocklength regime.

Finite Blocklength Rate Convergence

Animated plot showing how the maximum achievable rate

R^*(n, \epsilon)

converges to the Shannon capacity

C

as the blocklength

n

increases. The gap

\sqrt{V/n}\,Q^{-1}(\epsilon)

shrinks as

1/\sqrt{n}

and is annotated at

n = 100

to illustrate the URLLC regime penalty.

The normal approximation

R^*(n,\epsilon) \approx C - \sqrt{V/n}\,Q^{-1}(\epsilon)

at SNR = 10 dB with

\epsilon = 10^{-5}

. The gap to capacity is substantial at URLLC blocklengths (

n = 100

--

500

).

Finite Blocklength Rate vs. Shannon Capacity

Visualise the maximum achievable rate as a function of blocklength for different error probabilities. The plot shows the normal approximation $R^*(n, \epsilon) \approx C - \sqrt{V/n}\,Q^{-1}(\epsilon)$ compared to the Shannon capacity (horizontal asymptote). Adjust the SNR and error probability to observe: (1) the rate gap from capacity grows with tighter reliability, (2) the gap shrinks as $1/\sqrt{n}$ , and (3) higher SNR increases both $C$ and $V$ .

Parameters

SNR (dB)10

Error probability

\epsilon

0.00001

Example: Finite Blocklength Analysis for 5G URLLC

A 5G URLLC transmission uses $n = 256$ channel uses (a 2-symbol mini-slot at 120 kHz SCS with 128 subcarriers) at SNR = 0 dB. Target BLER: $10^{-5}$ .

(a) Compute the Shannon capacity. (b) Compute the channel dispersion. (c) Compute the maximum achievable rate using the normal approximation. (d) What is the rate penalty compared to Shannon capacity? (e) How many information bits can be transmitted?

Solution

Shannon capacity

(a) $C = \frac{1}{2}\log_2(1 + 1) = 0.50$ bits/c.u.

Channel dispersion

(b) $V = \frac{1}{2}(1 - 1/(1+1)^2)(\log_2 e)^2 = \frac{1}{2} \times 0.75 \times 2.081 = 0.780$ bits $^2$ .

Maximum rate

(c) $Q^{-1}(10^{-5}) = 4.265$ .

$R^* = 0.50 - \sqrt{0.780/256} \times 4.265 = 0.50 - 0.0552 \times 4.265 = 0.50 - 0.235 = 0.265$ bits/c.u.

Rate penalty

(d) Penalty: $0.50 - 0.265 = 0.235$ bits/c.u. $= 47$ % of capacity.

At $n = 256$ and $\epsilon = 10^{-5}$ , nearly half the capacity is lost to the finite blocklength penalty.

Information bits

(e) $k = n \times R^* = 256 \times 0.265 = 67.8 \approx 67$ bits $= 8.4$ bytes.

This is very few bytes for a 256-symbol block. At Shannon capacity: $256 \times 0.50 = 128$ bits = 16 bytes. The URLLC reliability requirement halves the payload. $\blacksquare$

Quick Check

In the finite blocklength regime, which quantity determines how quickly the achievable rate converges to Shannon capacity as blocklength $n$ increases?

The channel capacity $C$

The error exponent

The channel dispersion $V$

The signal-to-noise ratio

Correction:

The channel dispersion

V

The channel dispersion $V$ governs the rate of convergence: $R^*(n,\epsilon) = C - \sqrt{V/n}\,Q^{-1}(\epsilon) + O(\log n / n)$ . The penalty term $\sqrt{V/n}\,Q^{-1}(\epsilon)$ shrinks as $1/\sqrt{n}$ , and the constant in front is $\sqrt{V}$ . Channels with higher dispersion converge more slowly to capacity. The error exponent governs the exponential decay of error probability at rates below $C$ , which is a different regime.

⚠️Engineering Note

Finite Blocklength Impact on 5G NR MCS Selection

The normal approximation directly affects MCS (Modulation and Coding Scheme) selection in 5G NR URLLC:

Mini-slots: URLLC uses 2-symbol or 4-symbol mini-slots ( $n \approx 24$ – $512$ channel uses depending on bandwidth and SCS). At these blocklengths, the rate penalty $\sqrt{V/n}\,Q^{-1}(10^{-5}) \approx 0.2$ – $0.6$ bits/c.u. is a significant fraction of capacity.
Conservative MCS: The gNB must select a lower MCS than Shannon capacity suggests. Standard link adaptation algorithms (outer-loop BLER targeting 10%) must be re-tuned for URLLC targets ( $10^{-5}$ ). The effective SNR margin needed is 3–6 dB beyond the Shannon limit.
HARQ interaction: With HARQ, the effective error probability per transmission can be relaxed (e.g., $10^{-2}$ per HARQ attempt with 3 attempts gives $10^{-6}$ overall). This allows higher MCS per attempt, but the latency budget (1 ms) limits the number of HARQ rounds.
Channel estimation overhead: At $n = 128$ channel uses, $n_p = 24$ pilots (DMRS) consume 19% of the block. The effective payload blocklength is only $n - n_p = 104$ , further widening the rate gap from capacity. Joint pilot-data design (e.g., superimposed pilots) is an active research area.

Practical Constraints

•
URLLC MCS must account for $\sqrt{V/n}\,Q^{-1}(\epsilon)$ penalty
•
Pilot overhead at short blocklengths is 15-25% of total resources
•
HARQ rounds limited by 1 ms latency budget

📋 Ref: 3GPP TS 38.214, §5.1.3 (MCS determination for URLLC)

🔧Engineering Note

Inter-Cell Interference Management in 5G NR

The interference channel model directly applies to inter-cell interference in cellular networks:

Coordinated MultiPoint (CoMP): In the strong interference regime, joint decoding across base stations (network MIMO) converts the IC into a MAC — the approach used in C-RAN architectures. 3GPP TS 36.819 specifies CoMP for LTE-A.
Enhanced ICIC (eICIC): For the weak interference regime, 5G NR uses frequency-domain ICIC: adjacent cells avoid scheduling on overlapping PRBs for cell-edge users. This is a practical approximation of treating interference as noise with interference avoidance.
The 1-bit gap reality: The ETW result shows that a simple HK scheme (TIN with appropriate power control) achieves within 1 bit of IC capacity for all parameters. This validates the engineering practice of interference-aware power control combined with TIN — sophisticated interference alignment schemes provide marginal gains in practice due to CSI imperfections and finite SNR effects.
Interference alignment in practice: Despite the elegant $K/2$ DoF result, practical IA implementations achieve only modest gains due to: (1) channel estimation errors, (2) finite symbol extensions (the required $n$ grows exponentially in $K$ ), (3) suboptimality at moderate SNR. The consensus is that IA is a theoretical breakthrough but not a practical technique.

Practical Constraints

•
CoMP requires fronthaul capacity proportional to the number of cooperating cells
•
IA requires global CSI with accuracy proportional to SNR

📋 Ref: 3GPP TS 38.214, §5.2 (resource allocation and interference management)

Channel Dispersion

The variance of the information density $\log(p(Y|X)/p^*(Y))$ under the capacity-achieving input distribution. Governs the finite-blocklength rate penalty: higher dispersion means slower convergence to Shannon capacity.

Normal Approximation (Finite Blocklength)

The approximation $R^*(n,\epsilon) \approx C - \sqrt{V/n}\,Q^{-1}(\epsilon)$ for the maximum achievable rate at blocklength $n$ and error probability $\epsilon$ . Accurate for $n \gtrsim 100$ .

Related: Channel Dispersion

Finite Blocklength Regime