Detection in Fading Channels

Fading Changes Everything

In AWGN, BER decays exponentially with SNR: Pb∼eβˆ’cβ‹…Eb/N0P_b \sim e^{-c \cdot E_b/N_0}. In fading, the channel gain ∣h∣2|h|^2 is random, so the instantaneous SNR Ξ³=∣h∣2Es/N0\gamma = |h|^2 E_s/N_0 fluctuates. Even a single deep fade (∣hβˆ£β‰ˆ0|h| \approx 0) causes a burst of errors. The result is a dramatic degradation: BER decays only algebraically PbβˆΌΞ³Λ‰βˆ’dP_b \sim \bar{\gamma}^{-d} at high SNR, where dd is the diversity order. Understanding this transition and how to combat it (diversity, coding) is central to wireless system design.

Definition:

Coherent Detection

Coherent detection assumes the receiver has perfect knowledge of the channel state information (CSI), i.e., the complex channel gain hh is known. The received signal is

r=h sm+wr = h\, s_m + w

and the ML detector with known hh selects

m^=arg⁑min⁑m∣rβˆ’h sm∣2=arg⁑max⁑mRe⁑{hβˆ—r smβˆ—}βˆ’12∣h∣2∣sm∣2\hat{m} = \arg\min_m |r - h\, s_m|^2 = \arg\max_m \operatorname{Re}\{h^* r\, s_m^*\} - \tfrac{1}{2}|h|^2|s_m|^2

For equal-energy signals, this simplifies to

m^=arg⁑max⁑mRe⁑{hβˆ—r smβˆ—}\hat{m} = \arg\max_m \operatorname{Re}\{h^* r\, s_m^*\}

The instantaneous BER (conditioned on hh) is the AWGN BER evaluated at the instantaneous SNR γ=∣h∣2Es/N0\gamma = |h|^2 E_s / N_0.

The average BER is obtained by averaging over the fading distribution:

PΛ‰b=∫0∞Pb(Ξ³) pΞ³(Ξ³) dΞ³\bar{P}_b = \int_0^\infty P_b(\gamma)\, p_\gamma(\gamma)\, d\gamma

Coherent detection requires channel estimation (Section 9.5). Estimation errors degrade performance, and the analysis assumes perfect CSI unless stated otherwise.

Theorem: Average BER for BPSK over Rayleigh Fading

For BPSK with coherent detection over a Rayleigh fading channel (∣h∣2∼Exp(1)|h|^2 \sim \text{Exp}(1), so γ∼Exp(Ξ³Λ‰)\gamma \sim \text{Exp}(\bar{\gamma})), the average BER is

PΛ‰b=12(1βˆ’Ξ³Λ‰1+Ξ³Λ‰)\bar{P}_b = \frac{1}{2}\left(1 - \sqrt{\frac{\bar{\gamma}}{1 + \bar{\gamma}}}\right)

At high SNR (γˉ≫1\bar{\gamma} \gg 1):

PΛ‰bβ‰ˆ14Ξ³Λ‰\bar{P}_b \approx \frac{1}{4\bar{\gamma}}

This shows first-order diversity (d=1d = 1): the BER decays as 1/Ξ³Λ‰1/\bar{\gamma}, i.e., only 10 dB/decade, compared to the exponential decay in AWGN.

In Rayleigh fading, there is a significant probability of a deep fade (Ξ³β‰ˆ0\gamma \approx 0), where detection is unreliable. These deep fades dominate the average BER even at high average SNR, changing the decay from exponential to algebraic.

,

Definition:

MGF Approach for Averaging BER Over Fading

The moment generating function (MGF) approach exploits Craig's formula to express the average BER as

PΛ‰b=1Ο€βˆ«0Ο€/2Mγ ⁣(βˆ’gsin⁑2Ο•)dΟ•\bar{P}_b = \frac{1}{\pi}\int_0^{\pi/2} M_\gamma\!\left(-\frac{g}{\sin^2\phi}\right) d\phi

where gg is a modulation-dependent constant (g=1g = 1 for BPSK, g=1/2g = 1/2 for orthogonal BFSK) and MΞ³(s)=E[esΞ³]M_\gamma(s) = E[e^{s\gamma}] is the MGF of the instantaneous SNR.

The key advantage is that the MGF is known in closed form for all standard fading distributions:

  • Rayleigh: MΞ³(s)=(1βˆ’sΞ³Λ‰)βˆ’1M_\gamma(s) = (1 - s\bar{\gamma})^{-1}
  • Ricean (KK factor): MΞ³(s)=1+K1+Kβˆ’sΞ³Λ‰exp⁑ ⁣(KsΞ³Λ‰1+Kβˆ’sΞ³Λ‰)M_\gamma(s) = \frac{1+K}{1+K-s\bar{\gamma}} \exp\!\left(\frac{Ks\bar{\gamma}}{1+K-s\bar{\gamma}}\right)
  • Nakagami-mm: MΞ³(s)=(1βˆ’sΞ³Λ‰/m)βˆ’mM_\gamma(s) = (1 - s\bar{\gamma}/m)^{-m}

The remaining integral over Ο•\phi has finite limits (00 to Ο€/2\pi/2) and can be evaluated analytically or by simple numerical quadrature.

,

Theorem: Diversity Order

The diversity order of a communication system is defined as

d=βˆ’limβ‘Ξ³Λ‰β†’βˆžlog⁑PΛ‰elog⁑γˉd = -\lim_{\bar{\gamma} \to \infty} \frac{\log \bar{P}_e}{\log \bar{\gamma}}

At high SNR, the average error probability behaves as

PΛ‰eβ‰ˆ(Gcβ‹…Ξ³Λ‰)βˆ’d\bar{P}_e \approx (G_c \cdot \bar{\gamma})^{-d}

where GcG_c is the coding gain (a constant that shifts the BER curve horizontally).

For BPSK over:

  • AWGN: d=∞d = \infty (exponential decay, faster than any polynomial)
  • Rayleigh fading (no diversity): d=1d = 1
  • Rayleigh with LL-branch MRC diversity: d=Ld = L
  • Ricean fading: d=1d = 1 (same as Rayleigh at high SNR)
  • Nakagami-mm: d=md = m

The diversity order counts the number of independent copies of the signal available to the receiver. Each independent copy reduces the probability of a simultaneous deep fade, changing the BER slope by one decade per 10 dB of SNR.

Example: BER of BPSK in Rayleigh vs AWGN

Compare the BER of BPSK at Eb/N0=20E_b/N_0 = 20 dB in: (a) AWGN, (b) Rayleigh fading, (c) Rayleigh with L=2L = 2 MRC diversity. Quantify the SNR penalty of fading.

BER in Fading vs AWGN

Compare BER curves in AWGN and fading channels. The AWGN curve drops exponentially (steep waterfall), while the Rayleigh fading curve drops as 1/Ξ³Λ‰d1/\bar{\gamma}^d (gentle algebraic slope). Increase the diversity order LL to steepen the fading BER curve. Switch to Ricean fading to see how a line-of-sight component improves performance.

Parameters
1
6

AWGN vs Fading BER Slopes

A cinematic comparison of BER curves on a log scale. The AWGN curve drops steeply (exponential decay), while the Rayleigh fading curve follows a gentle 1/Ξ³Λ‰1/\bar{\gamma} slope. Then diversity branches (L=2,4L = 2, 4) are added, progressively steepening the fading BER curves.
Each additional diversity branch steepens the BER slope by one decade per 10 dB of SNR increase.

Monte Carlo BER Convergence

Watch a Monte Carlo BER simulation converge as the number of transmitted symbols increases. The estimated BER (with confidence interval) converges to the theoretical value. Observe that reliable estimation of Pb=10βˆ’kP_b = 10^{-k} requires approximately 100/Pb=10k+2100/P_b = 10^{k+2} symbols.

Parameters
10

Quick Check

A system with 4-branch MRC diversity over Rayleigh fading has diversity order d=4d = 4. If the average SNR increases by 10 dB, by approximately how many decades does the BER decrease?

4 decades (BER improves by a factor of 10410^4)

1 decade

10 decades

Infinite (error-free at high SNR)

Common Mistake: Error Floor in Fading Without Diversity

Mistake:

Expecting that increasing transmit power alone will achieve arbitrarily low BER in a Rayleigh fading channel without diversity.

Correction:

Without diversity (d=1d = 1), PΛ‰bβ‰ˆ1/(4Ξ³Λ‰)\bar{P}_b \approx 1/(4\bar{\gamma}). Achieving Pb=10βˆ’6P_b = 10^{-6} requires Ξ³Λ‰=250,000=54\bar{\gamma} = 250{,}000 = 54 dB, which is impractical.

Diversity is essential in fading channels. With L=4L = 4 MRC, the same Pb=10βˆ’6P_b = 10^{-6} requires only Ξ³Λ‰β‰ˆ16\bar{\gamma} \approx 16 dB β€” a 38 dB saving. Alternatively, coding provides time/frequency diversity, and MIMO provides spatial diversity.

The lesson: in fading channels, spending power on diversity is far more effective than simply increasing transmit power.

Non-Coherent Detection Trades 3 dB for No CSI

Non-coherent detection does not require knowledge of the channel phase, using only the signal envelope for detection. For orthogonal signaling (e.g., non-coherent BFSK), the BER in Rayleigh fading is

Pˉb=12+γˉ\bar{P}_b = \frac{1}{2 + \bar{\gamma}}

compared to PΛ‰bβ‰ˆ1/(4Ξ³Λ‰)\bar{P}_b \approx 1/(4\bar{\gamma}) for coherent BPSK. The penalty is approximately 3 dB (a factor of 2 in SNR).

Differentially coherent detection (DPSK, DQPSK) is intermediate: it uses the previous symbol as a phase reference, requiring no explicit channel estimation. The BER for DPSK in Rayleigh fading is

Pˉb=12(1+γˉ)\bar{P}_b = \frac{1}{2(1 + \bar{\gamma})}

The 3 dB penalty is often acceptable in rapidly time-varying channels where channel estimation is unreliable.

Coherent vs Non-Coherent vs Differential Detection

AspectCoherentDifferentialNon-coherent
CSI requiredFull (amplitude + phase)Previous symbol phaseNone
Example modulationsBPSK, QPSK, QAMDPSK, DQPSKNon-coherent FSK
BER (Rayleigh)14Ξ³Λ‰\frac{1}{4\bar{\gamma}}12(1+Ξ³Λ‰)\frac{1}{2(1+\bar{\gamma})}12+Ξ³Λ‰\frac{1}{2+\bar{\gamma}}
SNR penalty vs coherentBaselineβ‰ˆ\approx 1 dB (AWGN), 3 dB (fading)β‰ˆ\approx 3 dB
Applicable to QAM?YesLimited (DQPSK)No (amplitude info lost)
ComplexityHigh (channel estimator)MediumLow
⚠️Engineering Note

Link Budget Fading Margins in Practice

The algebraic BER decay in fading (PΛ‰bβˆΌΞ³Λ‰βˆ’d\bar{P}_b \sim \bar{\gamma}^{-d}) has direct implications for link budget design. To achieve a target BER without diversity, the required fading margin β€” the extra SNR relative to AWGN β€” is enormous:

Target BER AWGN Eb/N0E_b/N_0 Rayleigh (d=1d=1) Margin
10βˆ’310^{-3} 6.8 dB 24 dB 17.2 dB
10βˆ’510^{-5} 9.6 dB 44 dB 34.4 dB
10βˆ’610^{-6} 10.5 dB 54 dB 43.5 dB

In 5G NR and LTE, the standard approach is to use diversity (2-4 Rx antennas, transmit diversity via SFBC) to increase dd, combined with HARQ to handle residual errors. The link adaptation algorithm (AMC) selects MCS based on the effective post-diversity SNR, targeting a BLER of 10% for initial transmission (relying on HARQ retransmissions for reliability).

The 3GPP link budget methodology accounts for fading through the shadow fading margin (log-normal, typically 7-10 dB) and the fast fading margin (handled implicitly by HARQ and diversity).

Practical Constraints
  • β€’

    Without diversity: 30-40 dB fading margin needed for BER < 1e-5

  • β€’

    With 2-branch MRC: margin reduces by ~20 dB

  • β€’

    5G NR target BLER: 10% for eMBB, 0.001% for URLLC

πŸ“‹ Ref: 3GPP TR 38.901 (channel models), 3GPP TS 38.214 (link adaptation)

Key Takeaway

The core message of this section in three points:

  • Fading changes BER decay from exponential (eβˆ’cΞ³Λ‰e^{-c\bar{\gamma}}, AWGN) to algebraic (Ξ³Λ‰βˆ’d\bar{\gamma}^{-d}), where dd is the diversity order. This is the most fundamental difference between AWGN and fading channels.

  • Diversity steepens the BER slope: each independent replica of the signal increases dd by 1, and dd additional decades of BER improvement are gained per 10 dB of SNR.

  • The MGF approach (Craig's formula + fading MGF) provides a unified analytical framework for computing average BER over any fading distribution, avoiding numerical integration of nested integrals.

Why This Matters: Spatial Diversity and MIMO Detection

The diversity order analysis in this section considers receive diversity (MRC) with LL independent branches. In practice, spatial diversity is achieved through MIMO antenna arrays. The MIMO book develops this in full depth:

  • Spatial diversity: Alamouti code, space-time block codes (Chapter 10 of this book provides an introduction)
  • MIMO detection: ML, ZF, MMSE, sphere decoding β€” extending the detection theory of Section 9.1 to vector channels
  • Diversity-multiplexing tradeoff (DMT): the fundamental trade-off between diversity gain and spatial multiplexing gain in MIMO systems (includes CommIT contributions by Caire et al.)
  • Massive MIMO: when Lβ†’βˆžL \to \infty, the channel hardens and approaches AWGN behaviour β€” closing the gap between fading and AWGN performance analysed in this section

Diversity Order

The negative slope of the log-BER vs log-SNR curve at high SNR: d=βˆ’lim⁑log⁑Pe/log⁑γˉd = -\lim \log P_e / \log \bar{\gamma}. It equals the number of independent signal copies available to the receiver and determines the rate of BER improvement with increasing SNR.

Related: Maximal-Ratio Combining (MRC), Fading Changes Everything, MGF Approach for Averaging BER Over Fading

Coherent Detection

A detection method that requires knowledge of the channel's complex gain (amplitude and phase) at the receiver. Coherent detection achieves the best performance but requires accurate channel estimation, which may be impractical in rapidly varying channels.

Related: Channel Estimation in OFDM, Non-Coherent Detection Trades 3 dB for No CSI, Non Coherent