Ferkans — Interactive Telecom Tutor

The Nonlinear Shannon Limit

The point is that optical fibre is the most important physical channel in global communication — carrying 99% of international data traffic — and yet its fundamental capacity is NOT given by a Shannon-type theorem. The Kerr nonlinearity of silica glass induces signal-dependent distortion: at low launch power, ASE noise dominates; at high launch power, nonlinear interference dominates. The rate peaks at some OPTIMAL launch power, then DECREASES — the "nonlinear Shannon limit". Coded modulation for optical coherent systems has become dominated by probabilistic amplitude shaping (PAS, Ch 19), but the underlying information theory remains open.

Definition:
Gaussian-Noise (GN) Model for Optical Fibre

The Gaussian-Noise (GN) model approximates optical fibre nonlinearity as an additive Gaussian noise with power proportional to the CUBE of the launch power. For a single-mode fibre with launch power $P$ per span, ASE noise $P_{\rm ASE}$ , and nonlinear efficiency $\eta_{\rm NL}$ : $\mathrm{SNR}_{\rm eff}(P) = \frac{P}{P_{\rm ASE} + \eta_{\rm NL} P^3}.$ The achievable rate $R(P) = 2\log_2(1 + \mathrm{SNR}_{\rm eff}(P))$ (dual-polarisation) PEAKS at $P_{\rm opt} = (P_{\rm ASE}/(2\eta_{\rm NL}))^{1/3}$ , then decreases. This peak is the GN-model "capacity".

Theorem: Nonlinear Shannon Peak (Essiambre et al. 2010)

Under the GN model with ASE power $P_{\rm ASE}$ and nonlinear efficiency $\eta_{\rm NL}$ , the achievable rate per polarisation $R^*(P) = \log_2\!\left(1 + \frac{P}{P_{\rm ASE} + \eta_{\rm NL} P^3}\right)$ is maximised at $P_{\rm opt} = (P_{\rm ASE}/(2\eta_{\rm NL}))^{1/3}$ , giving $R^*(P_{\rm opt}) = \log_2(1 + (2\eta_{\rm NL})^{-1/3} P_{\rm ASE}^{-2/3}/3)$ . Beyond $P_{\rm opt}$ , the rate DECREASES as launch power grows.

Proof

Differentiate SNR w.r.t. P

$\frac{d\,\mathrm{SNR}_{\rm eff}}{dP} = \frac{(P_{\rm ASE} + \eta_{\rm NL} P^3) - P \cdot 3\eta_{\rm NL} P^2}{(P_{\rm ASE} + \eta_{\rm NL} P^3)^2} = \frac{P_{\rm ASE} - 2\eta_{\rm NL} P^3}{(\ldots)^2}$ .

Solve = 0

Setting numerator to zero: $P^3 = P_{\rm ASE} / (2\eta_{\rm NL})$ , hence $P_{\rm opt} = (P_{\rm ASE}/(2\eta_{\rm NL}))^{1/3}$ .

Peak rate

At $P_{\rm opt}$ : $\mathrm{SNR}_{\rm eff} = P_{\rm opt}/(3 P_{\rm ASE}/2) = (2\eta_{\rm NL})^{-1/3} P_{\rm ASE}^{-2/3}/3$ . Increasing $P$ beyond $P_{\rm opt}$ REDUCES $\mathrm{SNR}_{\rm eff}$ because $\eta_{\rm NL} P^3$ grows faster than $P$ . $\blacksquare$

Optical Fibre Achievable Rate vs Launch Power

Explore the nonlinear Shannon peak: as launch power increases, rate rises (dominated by ASE), peaks at the GN-optimal point, then falls (dominated by Kerr nonlinearity). Longer fibre spans shift the peak down and right.

Parameters

Fibre length [km]1000

Example: 400ZR Coherent Optical Link at 1000 km

A 400ZR coherent optical link spans 1000 km of SMF-28e fibre with ASE power $P_{\rm ASE} = 10^{-5}$ mW per 50 GHz channel and nonlinear efficiency $\eta_{\rm NL} = 10^{-3}$ mW $^{-2}$ . Compute the optimal launch power and the peak achievable rate per polarisation.

Solution

Optimal launch power

$P_{\rm opt} = (10^{-5} / (2 \cdot 10^{-3}))^{1/3} = (5 \cdot 10^{-3})^{1/3} \approx 0.171$ mW $\approx -7.7$ dBm.

SNR at peak

$\mathrm{SNR}_{\rm eff}(P_{\rm opt}) = 0.171 / (3 \cdot 10^{-5}/2) = 11{,}400$ $\approx 40.6$ dB.

Rate per polarisation

$R^* = \log_2(1 + 11400) \approx 13.5$ bits/symbol/pol. Dual-pol: $2 \cdot 13.5 = 27$ bits/symbol — matches the ~27 bits/symbol achievable with PAS-shaped 4096-QAM in 400ZR spec.

,

PAS Is Critical for Optical Systems

Optical systems operate at VERY high SNR (40-50 dB per channel). This is the regime where the $\pi e / 6 \approx 1.53$ dB shaping gap to capacity is most visible. PAS (Ch 19) delivers about 0.8-1.0 dB of this gain in production 400ZR systems — equivalent to a 25% reach extension at matched data rate, or a 10% rate increase at matched reach. OIF 400ZR (2020) was the first mass-market deployment of PAS, shortly followed by 800G ZR (2022).

🔧Engineering Note

400G, 800G, and 1.6T Coherent Optical

Modern optical coherent generations:

400ZR (2020): 64 GBaud, dual-pol 16-QAM or PAS-shaped 16-QAM, LDPC + BICM, 30 dB OSNR requirement. Reach: 80-120 km.
800ZR (2022): 128 GBaud, PAS-shaped 64-QAM. Reach: 40-80 km.
1.6T (2026): 256 GBaud projected, PAS-shaped 256-QAM. Reach depends on nonlinearity — typically metro only. Every generation pushes closer to the GN-model nonlinear Shannon peak. The remaining headroom is 1-3 dB depending on the link.

Historical Note: The Long Road to the Nonlinear Shannon Limit

Optical fibre capacity has been studied since Stolen-Ashkin (1973, first observation of self-phase modulation). Key milestones:

1990s: WDM (wavelength-division multiplexing) scales channels to dozens per fibre; nonlinear effects become limiting.
2000s: Digital signal processing at the receiver enables electronic compensation of chromatic dispersion.
2010: Essiambre-Kramer-Winzer-Foschini-Goebel publish the GN model and the nonlinear Shannon peak — a rigorous upper bound that is asymptotically tight in the weak-nonlinearity regime.
2015+: Probabilistic shaping (Böcherer-Steiner-Schulte) matches the GN-model peak within 0.5-1 dB in production systems.
Today: digital back-propagation (DBP) extends the achievable rate by pre-equalising nonlinearity at the transmitter. Beyond- GN capacity theorems remain open.

Common Mistake: Shannon's Theorem Does Not Apply Directly to Fibre

Mistake:

"Optical channel capacity is $\log_2(1 + \mathrm{SNR})$ where SNR includes everything."

Correction:

The fibre channel is NONLINEAR and has MEMORY (chromatic dispersion, Kerr, PMD). Shannon's AWGN theorem assumes memoryless and linear — neither holds. The GN model is a LINEARIZED approximation valid at modest launch power. At high launch power, more sophisticated models (enhanced GN, Volterra series, SPS) are required, and the true capacity is still being characterised.

Key Takeaway

Optical fibre is a nonlinear channel with no Shannon-type capacity theorem. The GN model gives a rate that PEAKS at an optimal launch power and DECREASES at higher power — the nonlinear Shannon limit. PAS-based coded modulation delivers 0.8-1 dB of the 1.53 dB shaping gap; beyond-GN models and digital back-propagation are ongoing research frontiers.

Coded Modulation for the Optical Fibre Channel