Exercises

$e^{-\lambda \cdot 9} \approx 0.165$ (for $a = \pm 3$). $e^{-\lambda \cdot 1} \approx 0.819$ (for $a = \pm 1$).

Sum = $2(0.165 + 0.819) = 1.968$. Probabilities: $P_A(\pm 3) = 0.165/1.968 = 0.084$; $P_A(\pm 1) = 0.819/1.968 = 0.416$.

$H(P_A) = -2(0.084 \log_2 0.084) - 2(0.416 \log_2 0.416) \approx 0.601 + 1.053 = 1.654$ bits (vs 2 bits uniform).

$R_{\rm PAS} = 0.75 \cdot (2.5 + 3) + 0.75 - 1 = 4.125 + (-0.25) = 3.875$ bits/symbol.

Vs uniform 64-QAM at rate 0.75: $R = 0.75 \cdot 6 + (-0.25) = 4.25$. Shaped rate is lower, but requires less SNR to achieve.

$\Delta R = \frac{7}{2 \cdot 500}\log_2 500 = 7/1000 \cdot 8.97 = 0.063$ bits/symbol.

$0.063/2.8 = 2.2\%$ — modest loss at moderate $n$.

$\lambda = 0$: $H(P_A) = \log_2 \sqrt{M} = 2$. Rate $= 0.9375 \cdot (2 + 2) + 0.9375 - 1 = 3.75 - 0.0625 = 3.6875$ bits/symbol/pol.

$\lambda \to \infty$: $H(P_A) \to 0$. Rate $= 0.9375 \cdot 2 + (-0.0625) = 1.8125$ bits/symbol/pol.

PAS sweeps rate from 1.81 to 3.69 bits/symbol/pol continuously — about a 2x span on a single (M, R) pair.

$\mathcal{L} = -\sum_x p(x)\log p(x) - \mu_1(\sum p(x) - 1) - \mu_2(\sum x^2 p(x) - \bar{E})$.

$\frac{\partial\mathcal{L}}{\partial p(x)} = -\log p(x) - 1 - \mu_1 - \mu_2 x^2 = 0$, giving $p(x) \propto e^{-\mu_2 x^2}$.

$\mu_2 = \lambda$ (scaling by the power constraint). This is the Maxwell-Boltzmann distribution. $\blacksquare$

The Maxwell-Boltzmann distribution $e^{-\lambda |x|^2}$ depends only on $|x|$ — it is symmetric around zero. So the sign of each shaped symbol is ALREADY uniform under MB.

This means the sign bits can be PRODUCED directly by the LDPC systematic output (which is uniform) without any shaping. Only the amplitudes need the CCDM. This is the key PAS insight: shaping only the amplitudes = FEC-compatibility.

$p(|a|) \propto e^{-0.04 a^2}$. Compute for $|a| \in \{1, 3, 5, \ldots, 15\}$: $e^{-0.04}, e^{-0.36}, e^{-1.0}, e^{-1.96}, ...$ $\approx 0.961, 0.698, 0.368, 0.141, 0.041, 0.009, 0.0015, 0.0002$. Sum = 2.22. Pairs (for $\pm a$): divide by $2 \cdot 2.22$. Each amplitude has $P_A(\pm 1) \approx 0.216, P_A(\pm 3) \approx 0.157, P_A(\pm 5) \approx 0.083, \ldots$.

$H(P_A) \approx -\sum P \log_2 P \approx 3.1$ bits (vs 4 bits uniform for 16-PAM).

Uniform per-axis entropy = 4 bits; shaped = 3.1 bits. The gap of 0.9 bits/axis corresponds to roughly $0.9 \cdot 6 = 5.4$ dB of rate reduction, but the SNR reduction (shaping gain) is much smaller — about 1.2 dB for this parameter choice, as measured numerically.

Between 3.33 and 4.5 bits/symbol, there is a 1.17 bit/symbol quantisation gap at the MCS boundary.

With 64-QAM + $R_c = 5/6$ and a PAS-tuned $H(P_A)$, the system can achieve any rate between $5/6 \cdot 2 - 1/6 = 1.5$ and $5/6 \cdot 6 - 1/6 = 4.83$ bits/symbol. In particular, a rate of 3.90 bits/symbol is achievable continuously.

The MCS gap (3.33 → 4.5) is wasteful — a link that could operate at 3.9 bits/symbol is forced to 3.33. PAS recovers this efficiency.

$I(X; Y) = H(Y) - H(Y|X)$ for a channel $Y = f(X) + W$. The shaping affects $H(Y)$ through the marginal of $X$ — but only via the DISTRIBUTION of $Y$ induced by the shaping.

If scheme A (PS) places uniform input on non-uniform point locations and scheme B (GS) places non-uniform input on a uniform grid, and BOTH induce the same marginal distribution of $Y$, then $I_A(X;Y) = I_B(X;Y)$ exactly.

In practice the two schemes LOOK different in the encoder but are information-theoretically identical. Hence the "asymptotic equivalence" in Thm. 1 of §4. The BER difference between the two comes from the RECEIVER processing, not from the encoder side. $\blacksquare$

Split the $n$-symbol output into $K$ sub-blocks of length $n/K$. Apply CCDM separately to each, with target distribution targeted at the sub-block level. The overall rate is close to $H(P_A)$ but with smaller finite-$n$ losses.

Per sub-block, the CCDM rate loss is $O(\log(n/K)/(n/K))$. Summed over $K$ sub-blocks: total loss $\approx K \cdot (\log(n/K))/(n/K) \cdot (1/K)$ $= (\log(n/K))/(n/K)$.

As $K$ grows, each sub-block is shorter and the local rate loss per sub-block grows like $\log(n/K)/(n/K)$. The optimal $K$ scales as $\log n / \log\log n$. Modern implementations use $K \approx 100-1000$ at $n \sim 10^4$.

The cross-entropy loss $-\mathbb{E}\log p(s|Y)$ equals $H(S) - I(S;Y)$, which is minimised when $I(S;Y)$ is maximised — the same objective as capacity.

Since MI-maximising constellations on AWGN are MB-shaped QAM (up to unitary rotation), the autoencoder converges to the same solution as analytical PS.

The autoencoder may learn a ROTATED version of MB-shaped QAM because rotation preserves MI. But modulo rotation, the solution is unique. This is a useful sanity check: if the autoencoder doesn't converge to MB on AWGN, the training setup is broken.

$G(S_n) \to \frac{1}{2\pi e}$ as $n \to \infty$ (Zador 1996).

$G(C_n) = 1/12$ for all $n$ (uniform on cube).

$\gamma_s = G(C_n)/G(S_n) \to (1/12)/(1/(2\pi e)) = 2\pi e / 12 = \pi e / 6$.

$\pi e/6 \approx 3.1416 \cdot 2.718/6 \approx 1.423$. In dB: $10\log_{10}(1.423) = 1.53$ dB. $\blacksquare$

For each of 1000 positions, the CCDM evaluates 8 candidates: $1000 \times 8 = 8000$ operations per block.

At 400 GBaud: 400,000 blocks/second. Total: $400{,}000 \times 8000 = 3.2 \cdot 10^9$ operations/second. Well within modern DSP chip capacity (~$10^{11}$ ops/s).

Each block needs a multinomial-lookup table of size $O(\binom{n}{n/2})$ — too large for $n = 1000$. Practical implementations use floating-point arithmetic coding with $O(n)$ memory per block.

A non-systematic code outputs $\mathbf{c} = \mathbf{G}\mathbf{u}$ with NO IDENTIFIABLE shaped and unshaped bit components. After passing through the non-systematic encoder, the output bit distribution is approximately UNIFORM regardless of input shape.

The BICM mapper then sees uniform input bits and produces uniform QAM output — the shaping is LOST.

Use a SYSTEMATIC code: its output includes the original shaped bits unchanged, and parity bits are added alongside. The uniformly-distributed parity bits serve as SIGN bits in the PAS architecture.

CDA codes' non-vanishing-determinant theorem (Ch 13) is proved for UNIFORM inputs. If the input is MB-shaped, does the NVD property still hold?

Likely yes, with a tighter constant in $\delta_{\min}$, because MB shaping compresses outer points toward the centre — possibly reducing codeword-pair determinants but preserving positivity.

(a) Show the minimum determinant is bounded below by a positive constant under MB input. (b) Characterise the DMT of PAS+CDA — likely still $d^*(r)$ but with modified coding gain. (c) Practical construction: how does the CCDM interact with the CDA codeword construction? This would be a solid PhD thesis topic.