The Separation Theorem — When It Holds and When It Doesn't

The rate-distortion function is $R(D) = \frac{1}{2}\log(\sigma_S^2 / D)$. The channel capacity is $C = \frac{1}{2}\log(1 + P/\sigma^2)$. At $\kappa = 1$, separation achieves distortion: $$D_{\text{sep}} = \sigma_S^2 \cdot 2^{-2C} = \frac{\sigma_S^2}{1 + P/\sigma^2} = \frac{\sigma_S^2 \sigma^2}{P + \sigma^2}.$$

With $X = \sqrt{P/\sigma_S^2} \cdot S$, the received signal is $Y = \sqrt{P/\sigma_S^2} \cdot S + Z$. The MMSE estimate is $$\hat{S} = \frac{\sqrt{P/\sigma_S^2} \cdot \sigma_S^2}{\sigma_S^2 \cdot P/\sigma_S^2 + \sigma^2} Y = \frac{\sqrt{P \sigma_S^2}}{P + \sigma^2} Y.$$ The distortion is $$D_{\text{uncoded}} = \sigma_S^2 - \frac{P \sigma_S^2}{P + \sigma^2} = \frac{\sigma_S^2 \sigma^2}{P + \sigma^2}.$$

Remarkably, $D_{\text{uncoded}} = D_{\text{sep}}$! At bandwidth ratio $\kappa = 1$, uncoded linear transmission achieves the optimal distortion for a Gaussian source over a Gaussian channel. This is one of the rare cases where uncoded transmission is optimal — it exploits the fact that the Gaussian distribution is both the capacity-achieving input and the source distribution, and linear MMSE estimation is optimal for Gaussian signals.

For $\kappa \neq 1$, uncoded transmission is generally suboptimal, and the gap grows with $|\kappa - 1|$.

The Slepian–Wolf sum rate for jointly Gaussian sources with correlation $\rho$ is $$H(S_1, S_2) = \frac{1}{2}\log\bigl((2\pi e)^2 \sigma_S^4 (1-\rho^2)\bigr).$$ The MAC sum capacity is $$C_{\text{sum}} = \frac{1}{2}\log(1 + \text{SNR}_{1} + \text{SNR}_{2}).$$ At $\rho = 0.9$ and $\text{SNR}_{1} = \text{SNR}_{2} = 10$, $C_{\text{sum}} = \frac{1}{2}\log(21) \approx 2.19$ bits/use.

The minimum sum distortion under separation is achieved when the compression rates match the channel rates, giving a total distortion that accounts for both quantization and channel noise.

Each user transmits $X_k = \sqrt{P_k/\sigma_S^2} \cdot S_k$. The receiver observes $Y = \sqrt{P_1/\sigma_S^2} S_1 + \sqrt{P_2/\sigma_S^2} S_2 + Z$.

Using MMSE estimation (which is linear for jointly Gaussian signals), the receiver exploits the known correlation $\rho$ between $S_1$ and $S_2$ to improve the estimate of each source. The MMSE distortion for each source is $$D_k = \sigma_S^2 \left(1 - \frac{\text{SNR}_{k}(1-\rho^2) + \text{SNR}_{k}}{(1 + \text{SNR}_{1} + \text{SNR}_{2} + 2\rho\sqrt{\text{SNR}_{1} \text{SNR}_{2}})}\right).$$

At high correlation ($\rho = 0.9$), the uncoded scheme achieves lower distortion than separate coding because it implicitly exploits the source correlation through the MAC's superposition property. The MMSE decoder "sees" the correlation and uses it to separate the sources — something that separate coding, which discards the correlation at the compression stage, cannot do as effectively.

This demonstrates the failure of separation for correlated sources over the MAC.