Supervised Learning for the Physical Layer

Let $\mathbf{y} \in \mathbb{R}^m$ denote the received observation (e.g., pilot observations) and $\mathbf{x} \in \mathbb{R}^n$ the quantity to be estimated (e.g., channel coefficients, transmitted symbols). A supervised learning approach trains a parametric function $f_\theta : \mathbb{R}^m \to \mathbb{R}^n$ by minimising the empirical risk:

$$\hat{\theta} = \arg\min_\theta \; \frac{1}{N}\sum_{i=1}^{N} \ell\bigl(f_\theta(\mathbf{y}_i),\, \mathbf{x}_i\bigr)$$

where $\{(\mathbf{y}_i, \mathbf{x}_i)\}_{i=1}^N$ is the training dataset and $\ell(\cdot, \cdot)$ is the loss function. Common choices:

• MSE loss (regression): $\ell(\hat{\mathbf{x}}, \mathbf{x}) = \|\hat{\mathbf{x}} - \mathbf{x}\|^2$
• Cross-entropy loss (classification): $\ell(\hat{\mathbf{p}}, c) = -\log \hat{p}_c$ where $\hat{\mathbf{p}} = \mathrm{softmax}(f_\theta(\mathbf{y}))$

The function $f_\theta$ is typically a feed-forward neural network (fully connected, convolutional, or residual) trained via stochastic gradient descent (SGD) or Adam:

$$\theta \leftarrow \theta - \eta \, \nabla_\theta \ell\bigl(f_\theta(\mathbf{y}), \mathbf{x}\bigr)$$

where $\eta > 0$ is the learning rate.

Consider an OFDM system with $N_c$ subcarriers. Pilots are inserted at $N_p$ known subcarrier positions $\mathcal{P}$. The received pilot observations are:

$$\mathbf{y}_p = \mathrm{diag}(\mathbf{x}_p)\,\mathbf{h}_p + \mathbf{n}_p$$

where $\mathbf{h}_p = [H(k)]_{k \in \mathcal{P}}$ is the channel at pilot subcarriers, $\mathbf{x}_p$ are the known pilot symbols, and $\mathbf{n}_p \sim \mathcal{CN}(\mathbf{0}, \sigma^2\mathbf{I})$.

The LS estimate at pilots is $\hat{\mathbf{h}}_p^{\mathrm{LS}} = \mathrm{diag}(\mathbf{x}_p)^{-1}\mathbf{y}_p$, and the full channel is recovered by interpolation.

A neural-network estimator replaces the interpolation by a learned mapping:

$$\hat{\mathbf{h}} = f_\theta\!\left( [\operatorname{Re}(\hat{\mathbf{h}}_p^{\mathrm{LS}}),\; \operatorname{Im}(\hat{\mathbf{h}}_p^{\mathrm{LS}})] \right)$$

where $f_\theta$ is a two-layer fully connected network:

$$f_\theta(\mathbf{z}) = \mathbf{W}_2\, \sigma(\mathbf{W}_1\mathbf{z} + \mathbf{b}_1) + \mathbf{b}_2$$

with ReLU activation $\sigma(\cdot) = \max(0, \cdot)$ and parameters $\theta = \{\mathbf{W}_1, \mathbf{b}_1, \mathbf{W}_2, \mathbf{b}_2\}$. The network is trained to minimise the MSE:

$$\mathcal{L}(\theta) = \frac{1}{N} \sum_{i=1}^N \|\hat{\mathbf{h}}_i - \mathbf{h}_i\|^2$$

where $\mathbf{h}_i$ is the true channel (available during training from the simulator or calibration).

The autoencoder approach treats the entire communication system --- encoder (transmitter), channel, and decoder (receiver) --- as a single neural network that is trained end-to-end to minimise the symbol error probability.

Architecture. For an $M$-ary modulation scheme over $n$ channel uses:

1. Encoder $f_{\theta_{\text{enc}}} : \{1, \ldots, M\} \to \mathbb{R}^{2n}$: Maps each message $s \in \{1, \ldots, M\}$ (represented as a one-hot vector) to a transmitted signal $\mathbf{x} = f_{\theta_{\text{enc}}}(s)$, subject to a power constraint $\frac{1}{M}\sum_{s=1}^M \|\mathbf{x}_s\|^2 \leq 1$.

2. Channel $p(\mathbf{y}|\mathbf{x})$: The physical channel (AWGN, Rayleigh fading, etc.) adds noise. For AWGN: $\mathbf{y} = \mathbf{x} + \mathbf{n}$, $\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \sigma^2\mathbf{I})$.

3. Decoder $g_{\theta_{\text{dec}}} : \mathbb{R}^{2n} \to \Delta^{M-1}$: Maps the received signal to a probability distribution over messages via softmax, and the estimated message is $\hat{s} = \arg\max_s \, g_{\theta_{\text{dec}}} (\mathbf{y})_s$.

Training. The system is trained by minimising the categorical cross-entropy:

$$\mathcal{L}(\theta_{\text{enc}}, \theta_{\text{dec}}) = -\frac{1}{N}\sum_{i=1}^N \log g_{\theta_{\text{dec}}} (\mathbf{y}_i)_{s_i}$$

Gradients pass through the channel layer: for AWGN, the channel is a simple addition node with a deterministic gradient of $\partial\mathbf{y}/\partial\mathbf{x} = \mathbf{I}$.