Ferkans — Interactive Telecom Tutor

Beyond Additive Gaussian Noise

The standard RF imaging model $\mathbf{y} = \mathbf{A}\mathbf{c} + \mathbf{w}$ with $\mathbf{w} \sim \mathcal{CN}(\mathbf{0}, \sigma^2\mathbf{I})$ is a special case of a much broader class: Generalized Linear Models (GLMs).

Three RF-relevant scenarios require non-Gaussian output channels:

1-bit receivers: Low-cost hardware quantizes each measurement to a single bit: $y_m = \text{sign}(\Re\{z_m\})$ . The measurement discards all amplitude information — only the sign survives.
Power-only / envelope detection: $y_m = |z_m|^2 + \text{noise}$ . Used in passive coherent location and angle-of-arrival systems.
Photon-counting / shot noise: $y_m \sim \text{Poisson}(\lambda_m)$ where $\lambda_m = |z_m|^2$ . Relevant for optical/THz coherent sensing.

GAMP handles all these via a single interface: the output function $g_{\text{out}}(y_m, \hat{p}_m, \tau_p)$ replaces the Gaussian residual computation and encapsulates the non-standard likelihood.

Definition:
Generalized Linear Model (GLM)

A generalized linear model (GLM) for RF imaging has the form:

$y_m \sim p(y_m \mid z_m), \quad z_m = \mathbf{a}_m^T \mathbf{c}, \quad x_q \sim p_0(x_q),$

where:

$p(y_m \mid z_m)$ is the output channel (likelihood), a function of the linear mixing output $z_m$ alone (conditional independence across $m$ ).
$p_0(x_q)$ is the input channel (prior) on each scene element.
The posterior is $p(\mathbf{c} \mid \mathbf{y}) \propto \prod_q p_0(x_q) \cdot \prod_m p(y_m \mid \mathbf{a}_m^T \mathbf{c})$ .

The Gaussian case $p(y \mid z) = \mathcal{CN}(y; z, \sigma^2)$ is the standard CS model. GAMP applies to any GLM where the output channel admits efficient computation of the posterior mean and variance.

Definition:
GAMP Output Function $g_{\text{out}}$

For any output channel $p(y \mid z)$ , define the output function:

$g_{\text{out}}(y, \hat{p}, \tau_p) \triangleq \frac{\hat{z}_{\text{post}} - \hat{p}}{\tau_p}, \quad \tau_s \triangleq -\frac{\partial g_{\text{out}}}{\partial \hat{p}},$

where $\hat{z}_{\text{post}} = \mathbb{E}[z \mid y, z \sim \mathcal{N}(\hat{p}, \tau_p)]$ is the posterior mean of $z$ under the combined channel:

$p(z \mid y, \hat{p}, \tau_p) \propto p(y \mid z)\,\mathcal{N}(z; \hat{p}, \tau_p).$

Interpretation: $g_{\text{out}}$ computes the MMSE estimate of $z$ given the measurement $y_m$ and a Gaussian "cavity" distribution $\mathcal{N}(\hat{p}, \tau_p)$ representing the extrinsic belief on $z$ from all other variables. The ratio $(\hat{z}_{\text{post}} - \hat{p})/\tau_p$ is the normalized innovation.

Example: Output Function for Gaussian Likelihood

Derive $g_{\text{out}}(y, \hat{p}, \tau_p)$ for $p(y \mid z) = \mathcal{N}(y; z, \sigma^2)$ . Verify that GAMP reduces to standard AMP in this case.

Solution

Compute posterior distribution

The posterior $p(z \mid y, \hat{p}, \tau_p) \propto \mathcal{N}(y; z, \sigma^2) \cdot \mathcal{N}(z; \hat{p}, \tau_p)$ is Gaussian with mean and variance:

$\hat{z}_{\text{post}} = \frac{\tau_p\,y + \sigma^2\,\hat{p}}{\tau_p + \sigma^2}, \quad \tau_{\text{post}} = \frac{\tau_p\,\sigma^2}{\tau_p + \sigma^2}.$

Compute $g_{ ext{out}}$

$g_{\text{out}}(y, \hat{p}, \tau_p) = \frac{\hat{z}_{\text{post}} - \hat{p}}{\tau_p} = \frac{y - \hat{p}}{\tau_p + \sigma^2},$ $

which is exactly the AMP residual divided by the total variance.

Verify reduction to AMP

For i.i.d. Gaussian $\mathbf{A}$ with $\bar{a}^2 = 1/M$ , $\tau_p = \tau_x / \delta$ and $g_{\text{out}} = (y - \hat{p})/(\sigma^2 + \tau_p)$ . The GAMP output step becomes:

$\hat{s}_m = \frac{y_m - \hat{p}_m}{\sigma^2 + \tau_p},$

which is precisely AMP's residual computation. GAMP reduces to standard AMP. $\blacksquare$

Example: Output Function for 1-Bit (Probit) Likelihood

Derive $g_{\text{out}}$ for the 1-bit measurement model: $p(y \mid z) = \Phi(y\,z/\sigma_{\text{dither}})$ , where $y \in \{+1, -1\}$ and $\Phi(\cdot)$ is the Gaussian CDF (probit link).

Solution

Posterior mean via Gaussian integral

The cavity distribution $\mathcal{N}(z; \hat{p}, \tau_p)$ combined with the probit likelihood gives:

$p(z \mid y, \hat{p}, \tau_p) \propto \Phi\!\left(\frac{yz}{\sigma}\right) \mathcal{N}(z; \hat{p}, \tau_p).$

The posterior mean is:

$\hat{z}_{\text{post}} = \hat{p} + \tau_p\,y\,\frac{\phi\!\left(y\hat{p}/\sqrt{\sigma^2 + \tau_p}\right)} {\Phi\!\left(y\hat{p}/\sqrt{\sigma^2 + \tau_p}\right)},$

where $\phi$ is the standard Gaussian PDF (the inverse Mills ratio).

Output function and variance

$g_{\text{out}} = y\,\frac{\phi(u)}{\Phi(u\,y) + \epsilon}\cdot\frac{1}{\sqrt{\sigma^2 + \tau_p}}, \quad u = \frac{\hat{p}}{\sqrt{\sigma^2 + \tau_p}}.$ $This is computed via the **Mills ratio**$ \phi(x)/\Phi(x)$, which is numerically stable via the scipy.special.erfcx function.

,

Output Channel

In a generalized linear model (GLM), the output channel $p(y_m \mid z_m)$ specifies the conditional distribution of the measurement $y_m$ given the linear mixing output $z_m = \mathbf{a}_m^T\mathbf{x}$ . In standard compressed sensing, the output channel is Gaussian. GAMP can handle any output channel for which the posterior mean $\mathbb{E}[z \mid y, z \sim \mathcal{N}(\hat{p}, \tau_p)]$ is computable.

Mills Ratio

The Mills ratio (or hazard function of the Gaussian) is $\phi(x)/\Phi(x)$ , where $\phi$ is the standard Gaussian PDF and $\Phi$ is the CDF. It appears in the GAMP output function for probit (1-bit) and truncated-Gaussian likelihoods. Numerically, it is computed as $\sqrt{2}\,\text{erfcx}(x/\sqrt{2})\,e^{-x^2/2}/2$ for stability in the tail region $x \gg 1$ .

GAMP Output Channels for RF Imaging

Likelihood Model	Expression $p(y\|z)$	$g_{\text{out}}$ Key Formula	RF Application
Gaussian	$\mathcal{N}(y; z, \sigma^2)$	$(y - \hat{p})/(\sigma^2 + \tau_p)$	Standard CS, homodyne radar
1-bit (probit)	$\Phi(yz/\sigma)$ , $y\in\{+1,-1\}$	Mills ratio (inverse): $y\phi(u)/\Phi(yu)$	Low-cost ADC, compressive receivers
Power-only	$p(y\|z) \propto e^{-y/\|z\|^2}/\|z\|^2$	Truncated Gaussian posterior	Phase-less radar, passive sensing
Poisson	$e^{yz - e^z}/y!$	Laplace approx: $y - e^{\hat{p}+\tau_p/2}$	Photon-counting, THz sensing
Logistic (binary)	$\sigma(yz) = 1/(1+e^{-yz})$	Probit approximation	Occupancy grid mapping

Theorem: State Evolution for GAMP

Under i.i.d. Gaussian $\mathbf{A}$ with i.i.d. entries $\mathcal{CN}(0, 1/M)$ , as $M, N \to \infty$ with $M/N \to \delta$ , the GAMP state variables satisfy:

$\tau_p^t = \frac{\tau_x^t}{\delta}, \quad \tau_s^t = \mathbb{E}_{Y,Z,W}\!\left[-\frac{\partial g_{\text{out}}}{\partial\hat{p}}(Y, Z + \sqrt{\tau_p^t}\,W, \tau_p^t)\right],$

$(\tau_r^{t+1})^{-1} = \delta\,\tau_s^t, \quad \tau_x^{t+1} = \mathbb{E}_{X_0, Z}\!\left[\left(g_{\text{in}}\!\left(X_0 + \sqrt{\tau_r^{t+1}}\,Z, \tau_r^{t+1}\right) - X_0\right)^2\right],$

where expectations are over the joint distributions $(Y, Z) \sim p_0 \times p(\cdot | z)$ and $W, Z \sim \mathcal{CN}(0,1)$ .

State evolution predicts the GAMP MSE without running the algorithm: it reduces the high-dimensional problem to a pair of scalar recursions. For non-Gaussian likelihoods, the effective noise seen by the input denoiser is determined by the average Jacobian of $g_{\text{out}}$ — a scalar quantity that captures the "information content" of one measurement.

Show Hint

The proof follows by showing that each GAMP residual converges in distribution to an AWGN channel with variance given by SE.

The key step is the Onsager-correction term, which decorrelates the effective observations.

Proof

Reduction to equivalent scalar channels

By the rotational invariance argument (Bayati & Montanari 2011, extended to GLMs by Rangan 2011), the GAMP iterates $\hat{r}_i^t$ are asymptotically distributed as:

$\hat{r}_i^t \xrightarrow{d} x_i + \sqrt{\tau_r^t}\,Z, \quad Z \sim \mathcal{CN}(0,1).$

Output variance via Jacobian average

Similarly, $\hat{p}_m^t \sim z_m + \sqrt{\tau_p^t}\,W$ and the SE for $\tau_s^t$ follows from the average Jacobian of $g_{\text{out}}$ over the randomness in $(y_m, W)$ . $\blacksquare$

Why This Matters: 1-Bit CS for Low-Cost RF Imaging Receivers

High-resolution ADCs (12–16 bits) consume significant power: roughly 1 mW/GHz per bit of resolution. For a wideband radar with $W = 1$ GHz, a 12-bit ADC at Nyquist rate consumes $\sim 12$ mW — per receiver channel.

1-bit receivers replace each ADC with a single comparator (< 0.1 mW), enabling dense receiver arrays at a fraction of the power. The GAMP output function for the probit model recovers near-standard performance at oversampling ratios $M/N \geq 2$ – $3$ , with a penalty of $\sim 2$ dB compared to full-precision measurements.

This is the core design principle behind the CommIT group's compressive RF imaging receivers in Chapter 11.

See full treatment in Multi-View, Multi-Frequency Sensing Geometry

1-Bit CS: GAMP vs Mismatched Gaussian Approximation

Reconstruction NMSE as a function of oversampling ratio $M/N$ for the 1-bit measurement model. GAMP (correct model) uses the probit output function; Mismatched GAMP treats the signed measurements as linear Gaussian observations.

At high oversampling ( $M/N > 3$ ), both approaches converge; at low oversampling ( $M/N < 1.5$ ), the model-matched GAMP gives 5–10 dB advantage.

Parameters

Sparsity

\rho

0.1

SNR (dB, dithering)20

GAMP Convergence for Different Likelihood Models

NMSE versus iteration for three likelihood models: Gaussian (standard CS), 1-bit, and Poisson. The solid curve uses the correct GAMP output function; the dashed curve shows mismatched GAMP (always uses Gaussian likelihood).

At convergence, the model-matched GAMP reaches the Bayes-optimal MSE predicted by state evolution; mismatched GAMP converges to a higher floor.

Parameters

Likelihood model

Oversampling ratio

M/N

1.5

Common Mistake: GAMP Convergence Is Fragile for Non-Log-Concave Likelihoods

Mistake:

GAMP's convergence guarantees (via state evolution) require the output channel to be log-concave in $z$ — i.e., $-\log p(y|z)$ is convex. For Poisson, power-only, and logistic channels this fails for certain measurement regimes, creating multiple Bethe free energy minima.

Symptoms: oscillation in the residual, divergence of $\tau_s^t$ , or NMSE stuck well above the SE prediction.

Correction:

(1) Damp the GAMP updates: $\hat{s}_m^{t+1} \leftarrow \alpha g_{\text{out}} + (1-\alpha)\hat{s}_m^t$ with $\alpha = 0.5$ – $0.8$ . (2) For structured sensing matrices (real-world physical arrays, not i.i.d. Gaussian), switch to the VAMP framework (Chapter 18) which has more robust convergence properties. (3) Monitor $\tau_s^t$ : if it becomes negative or exceeds $1/\tau_p$ , reset to the Gaussian output function temporarily.

🔧Engineering Note

1-Bit ADC Implementation Constraints

Practical 1-bit compressed sensing receivers use a comparator with a programmable threshold (dithering) to break the symmetry of the sign measurement. Without dithering, signed measurements cannot distinguish between $x$ and $2x$ , making amplitude recovery impossible.

The dither signal is typically a pseudo-random sequence with known statistical properties, which enters the probit model as $\sigma_{\text{dither}}$ . Optimal dither variance is $\sigma_{\text{dither}}^2 \approx 0.5\,\text{Var}[z_m]$ , striking a balance between information preservation and quantization distortion.

Practical Constraints

•
Comparator latency: 1–5 ps at GHz rates (vs. 50–100 ps for 12-bit ADC)
•
Power: < 1 mW/GHz per comparator vs. ~12 mW/GHz for 12-bit ADC
•
Dither must be synchronized across all receiver channels for coherent processing
•
GAMP with probit model requires $M/N \geq 2$ for reliable recovery at SNR = 20 dB

Historical Note: 1-Bit Compressed Sensing: Origins

2008–2015

The 1-bit compressed sensing problem was introduced by Boufounos and Baraniuk in 2008 (IEEE ICASSP), who showed that sparse signals are recoverable from their sign measurements alone. The key insight was that the sign function preserves the geometry of sparse vectors in a metric space sense.

Optimal Bayesian recovery via GAMP for 1-bit measurements was developed by Schniter and Rangan (2015, IEEE Trans. SP), who derived the probit output function and showed that GAMP achieves near-oracle MSE even for $M/N < 1$ when the signal is sparse enough.

Quick Check

For Gaussian likelihood $p(y|z) = \mathcal{N}(y; z, \sigma^2)$ , the GAMP output function $g_{\text{out}}(y, \hat{p}, \tau_p)$ evaluates to:

$(y - \hat{p})/\tau_p$

$(y - \hat{p})/(\sigma^2 + \tau_p)$

$(y - \hat{p})/\sigma^2$

$\Phi((y - \hat{p})/\sqrt{\sigma^2+\tau_p})$

Correction:

(y - \hat{p})/(\sigma^2 + \tau_p)

Correct. The posterior mean of $z$ given $y$ and cavity $\mathcal{N}(\hat{p},\tau_p)$ is $(\tau_p y + \sigma^2\hat{p})/(\sigma^2+\tau_p)$ . Subtracting $\hat{p}$ and dividing by $\tau_p$ gives $(y - \hat{p})/(\sigma^2+\tau_p)$ . This is exactly the AMP residual.

Key Takeaway

GLMs extend the CS model by replacing the Gaussian noise assumption with any separable output channel $p(y_m | z_m)$ . GAMP handles GLMs via a single interface: the output function $g_{\text{out}}$ , which computes the posterior mean of $z_m$ under the combined cavity + likelihood distribution. For 1-bit receivers, the probit $g_{\text{out}}$ (Mills ratio) recovers near-standard performance at 2–3× oversampling — enabling dense, low-power receiver arrays in RF imaging systems.

Generalized Linear Models

Beyond Additive Gaussian Noise

Definition: Generalized Linear Model (GLM)

Definition: GAMP Output Function goutg_{\text{out}}gout​

Example: Output Function for Gaussian Likelihood

Compute posterior distribution

Compute $g_{ ext{out}}$

Verify reduction to AMP

Example: Output Function for 1-Bit (Probit) Likelihood

Posterior mean via Gaussian integral

Output function and variance

Output Channel

Mills Ratio

GAMP Output Channels for RF Imaging

Theorem: State Evolution for GAMP

Reduction to equivalent scalar channels

Output variance via Jacobian average

Why This Matters: 1-Bit CS for Low-Cost RF Imaging Receivers

1-Bit CS: GAMP vs Mismatched Gaussian Approximation

Parameters

GAMP Convergence for Different Likelihood Models

Parameters

Common Mistake: GAMP Convergence Is Fragile for Non-Log-Concave Likelihoods

1-Bit ADC Implementation Constraints

Historical Note: 1-Bit Compressed Sensing: Origins

Quick Check

Key Takeaway

Definition:
Generalized Linear Model (GLM)

Definition:
GAMP Output Function $g_{\text{out}}$