Prerequisites & Notation

Prerequisites for Chapter 19

This chapter extends the AMP/VAMP framework from Chapters 17–18 in two directions: (1) unknown hyperparameters — EM-GAMP estimates noise variance and sparsity from data automatically; (2) non-Gaussian likelihoods — GAMP handles 1-bit, Poisson, and power-only measurements via a general output channel $g_{\text{out}}$ . A third extension, multi-layer inference, connects message passing to deep generative priors.

AMP and OAMP/VAMP (Chapter 17)(Review ch17)
Self-check: Can you write the AMP iteration, state evolution recursion, and name one advantage of VAMP over AMP for structured matrices?
Expectation-Maximization (EM) algorithm
Self-check: Can you state the E-step and M-step for a latent variable model and explain why EM is guaranteed to not decrease the log-likelihood?
Bernoulli-Gaussian prior and MMSE denoiser(Review ch17)
Self-check: Can you compute the posterior inclusion probability $\pi_i$ and posterior mean for a Bernoulli-Gaussian prior given a noisy observation?
Generalized linear models (GLM)
Self-check: Can you write the GLM likelihood $p(y_m | z_m)$ for Gaussian, Poisson, and logistic output channels?

Notation for Chapter 19

Symbols used throughout this chapter. The sensing model $\mathbf{y} = \mathbf{A}\mathbf{c} + \mathbf{w}$ carries through all three sections with different likelihood structures.

Symbol	Meaning	Introduced
$z_m = \mathbf{a}_m^T \mathbf{c}$	Linear mixing output (pre-likelihood) at measurement $m$	s01
$p(y_m \| z_m)$	Output channel (generalized likelihood)	s01
$g_{\text{out}}(y, \hat{p}, \tau_p)$	Output function: MMSE of $z_m$ given $y_m$ and Gaussian cavity $\mathcal{N}(\hat{p}, \tau_p)$	s01
$g_{\text{in}}(r, \tau_r)$	Input function (prior denoiser): MMSE of $x_i$ given $\mathcal{N}(r, \tau_r)$	s01
$\boldsymbol{\theta} = (\sigma^2, \rho, \sigma_x^2)$	Unknown hyperparameters: noise variance, sparsity, signal variance	s01
$\hat{\boldsymbol{\theta}}^{(k)}$	EM estimate of hyperparameters at outer iteration $k$	s01
$\pi_i$	Posterior inclusion probability: $p(x_i \neq 0 \mid \hat{r}_i, \tau_r)$	s01
$\tau_p^t, \tau_r^t$	GAMP extrinsic variances at output and input sides, iteration $t$	s01
$\Phi(\cdot)$	Standard Gaussian CDF (used in probit / 1-bit likelihood)	s02
$\mathbf{z}^{(\ell)}$	Hidden activations at layer $\ell$ of a multi-layer generative model	s03
$\mathbf{A}^{(\ell)}$	Linear mixing matrix at layer $\ell$	s03

← Ch 18 EM-GAMP for RF Imaging