Ferkans — Interactive Telecom Tutor

The Denoiser Is the Knob You Turn

OAMP separates the reconstruction problem into two independent modules: the LMMSE step (which depends only on $\mathbf{A}$ and $\sigma^2$ ) and the denoiser step (which depends only on the signal prior). The LMMSE step is fixed once the measurement system is designed. All the modeling flexibility lives in the denoiser.

This modularity is OAMP's greatest strength: we can plug in any denoiser — from classical soft thresholding to a pre-trained neural network — without modifying the rest of the algorithm. State evolution still tracks the performance, provided the denoiser satisfies mild regularity conditions (Lipschitz continuity, convergent divergence).

Definition:
Soft-Thresholding Denoiser

The soft-thresholding denoiser with threshold $\lambda$ is

$\eta_{\text{ST}}(r; \lambda) = r \cdot \max\!\bigl(1 - \lambda/|r|,\; 0\bigr),$

applied component-wise. For complex-valued $r$ , this shrinks the magnitude toward zero while preserving the phase.

Divergence: $\text{div}(\eta_{\text{ST}}) = \frac{1}{N} \#\{i : |r_i| > \lambda\}$ — the fraction of active (non-thresholded) components.

Soft thresholding is the MAP denoiser for a Laplace prior ( $p_0(c) \propto e^{-\lambda|c|}$ ). The threshold $\lambda$ should be set as a function of the effective noise variance $v_1^t$ ; the SE-optimal choice is $\lambda^* = \sqrt{2 v_1^t \log(1/\rho)}$ for sparsity $\rho$ .

Definition:
Bernoulli-Gaussian MMSE Denoiser

For the Bernoulli-Gaussian prior $p_0(c) = (1-\rho)\,\delta(c) + \rho\,\mathcal{CN}(0, \sigma_c^2)$ , the Bayes-optimal (posterior mean) denoiser for the observation $r = c + \sqrt{v}\,z$ is

$\eta_{\text{BG}}(r; v, \rho, \sigma_c^2) = \frac{\rho\,\sigma_c^2}{\sigma_c^2 + v}\,r \cdot \frac{1}{1 + \frac{1-\rho}{\rho} \frac{\sigma_c^2 + v}{v} \exp\!\bigl(-\frac{\sigma_c^2\,|r|^2} {v(\sigma_c^2 + v)}\bigr)}.$

Divergence:

$\text{div}(\eta_{\text{BG}}) = \frac{1}{N}\sum_{i=1}^N \frac{\partial [\eta_{\text{BG}}(r_i)]_i}{\partial r_i},$

which has a closed-form expression involving the posterior probability of activity $\pi_i = \mathbb{P}(c_i \neq 0 \mid r_i)$ .

The BG-MMSE denoiser achieves the Bayes-optimal MSE when the prior is correctly specified. It is the default choice for OAMP in RF imaging when the scene is known to be sparse.

Theorem: Bayes-Optimal OAMP

When the denoiser $\eta_t$ is the posterior mean (MMSE denoiser) under the true prior $p_0$ , OAMP achieves the Bayes-optimal MSE — the minimum MSE achievable by any estimator — among all algorithms that use only the singular values of $\mathbf{A}$ (not the specific structure of $\mathbf{V}$ ) and operate iteratively.

Formally, the OAMP state evolution fixed point satisfies

$v_2^* = \text{mmse}(p_0, v_1^*),$

where $v_1^*$ is the LMMSE MSE at the fixed point, and this matches the replica prediction from statistical physics.

OAMP with the optimal denoiser extracts all the information that any iterative algorithm can extract from the singular value distribution of $\mathbf{A}$ and the prior $p_0$ . The only way to do better is to exploit specific structure in the right singular vectors $\mathbf{V}$ (which OAMP treats as Haar-random by assumption).

Proof

State evolution fixed-point equations

At the fixed point, $v_1^* = F(v_2^*)$ (LMMSE MSE as a function of prior variance) and $v_2^* = G(v_1^*)$ (denoiser MSE). With the MMSE denoiser, $G(v) = \text{mmse}(p_0, v)$ .

Connection to replica prediction

The replica method from statistical physics predicts the same fixed-point equations for the Bayes-optimal MSE of any right-rotationally invariant sensing system. OAMP's state evolution matches this prediction, confirming its optimality. $\blacksquare$

From Hand-Crafted to Learned Denoisers

The modularity of OAMP invites a powerful idea: replace the hand-crafted denoiser with a neural network trained for image denoising. The LMMSE step ensures that the denoiser input is approximately $\mathbf{c} + \sqrt{v_1^t}\,\mathbf{z}$ — a standard Gaussian denoising problem. Any denoiser trained to remove Gaussian noise at level $\sqrt{v_1^t}$ can be plugged in.

Popular choices:

DnCNN (Zhang et al., 2017): A residual CNN trained to predict the noise. Fast, effective, and well-suited to natural images.
U-Net: An encoder-decoder architecture with skip connections, effective for images with multi-scale structure.
DRUNet (Zhang et al., 2021): A noise-level-aware U-Net that accepts $\sqrt{v_1^t}$ as an input channel, enabling a single network to denoise at all noise levels encountered during OAMP iterations.

The Hutchinson estimator (DHutchinson Trace Estimator) provides the divergence needed for the MSE update, since the network Jacobian is not available in closed form.

,

🎓CommIT Contribution(2024)

OAMP with Learned Denoisers for RF Imaging

G. Caire, A. Rezaei — npj Imaging

The CommIT group demonstrated that replacing the BG-MMSE denoiser in OAMP with a trained DnCNN improves RF image reconstruction by 3–5 dB NMSE for scenes with non-sparse structure (e.g., extended targets, smooth surfaces).

Key findings:

A noise-level-conditional DnCNN, trained on a dataset of RF reflectivity maps, consistently outperforms BG-MMSE for non-sparse scenes.
For truly sparse scenes (few point scatterers), BG-MMSE matches or slightly outperforms the learned denoiser — the prior is correctly specified and hard to beat.
The Hutchinson divergence estimator introduces negligible overhead ( $< 5\%$ ) compared to the denoiser evaluation itself.
State evolution with the empirical MSE of the learned denoiser accurately predicts the algorithm's convergence trajectory.

This work bridges the classical message-passing framework of this chapter with the deep unfolding approach of Chapter 27.

learned denoiserDnCNNRF imagingOAMPCommIT

Denoiser Comparison in OAMP

Compare OAMP convergence with different denoisers: soft thresholding, BG-MMSE, and a simulated learned denoiser. The learned denoiser excels for extended (non-sparse) targets.

Parameters

Problem size

N

1024

\delta

0.4

Sparsity

\rho

(BG scenes only)0.1

SNR (dB)25

Scene type

Denoiser Properties for OAMP

Denoiser	Prior assumption	Divergence	Best for	Limitation
Soft threshold	Laplace (sparse)	Closed form (fraction active)	Very sparse signals	Bias on large coefficients; suboptimal threshold
BG-MMSE	Bernoulli-Gaussian	Closed form	Sparse scenes with known statistics	Assumes specific parametric prior
Minimax	Worst-case over $\ell_2$ ball	Analytical	Unknown prior, robust recovery	Conservative; ignores prior structure
DnCNN	Learned from data	Hutchinson estimate	Natural images, extended targets	Requires training data; generalization to unseen scenes
U-Net / DRUNet	Learned from data (noise-aware)	Hutchinson estimate	Multi-scale structure, varying noise	Larger model; slower per iteration

Example: Choosing a Denoiser for an RF Imaging Scenario

An ISAC base station images an urban scene containing:

3 strong point scatterers (vehicles) at unknown positions,
Extended building facades producing distributed reflections,
Background clutter.

The scene has $N = 4096$ voxels, $M = 1600$ measurements, $\text{SNR} = 20\,\text{dB}$ .

Which denoiser should be used in OAMP?

Solution

Scene characteristics

The scene is not purely sparse — it contains extended targets alongside point scatterers. A Bernoulli-Gaussian prior is a poor model for the building facades.

Denoiser choice

A noise-level-conditional U-Net (DRUNet) trained on a dataset of urban RF reflectivity maps is the best choice:

It handles both point scatterers (which it learns to preserve) and extended structures (which it smooths appropriately).
The noise-level input $\sqrt{v_1^t}$ adapts the denoiser behavior across OAMP iterations.

Fallback

If no training data is available, BG-MMSE with conservatively chosen sparsity ( $\rho = 0.15$ , higher than the true point-scatterer fraction) provides a reasonable baseline. It will over-smooth the building facades but correctly recover the point scatterers.

OAMP Iteration — LMMSE and Denoiser Alternating

Visualization of the OAMP iteration on a 2D RF imaging problem. The left panel shows the LMMSE output (noisy but unbiased), and the right panel shows the denoiser output (denoised but potentially biased). Watch the estimate converge to the true scene over 15 iterations.

OAMP alternates LMMSE estimation (left, exploiting

\mathbf{A}

) and denoising (right, exploiting the prior). The effective noise decreases at each iteration as the state evolution predicts.

Historical Note: D-AMP — The Bridge to Learned Denoisers

2016–2017

The idea of plugging a generic denoiser into AMP was pioneered by Metzler, Maleki, and Baraniuk (2016) as D-AMP (Denoising-based AMP). They showed that BM3D — a non-local means denoiser originally designed for natural image denoising — could be used as the AMP denoiser, dramatically improving reconstruction quality for natural images.

D-AMP used the Stein divergence estimator to compute the Onsager correction for the black-box denoiser. However, D-AMP inherited AMP's limitation to i.i.d. Gaussian matrices. The combination of D-AMP's denoiser flexibility with OAMP's matrix generality yields the learned OAMP framework described in this section.

Common Mistake: The Denoiser Must Be Matched to the Effective Noise Level

Mistake:

Using a denoiser trained at a fixed noise level $\sigma_0$ for all OAMP iterations, even though the effective noise $\sqrt{v_1^t}$ changes at each iteration (typically decreasing from a large initial value to a small final value).

Correction:

The denoiser must be noise-level-aware. Options:

Train a separate denoiser for each noise level (wasteful).
Use a noise-level-conditional network (DRUNet): feed $\sqrt{v_1^t}$ as an extra input channel.
Use a denoiser with an explicit noise parameter (BG-MMSE: the parameter $v$ is set to $v_1^t$ at each iteration).

A mismatched noise level causes the denoiser to either over-smooth (if it thinks the noise is too high) or leave residual noise (if it thinks the noise is too low), degrading the OAMP fixed point.

Quick Check

OAMP with BG-MMSE denoiser achieves NMSE = $-22$ dB on a sparse Bernoulli-Gaussian scene. Replacing BG-MMSE with a learned DnCNN denoiser is most likely to improve performance when:

The scene is truly Bernoulli-Gaussian and the prior parameters are known

The scene contains extended targets (building facades, ground surface) not well modeled by a sparse prior

The SNR is very high (>40 dB)

Correction:

The scene contains extended targets (building facades, ground surface) not well modeled by a sparse prior

Learned denoisers excel when the true scene statistics deviate from the parametric prior assumed by analytical denoisers. Extended targets violate the sparsity assumption and are better handled by a network trained on realistic scenes.

Key Takeaway

OAMP's modularity allows any denoiser to be plugged in: from classical soft thresholding (which gives LASSO) to Bayes-optimal BG-MMSE (which achieves the information-theoretic limit for sparse signals) to learned neural denoisers (which handle realistic, non-sparse RF scenes). The choice of denoiser is the primary modeling decision in OAMP-based RF imaging.

Denoiser Design for Imaging

The Denoiser Is the Knob You Turn

Definition: Soft-Thresholding Denoiser

Definition: Bernoulli-Gaussian MMSE Denoiser

Theorem: Bayes-Optimal OAMP

State evolution fixed-point equations

Connection to replica prediction

From Hand-Crafted to Learned Denoisers

OAMP with Learned Denoisers for RF Imaging

Denoiser Comparison in OAMP

Parameters

Denoiser Properties for OAMP

Example: Choosing a Denoiser for an RF Imaging Scenario

Scene characteristics

Denoiser choice

Fallback

OAMP Iteration — LMMSE and Denoiser Alternating

Historical Note: D-AMP — The Bridge to Learned Denoisers

Common Mistake: The Denoiser Must Be Matched to the Effective Noise Level

Quick Check

Key Takeaway

Definition:
Soft-Thresholding Denoiser

Definition:
Bernoulli-Gaussian MMSE Denoiser