Deep Denoisers as Implicit Priors

Deep Denoisers Transform PnP Performance

The advent of deep-learning denoisers (DnCNN, DRUNet, SwinIR) has dramatically improved PnP reconstruction quality. These networks capture complex image priors — textures, edges, semantic structure — that handcrafted priors (TV, wavelets, BM3D) cannot represent.

The central question is: how do we harness this expressive power while preserving the modularity and (ideally) the convergence guarantees of the PnP framework? This section reviews the three dominant architectures and the noise-scheduling strategy that bridges them to PnP.

Definition:

DnCNN — Residual Convolutional Denoiser

DnCNN (Zhang et al., 2017) trains a deep CNN fθf_\theta to predict the noise residual rather than the clean image:

Dσ(v)=v−fθ(v).\mathcal{D}_\sigma(\mathbf{v}) = \mathbf{v} - f_\theta(\mathbf{v}).

Architecture: 17 convolutional layers with batch normalisation and ReLU. No pooling or skip connections; fixed receptive field.

Training: Minimise ∑i∥fθ(vi)−ni∥2\sum_i \|f_\theta(\mathbf{v}_i) - \mathbf{n}_i\|^2 over noisy/clean image pairs at a fixed noise level σ\sigma.

Limitation for PnP: One model per noise level. In PnP iterations where σ\sigma varies, a separate model is needed per level or a noise-blind approximation is required.

Definition:

DRUNet — Noise-Level-Aware U-Net Denoiser

DRUNet (Zhang et al., 2021) conditions on the noise level σ\sigma by concatenating a constant channel σ⋅1\sigma \cdot \mathbf{1} to the input:

x^=Dθ(v, σ).\hat{\mathbf{x}} = \mathcal{D}_\theta(\mathbf{v},\, \sigma).

Architecture: Encoder–decoder U-Net with residual blocks at four scales ([64,128,256,512][64, 128, 256, 512] channels).

Training: Diverse natural images with noise levels σ∈[0,50]/255\sigma \in [0, 50]/255 drawn uniformly.

PnP advantage: The noise-level input allows the denoiser strength to be adjusted at each iteration without retraining. In PnP-ADMM, σ\sigma decreases across iterations as the estimate converges, implementing a coarse-to-fine strategy.

DRUNet achieves state-of-the-art denoising PSNR across all noise levels with a single model. Its noise-level conditioning makes it the default choice for PnP in practice.

Definition:

SwinIR — Transformer-Based Denoiser

SwinIR (Liang et al., 2021) uses Swin Transformer blocks with shifted-window self-attention instead of convolutions. The key architectural difference from DRUNet:

  • Non-local receptive field: attention over large windows captures long-range dependencies across the image
  • Shift invariance: unlike CNNs, the receptive field adapts to image content
  • PnP application: SwinIR achieves ∼0.2\sim 0.2–0.30.3 dB improvement over DRUNet at the cost of higher computational complexity per inference

For RF imaging applications, SwinIR's long-range attention is particularly beneficial for capturing the structured sparsity of RF scenes.

Example: Noise-Level Schedule for PnP-ADMM with DRUNet

Design a noise-level schedule {σk}k=1K\{\sigma_k\}_{k=1}^K for PnP-ADMM with DRUNet, where the denoiser strength decreases across iterations. Compare geometric and cosine schedules.

Solution
Motivation for a decreasing schedule

In early iterations, the estimate c(k)\mathbf{c}^{(k)} is far from the truth — the effective residual error is large, so the denoiser should be strong (large σ\sigma, heavy smoothing). As iterations progress, the estimate refines and the denoiser should be gentle (small σ\sigma) to preserve fine-scale features.

Geometric schedule

σk=σ1⋅γk−1,k=1,…,K\sigma_k = \sigma_1 \cdot \gamma^{k-1}, \qquad k = 1, \ldots, Kwithwith\gamma = (\sigma_K/\sigma_1)^{1/(K-1)}.∗∗Example:∗∗. **Example:**\sigma_1 = 49/255,,\sigma_K = 1/255,,K = 30::\gamma = (1/49)^{1/29} \approx 0.875$. The schedule decreases rapidly early and slowly at the end.

Cosine schedule

σk=σK+σ1−σK2 ⁣(1+cos⁡ ⁣(π(k−1)K−1)).\sigma_k = \sigma_K + \frac{\sigma_1 - \sigma_K}{2} \!\left(1 + \cos\!\left(\frac{\pi(k-1)}{K-1}\right)\right).Thisdecreasesslowlyatbothendpointsandrapidlyinthemiddle.Forsceneswithstructuredartefacts(commoninRFimaging),thecosinescheduleisoftenpreferredasitspendsmoreiterationsatintermediatenoiselevels.This decreases slowly at both endpoints and rapidly in the middle. For scenes with structured artefacts (common in RF imaging), the cosine schedule is often preferred as it spends more iterations at intermediate noise levels.\square$

Theorem: The Implicit Regulariser of a Deep Denoiser

If a denoiser Dσ\mathcal{D}_\sigma is the gradient of a scalar potential, i.e., Dσ(v)=∇Φσ(v)\mathcal{D}_\sigma(\mathbf{v}) = \nabla \Phi_\sigma(\mathbf{v}) for some Φσ ⁣:RN→R\Phi_\sigma \colon \mathbb{R}^N \to \mathbb{R}, then PnP-PGD minimises the explicit objective:

min⁡x  12∥y−Ax∥2+Rσ(x),\min_\mathbf{x} \; \frac{1}{2}\|\mathbf{y} - \mathbf{A}\mathbf{x}\|^2 + R_\sigma(\mathbf{x}),

where Rσ(x)=12∥x∥2−Φσ(x)R_\sigma(\mathbf{x}) = \frac{1}{2}\|\mathbf{x}\|^2 - \Phi_\sigma(\mathbf{x}) is the implicit regulariser induced by Dσ\mathcal{D}_\sigma.

The denoiser defines a "score function" ∇Φσ\nabla\Phi_\sigma that points toward regions of high image probability. The regulariser RσR_\sigma penalises images far from the denoiser's implicit manifold of natural images. Without the gradient-potential property, PnP may not be minimising any explicit objective — it is a fixed-point iteration whose limit depends on initialisation and the denoiser's geometry.

Proof
PnP-PGD fixed-point condition

At a fixed point x∗\mathbf{x}^* of PnP-PGD with step size α\alpha: x∗=Dσ(x∗−α∇f(x∗)),\mathbf{x}^* = \mathcal{D}_\sigma(\mathbf{x}^* - \alpha\nabla f(\mathbf{x}^*)), where f(x)=12∥y−Ax∥2f(\mathbf{x}) = \frac{1}{2}\|\mathbf{y} - \mathbf{A}\mathbf{x}\|^2.

Identify the regulariser via Tweedie's formula

When Dσ(v)=v−σ2∇Rσ(v)\mathcal{D}_\sigma(\mathbf{v}) = \mathbf{v} - \sigma^2\nabla R_\sigma(\mathbf{v}) (Tweedie's formula for score-based denoisers): x∗−σ2∇Rσ(x∗−α∇f(x∗))=x∗−α∇f(x∗)−σ2∇Rσ(⋯ ).\mathbf{x}^* - \sigma^2\nabla R_\sigma(\mathbf{x}^* - \alpha\nabla f(\mathbf{x}^*)) = \mathbf{x}^* - \alpha\nabla f(\mathbf{x}^*) - \sigma^2\nabla R_\sigma(\cdots). For small α\alpha, the fixed-point condition reduces to: α∇f(x∗)+σ2∇Rσ(x∗)≈0,\alpha\nabla f(\mathbf{x}^*) + \sigma^2\nabla R_\sigma(\mathbf{x}^*) \approx 0, which is the stationarity condition of f+(σ2/α)Rσf + (\sigma^2/\alpha) R_\sigma. ■\blacksquare

,

Historical Note: Tweedie's Formula and the Score Function Connection

1950s–present

Tweedie's formula, derived by Maurice Tweedie in the 1950s and popularised by Robbins (1956) in empirical Bayes, states that the MMSE estimate of x\mathbf{x} from v=x+σn\mathbf{v} = \mathbf{x} + \sigma\mathbf{n} is: E[x∣v]=v+σ2∇vlog⁡pσ(v).\mathbb{E}[\mathbf{x} | \mathbf{v}] = \mathbf{v} + \sigma^2 \nabla_\mathbf{v} \log p_\sigma(\mathbf{v}). This connects the MMSE denoiser to the score function ∇log⁡pσ\nabla\log p_\sigma. In diffusion models (Chapter 22), this identity is central to the reverse process. In PnP, it implies that a good denoiser approximates the score function of the image distribution, giving PnP a principled Bayesian interpretation even without an explicit prior.

Why This Matters: PnP for RF Imaging

PnP is particularly attractive for RF imaging because:

  1. Modular forward models: The sensing matrix A\mathbf{A} changes with array configuration, frequency, and scene geometry. PnP allows the same denoiser to be reused across all configurations.

  2. Complex-valued signals: RF images are complex (magnitude + phase). DRUNet can handle complex inputs by treating real and imaginary parts as separate channels.

  3. Limited training data: PnP denoisers are trained on natural images, which are abundant. This sidesteps the data-scarcity problem in RF imaging without requiring domain-specific training.

  4. Interpretability: Each PnP iteration has a clear physical meaning (enforce measurement consistency, then denoise), which aids debugging and validation.

See full treatment in PnP and RED for RF Imaging

DnCNN vs DRUNet vs BM3D Denoising Quality

Compare the denoising quality of BM3D, DnCNN, and DRUNet at various noise levels on a test image. The plot shows PSNR (dB) vs noise level σ\sigma for each denoiser. Observe that DRUNet consistently outperforms the others, especially at low noise levels where fine-detail preservation is critical.

Parameters
25

Quick Check

For a PnP algorithm to be provably minimising an explicit objective function, what property must the denoiser satisfy?

The denoiser must be a convolutional neural network.

The denoiser must be the gradient of a scalar potential function.

The denoiser must be non-expansive.

The denoiser must be trained end-to-end with the forward model.

Correction:
The denoiser must be the gradient of a scalar potential function.

If Dσ=∇Φσ\mathcal{D}_\sigma = \nabla\Phi_\sigma, the PnP fixed point is the stationary point of f(x)+Rσ(x)f(\mathbf{x}) + R_\sigma(\mathbf{x}) where RσR_\sigma is the implicit regulariser. Without this property, PnP may not be minimising any objective.

Key Takeaway

DRUNet with noise-level conditioning is the recommended denoiser for PnP, enabling adaptive denoising strength via a decreasing noise schedule. When the denoiser is a gradient potential, PnP minimises an explicit implicit-regulariser objective; otherwise, it remains a powerful empirical algorithm whose fixed points lack a clean variational interpretation.

The Plug-and-Play PrincipleConvergence Theory for PnP