Ferkans — Interactive Telecom Tutor

A Reconstruction Without Uncertainty Is Incomplete

A point estimate $\hat{\boldsymbol{\gamma}}$ (MAP or MMSE) tells us what we think the scene looks like, but not how confident we are. In safety-critical applications — autonomous driving radar, non-destructive testing, medical imaging — decision-makers need to know which features of the reconstruction are reliable and which are uncertain. Declaring a scatterer present when the posterior assigns only 60% probability to that pixel is fundamentally different from 99% probability.

Uncertainty quantification (UQ) extracts this confidence information from the posterior distribution, turning a Bayesian model into actionable guidance for the engineer or clinician. A reconstruction without uncertainty bars is, in this sense, scientifically incomplete.

Definition:
Credible Intervals and Credible Regions

A $(1-\alpha)$ credible interval for a scalar quantity $\phi(\boldsymbol{\gamma})$ (e.g., a single pixel value) is an interval $[a, b]$ such that

$P\bigl(\phi(\boldsymbol{\gamma}) \in [a, b] \mid \mathbf{y}\bigr) = 1 - \alpha.$

Common choices:

Highest posterior density (HPD): The shortest interval containing probability $1-\alpha$ . For unimodal posteriors, it is symmetric about the mode.
Equal-tailed: $P(\phi < a \mid \mathbf{y}) = P(\phi > b \mid \mathbf{y}) = \alpha/2$ .

For the Gaussian posterior $\boldsymbol{\gamma} \mid \mathbf{y} \sim \mathcal{N}(\hat{\boldsymbol{\gamma}}_{\text{post}}, \mathbf{\Gamma}_{\text{post}})$ , the $95\%$ credible interval for pixel $i$ is

$\hat{\gamma}_i^{\text{post}} \pm 1.96\sqrt{[\mathbf{\Gamma}_{\text{post}}]_{ii}}.$

In higher dimensions, the $(1-\alpha)$ credible region (ellipsoidal) is

$\mathcal{C}_\alpha = \left\{\boldsymbol{\gamma} : (\boldsymbol{\gamma} - \hat{\boldsymbol{\gamma}}_{\text{post}})^H \mathbf{\Gamma}_{\text{post}}^{-1} (\boldsymbol{\gamma} - \hat{\boldsymbol{\gamma}}_{\text{post}}) \leq \chi^2_{n,\,1-\alpha}\right\}.$

Definition:
Posterior Variance Map

The posterior variance map displays the diagonal of the posterior covariance as an image:

$\text{Var}(\gamma_i \mid \mathbf{y}) = [\mathbf{\Gamma}_{\text{post}}]_{ii}, \qquad i = 1, \ldots, n.$

This pixel-wise uncertainty map reveals:

Low-variance regions: Well-constrained by data (dense measurements, high SNR, good forward-operator coverage).
High-variance regions: Poorly constrained (few measurements, null-space directions of $\mathbf{A}$ , low SNR).

For the Gaussian model, $\mathbf{\Gamma}_{\text{post}} = (\sigma^{-2}\mathbf{A}^H\mathbf{A} + \mathbf{\Gamma}^{-1})^{-1}$ , so the variance map depends on the measurement geometry through $\mathbf{A}^H\mathbf{A}$ — a direct tool for optimal sensor placement (A-optimal experimental design, Eex-optimal-design).

Theorem: Posterior Contraction Rate

Under regularity conditions on the forward operator $\mathbf{A}$ and a Gaussian prior $\mu_0 = \mathcal{N}(0, \mathcal{C}_0)$ with the true scene $\boldsymbol{\gamma}^\dagger \in \mathcal{H}$ (the Cameron-Martin space), the posterior contracts around $\boldsymbol{\gamma}^\dagger$ as noise level $\sigma \to 0$ :

$\mathbb{E}\bigl[\|\boldsymbol{\gamma} - \boldsymbol{\gamma}^\dagger\|^2 \mid \mathbf{y}\bigr] = O\!\left(\sigma^{2\beta/(2\beta + 1)}\right),$

where $\beta > 0$ is the Sobolev regularity of $\boldsymbol{\gamma}^\dagger$ relative to the prior covariance. This rate matches the minimax-optimal rate for the corresponding deterministic inverse problem.

Proof

Spectral analysis in the SVD basis

Expand in the joint SVD of $\mathbf{A}$ (singular values $\sigma_k$ ) and eigenfunctions of $\mathcal{C}_0$ (eigenvalues $\lambda_k$ ). In this basis, the posterior variance in mode $k$ is

$\text{Var}_k = \left(\frac{\sigma_k^2}{\sigma^2} + \lambda_k^{-1}\right)^{-1}.$

The bias in mode $k$ is $\left(\frac{\sigma_k^2}{\sigma^2} \text{Var}_k\right)^2 |\gamma_k^\dagger|^2$ . Summing over $k$ with $\sigma_k \sim k^{-s/d}$ (forward operator decay) and $\lambda_k \sim k^{-2\beta/d}$ (prior regularity), the bias-variance trade-off gives the stated rate. $\blacksquare$

Laplace Approximation for Non-Gaussian Posteriors

For non-Gaussian priors (Laplace, horseshoe) the posterior has no closed-form covariance. The Laplace approximation fits a Gaussian to the posterior at its mode:

$p(\boldsymbol{\gamma} \mid \mathbf{y}) \approx \mathcal{N}\!\left(\hat{\boldsymbol{\gamma}}_{\text{MAP}},\; \mathbf{H}^{-1}\right),$

where $\mathbf{H} = -\nabla^2_{\boldsymbol{\gamma}} \log p(\boldsymbol{\gamma} \mid \mathbf{y}) \big|_{\hat{\boldsymbol{\gamma}}_{\text{MAP}}}$ is the Hessian of the negative log-posterior at the MAP estimate.

For Gaussian noise: $\mathbf{H} = \sigma^{-2}\mathbf{A}^H\mathbf{A} + \nabla^2(-\log\pi)(\hat{\boldsymbol{\gamma}}_{\text{MAP}})$ .

Warning: The Laplace approximation systematically underestimates uncertainty in multimodal or heavy-tailed posteriors — it only sees the local curvature at the mode, missing the tails.

Definition:
Metropolis-Hastings MCMC

The Metropolis-Hastings (MH) algorithm generates a Markov chain $\{\boldsymbol{\gamma}^{(0)}, \boldsymbol{\gamma}^{(1)}, \ldots\}$ with stationary distribution $p(\boldsymbol{\gamma} \mid \mathbf{y})$ :

Propose $\boldsymbol{\gamma}' \sim q(\boldsymbol{\gamma}' \mid \boldsymbol{\gamma}^{(t)})$ from a proposal distribution $q$ .
Accept with probability $\alpha(\boldsymbol{\gamma}', \boldsymbol{\gamma}^{(t)}) = \min\!\left(1,\; \frac{p(\boldsymbol{\gamma}' \mid \mathbf{y})\, q(\boldsymbol{\gamma}^{(t)} \mid \boldsymbol{\gamma}')}{p(\boldsymbol{\gamma}^{(t)} \mid \mathbf{y})\, q(\boldsymbol{\gamma}' \mid \boldsymbol{\gamma}^{(t)})}\right).$
Set $\boldsymbol{\gamma}^{(t+1)} = \boldsymbol{\gamma}'$ if accepted, otherwise $\boldsymbol{\gamma}^{(t+1)} = \boldsymbol{\gamma}^{(t)}$ .

The acceptance ratio involves only the unnormalized posterior $p(\mathbf{y} \mid \boldsymbol{\gamma})\,\pi(\boldsymbol{\gamma})$ , since $\mathcal{Z}(\mathbf{y})$ cancels — no partition function needed.

pCN: Preconditioned Crank-Nicolson Sampler

Complexity:

O(C_{\mathcal{A}} + n)

per iteration where

C_{\mathcal{A}}

is the cost of evaluating

\Phi(\boldsymbol{\gamma}') = \frac{1}{2\sigma^2}\|\mathbf{A}\boldsymbol{\gamma}' - \mathbf{y}\|^2

. Total:

O(T \cdot mn)

for a dense

\mathbf{A}

.

Input: Prior covariance

\mathcal{C}_0

, potential

\Phi(\boldsymbol{\gamma}) = -\log p(\mathbf{y} \mid \boldsymbol{\gamma})

, step size

\beta \in (0,1]

Output: Posterior samples

\{\boldsymbol{\gamma}^{(t)}\}_{t=1}^T

1. Draw

\boldsymbol{\gamma}^{(0)} \sim \mu_0 = \mathcal{N}(\mathbf{0}, \mathcal{C}_0)

2. for

t = 0, 1, 2, \ldots, T-1

do

3. Propose:

\boldsymbol{\gamma}' = \sqrt{1 - \beta^2}\,\boldsymbol{\gamma}^{(t)} + \beta\,\boldsymbol{\xi}

, where

\boldsymbol{\xi} \sim \mu_0

4. Compute acceptance:

a = \min\!\bigl(1,\; \exp(\Phi(\boldsymbol{\gamma}^{(t)}) - \Phi(\boldsymbol{\gamma}'))\bigr)

5. Draw

u \sim \text{Uniform}(0,1)

6. if

u < a

then

\boldsymbol{\gamma}^{(t+1)} \leftarrow \boldsymbol{\gamma}'

else

\boldsymbol{\gamma}^{(t+1)} \leftarrow \boldsymbol{\gamma}^{(t)}

7. end for

The pCN proposal preserves $\mu_0$ : if $\boldsymbol{\gamma}^{(t)} \sim \mu_0$ then $\boldsymbol{\gamma}' \sim \mu_0$ . As a consequence, the acceptance rate depends only on the likelihood ratio — it is independent of the discretization dimension $n$ . In contrast, optimal random-walk MH requires step size $\delta \propto n^{-1/2}$ , giving acceptance rate $\to 0$ as $n \to \infty$ .

MCMC Samplers for Imaging-Scale Posterior Inference

Method	Gradient needed?	Dimension scaling	Best for
Random Walk MH	No	$O(n^{-1/2})$ step size	Low-dim, simple posteriors
pCN	No	Dimension-independent	Gaussian priors, function space
Gibbs	No	Depends on conditionals	Conjugate hierarchical models (SBL)
HMC	Yes ( $\nabla \log p$ )	$O(n^{1/4})$ leapfrog steps	High-dim, smooth posteriors
NUTS (auto-HMC)	Yes	$O(n^{1/4})$ , auto-tuned	General-purpose; Stan/PyMC
Proximal MCMC (MYULA)	Yes (proximal)	$O(n)$ per step	Non-smooth priors (TV, $\ell_1$ )

Scalable Uncertainty Quantification for Imaging

Computing the full posterior covariance $\mathbf{\Gamma}_{\text{post}} \in \mathbb{R}^{n \times n}$ is infeasible for imaging-scale problems ( $n \sim 10^4$ -- $10^6$ ). Scalable alternatives:

Diagonal approximation: Compute only $[\mathbf{\Gamma}_{\text{post}}]_{ii}$ via Hutchinson's randomized trace estimator: $\operatorname{tr}(\mathbf{B}) \approx \frac{1}{K}\sum_{k=1}^K \mathbf{z}_k^T \mathbf{B} \mathbf{z}_k$ with random $\mathbf{z}_k \sim \mathcal{N}(\mathbf{0},\mathbf{I})$ .
Low-rank approximation: $\mathbf{\Gamma}_{\text{post}} \approx \mathbf{V}_r \mathbf{\Lambda}_r \mathbf{V}_r^H + \mathbf{\Gamma}$ using the $r$ leading eigenpairs of $\sigma^{-2}\mathbf{A}^H\mathbf{A}$ (computed via randomized SVD).
MCMC-based: Posterior variance estimated from sample variance: $\widehat{\operatorname{Var}}(\gamma_i) \approx \frac{1}{T}\sum_t (\gamma_i^{(t)} - \bar{\gamma}_i)^2$ .
Bootstrap: Resample data, re-solve, use ensemble spread as uncertainty proxy.

Posterior Credible Intervals — Bayesian vs Bootstrap

This plot compares uncertainty quantification methods for a 1D imaging inverse problem $\mathbf{y} = \mathbf{A}\boldsymbol{\gamma} + \mathbf{w}$ .

Top panel: True signal (black), posterior mean reconstruction (blue), and $95\%$ credible bands (shaded). Wide bands indicate poor observability of those pixels; narrow bands indicate high confidence.

Bottom panel: Posterior standard deviation map $\sqrt{[\mathbf{\Gamma}_{\text{post}}]_{ii}}$ , showing how observability depends on position through the measurement operator. Compare Bayesian credible bands with bootstrap confidence bands — Bayesian UQ correctly reflects the spatial structure of $\mathbf{A}^H\mathbf{A}$ .

Parameters

Noise

\sigma

0.1

Prior variance

\gamma^2

1

UQ method

MCMC Posterior Sampling for a 2D Inverse Problem

Visualize MCMC sampling on a 2D posterior arising from a simple imaging problem with two unknown pixels and three measurements.

Left panel: Posterior contours with MCMC sample trajectory overlaid. Random-walk MH shows diffusive, slow exploration; pCN shows more efficient traversal of the posterior.

Center panel: Trace plots of each coordinate showing mixing. Well-mixed chains explore the full support rapidly; slow chains exhibit long autocorrelations.

Right panel: Running posterior mean estimate with $\pm 2$ standard error bands, illustrating convergence speed for each algorithm.

Parameters

MCMC algorithm

Number of samples1000

Noise

\sigma

0.2

Calibration — Are Credible Intervals Trustworthy?

A posterior is well-calibrated if its credible intervals have the correct frequentist coverage:

$P_{\mathbf{y}}\bigl(\gamma_i^\dagger \in \mathcal{C}_\alpha^{(i)}\bigr) \approx 1 - \alpha$

where the probability is over repeated data realizations. Calibration can be assessed by:

Simulation studies: Generate many $(\boldsymbol{\gamma}^\dagger, \mathbf{y})$ pairs, compute credible intervals, and check empirical coverage vs nominal level.
Calibration plots: Plot observed coverage vs nominal level. A well-calibrated posterior lies on the diagonal.
CRPS (Continuous Ranked Probability Score): A proper scoring rule that jointly evaluates sharpness and calibration.

Miscalibration arises from: misspecified noise models, incorrect priors, approximate inference (Laplace approximation underestimates uncertainty in multimodal posteriors), or model mismatch (e.g., using a Gaussian prior for a clearly sparse scene).

Common Mistake: Credible Intervals Are Not Confidence Intervals

Mistake:

A Bayesian $95\%$ credible interval $[a, b]$ is interpreted as having a $95\%$ frequentist coverage probability — i.e., "in repeated experiments, the true value lies in this interval $95\%$ of the time."

Correction:

A credible interval $[a, b]$ means: given the observed data $\mathbf{y}$ and the model, the posterior assigns $95\%$ probability to $[a, b]$ . This is a conditional probability, conditioned on $\mathbf{y}$ . It coincides with frequentist coverage only when the prior is correct. A frequentist confidence interval $[a(\mathbf{y}), b(\mathbf{y})]$ , by contrast, is a random interval with the property that $P_{\boldsymbol{\gamma}^\dagger}(\boldsymbol{\gamma}^\dagger \in [a,b]) = 0.95$ for all $\boldsymbol{\gamma}^\dagger$ — a different statement. Both are valid uncertainty quantifiers, but they answer different questions.

⚠️Engineering Note

Practical UQ in Deployed RF Imaging Systems

In commercially deployed radar and SAR systems, full posterior UQ is rarely implemented due to computational cost. The standard practice is:

Matched filter + empirical noise floor: Report reconstructed reflectivity with a detection threshold based on empirical clutter statistics. No formal UQ — binary detect/non-detect.
Sparse recovery (LASSO/OMP) + posterior linearization: Run sparse recovery, then compute the Laplace approximation covariance on the estimated support. Fast but underestimates uncertainty.
Full Bayesian (SBL or MCMC): Deployed in high-value applications (medical imaging, subsurface sensing, ISAR tracking) where decision quality justifies the $10$ -- $100\times$ computational overhead vs matched filter.

The trend toward GPU-accelerated MCMC and differentiable probabilistic programming (PyMC, NumPyro) is lowering this barrier. For real-time radar ( $>1$ kHz update rate), variational Bayes and approximate MCMC remain the only feasible options.

Practical Constraints

•
Real-time radar: $< 1$ ms per frame — precludes MCMC, requires LASSO or matched filter
•
SAR post-processing: seconds to minutes per image — SBL feasible for $n \leq 10^4$
•
ISAR target classification: minutes per target — full Bayesian with pCN viable

Key Takeaway

Credible intervals extract pixel-wise uncertainty from the posterior: for Gaussian posteriors, $\hat{\gamma}_i^{\text{post}} \pm 1.96\sqrt{[\mathbf{\Gamma}_{\text{post}}]_{ii}}$ .
Posterior variance maps reveal which regions are well-constrained by data (near-zero variance) and which are dominated by the prior (high variance, null-space directions of $\mathbf{A}$ ).
The posterior contracts at the minimax-optimal rate $O(\sigma^{2\beta/(2\beta+1)})$ when the prior matches the regularity of the truth.
The Laplace approximation provides fast Gaussian UQ at the MAP solution but underestimates uncertainty for non-Gaussian posteriors.
pCN is the sampler of choice for Gaussian priors in high dimensions — dimension-independent acceptance rates via Cameron-Martin space proposals.
Calibration is essential: always validate that reported credible intervals achieve their nominal coverage before trusting the UQ in production.

Quick Check

What property of the pCN proposal $\boldsymbol{\gamma}' = \sqrt{1-\beta^2}\boldsymbol{\gamma} + \beta\boldsymbol{\xi}$ , $\boldsymbol{\xi} \sim \mu_0$ , makes it dimension-independent?

It uses gradient information to make directed proposals

It preserves the prior measure $\mu_0$ , so the acceptance probability depends only on the likelihood ratio

It adapts the step size $\beta$ automatically to the local posterior curvature

It uses a Kronecker product structure to reduce per-step cost from $O(n^2)$ to $O(n)$

Correction:

It preserves the prior measure

\mu_0

, so the acceptance probability depends only on the likelihood ratio

Correct. Since the pCN proposal preserves $\mu_0$ , the prior ratio cancels in the MH acceptance probability, leaving only the likelihood ratio. This ratio does not depend on the dimension $n$ of the parameter space (only on $m$ measurements), giving an $O(1)$ acceptance rate independent of grid refinement.

Why This Matters: Uncertainty Maps for ISAC System Design

In integrated sensing and communications (ISAC) systems, the posterior variance map $[\mathbf{\Gamma}_{\text{post}}]_{ii}$ directly informs adaptive resource allocation: pixels with high uncertainty should receive more measurements (additional transmit beams, wider bandwidth), while confident pixels need no further sensing.

This posterior-variance-driven adaptive sensing is the Bayesian analogue of A-optimal experimental design (Eex-optimal-design) and connects to the capacity-distortion tradeoff in ISAC ([?ch34:s01]): reducing the posterior variance of the sensing channel corresponds to increasing the sensing mutual information term in the capacity-distortion region derived in Caire et al.

See full treatment in Chapter 34، Section 1

Uncertainty Quantification

A Reconstruction Without Uncertainty Is Incomplete

Definition: Credible Intervals and Credible Regions

Definition: Posterior Variance Map

Theorem: Posterior Contraction Rate

Spectral analysis in the SVD basis

Laplace Approximation for Non-Gaussian Posteriors

Definition: Metropolis-Hastings MCMC

pCN: Preconditioned Crank-Nicolson Sampler

MCMC Samplers for Imaging-Scale Posterior Inference

Scalable Uncertainty Quantification for Imaging

Posterior Credible Intervals — Bayesian vs Bootstrap

Parameters

MCMC Posterior Sampling for a 2D Inverse Problem

Parameters

Calibration — Are Credible Intervals Trustworthy?

Common Mistake: Credible Intervals Are Not Confidence Intervals

Practical UQ in Deployed RF Imaging Systems

Key Takeaway

Quick Check

Why This Matters: Uncertainty Maps for ISAC System Design

Definition:
Credible Intervals and Credible Regions

Definition:
Posterior Variance Map

Definition:
Metropolis-Hastings MCMC