Chapter Summary

Chapter 3 Summary: Bayesian Inverse Problems

Key Points

• 1.

  Bayes' theorem combines the likelihood $p(\mathbf{y} \mid \boldsymbol{\gamma})$ and prior $\pi(\boldsymbol{\gamma})$ to yield the full posterior $p(\boldsymbol{\gamma} \mid \mathbf{y})$ — the complete solution to the Bayesian inverse problem. MAP equals variational regularization; MMSE equals the posterior mean.

• 2.

  For Gaussian priors $\boldsymbol{\gamma} \sim \mathcal{N}(\mathbf{0}, \mathbf{\Gamma})$ with Gaussian noise, the posterior is Gaussian with mean $(\sigma^{-2}\mathbf{A}^H\mathbf{A} + \mathbf{\Gamma}^{-1})^{-1}\sigma^{-2}\mathbf{A}^H\mathbf{y}$ — exactly the Tikhonov solution with $\lambda = \sigma^2/\gamma^2$ having a clear probabilistic interpretation as the noise-to-signal variance ratio.

• 3.

  Sparsity-promoting priors — Laplace (yields LASSO MAP), Bernoulli-Gaussian (standard RF scene model), spike-and-slab (ideal but intractable), and horseshoe (adaptive, near-minimax) — assign high prior probability to sparse scenes and produce sparser reconstructions than Gaussian priors.

• 4.

  Sparse Bayesian Learning (SBL) uses ARD per-component precisions $\{\alpha_i\}$ updated via EM. The algorithm automatically drives irrelevant $\alpha_i \to \infty$ (pruning those components) without pre-specifying sparsity level $k$. SBL provides posterior uncertainty on the active support that LASSO cannot.

• 5.

  Gaussian measures on Hilbert spaces provide discretization-invariant priors via trace-class covariance operators. The Cameron-Martin theorem characterizes which MAP estimates live in function space. Stuart's theorem guarantees existence, uniqueness, and stability of the posterior as a measure — the Bayesian analogue of Tikhonov well-posedness.

• 6.

  Posterior credible intervals and variance maps quantify reconstruction confidence. The posterior contracts at the minimax-optimal rate $O(\sigma^{2\beta/(2\beta+1)})$ when the prior regularity matches the truth.

• 7.

  The pCN sampler achieves dimension-independent MCMC acceptance rates by proposing in the Cameron-Martin space. HMC scales as $O(n^{1/4})$ using gradients. Calibration checks (empirical coverage vs nominal level) are mandatory before deploying UQ in safety-critical applications.