Prerequisites & Notation

Prerequisites for This Chapter

Linear Algebra and Operator Theory (Ch 01) — Hilbert spaces, bounded linear operators, adjoints, singular value decomposition(Review ch01)
Self-check: Can you state the SVD of a bounded linear operator and define its pseudoinverse?
Ill-Posed Problems and Regularization Theory (Ch 02) — Forward models, Tikhonov regularization, spectral filtering, discrepancy principle(Review ch02)
Self-check: Can you derive the Tikhonov solution and interpret the regularization parameter $\lambda$ as a bias-variance tradeoff?
Probability Spaces and Random Variables (Telecom Ch 02) — Probability distributions, conditional probability, Bayes theorem, expectation, variance(Review ch02)
Self-check: Can you state Bayes theorem and compute the posterior for a Gaussian prior with Gaussian likelihood?
Estimation Theory and LMMSE (Telecom Ch 09) — LMMSE estimation, MSE, bias-variance trade-off(Review ch09)
Self-check: Can you derive the LMMSE estimator and connect it to the Wiener filter?

Notation for This Chapter

Symbols introduced in this chapter. All distributions are on $\mathbb{R}^n$ unless stated otherwise. The forward model is $\mathbf{y} = \mathbf{A}\boldsymbol{\gamma} + \mathbf{w}$ with $\mathbf{w} \sim \mathcal{N}(0, \sigma^2 \mathbf{I})$ and $\boldsymbol{\gamma}$ the scene reflectivity.

Symbol	Meaning	Introduced
$p(\mathbf{y} \mid \boldsymbol{\gamma})$	Likelihood: probability of data $\mathbf{y}$ given unknown scene $\boldsymbol{\gamma}$	s01
$\pi(\boldsymbol{\gamma})$	Prior distribution on the scene reflectivity $\boldsymbol{\gamma}$	s01
$p(\boldsymbol{\gamma} \mid \mathbf{y})$	Posterior distribution of $\boldsymbol{\gamma}$ given data $\mathbf{y}$	s01
$\hat{\boldsymbol{\gamma}}_{\text{MAP}}$	Maximum a posteriori estimate (mode of posterior)	s01
$\hat{\boldsymbol{\gamma}}_{\text{MMSE}}$	Posterior mean (MMSE) estimate	s01
$\mathbf{\Gamma}$	Prior covariance matrix	s01
$\mathbf{\Gamma}_{\text{post}}$	Posterior covariance matrix	s01
$\mathcal{Z}(\mathbf{y})$	Evidence (marginal likelihood): $\int p(\mathbf{y} \mid \boldsymbol{\gamma})\,\pi(\boldsymbol{\gamma})\,\mathrm{d}\boldsymbol{\gamma}$	s01
$\alpha_i$	Per-component precision hyperparameter (ARD/SBL)	s03
$\mathcal{C}_0$	Covariance operator of a Gaussian measure on a Hilbert space	s04
$\mathcal{H}$	Cameron-Martin space $= \operatorname{Range}(\mathcal{C}_0^{1/2})$	s04

← Ch 2 The Bayesian Framework for Inverse Problems