Ferkans — Interactive Telecom Tutor

The Spectral Decomposition — Why It Determines Reconstruction

The singular value decomposition of the forward operator $\mathcal{A}$ is the central tool of classical imaging theory. It answers three fundamental questions simultaneously:

What does the sensor see? The right singular functions $\{v_n\}$ are the "scene modes" — orthogonal components of the scene that each contribute independently to the measurements.
How much does each mode contribute? The singular value $\sigma_n$ is the gain for mode $n$ . Large $\sigma_n$ : well observed; small $\sigma_n$ : barely observable.
Is the inverse stable? Inverting $\mathcal{A}$ requires dividing by $\sigma_n$ . When $\sigma_n \to 0$ , the inverse amplifies noise without bound. The Picard condition quantifies exactly when stable inversion is possible.

This section develops the SVD for compact operators — the infinite-dimensional generalisation of the matrix SVD from Telecom Ch~1 (TSVD Existence Theorem).

Theorem: Spectral Theorem for Compact Self-Adjoint Operators

Let $\mathcal{B}$ be a compact self-adjoint operator on a separable Hilbert space $\mathcal{H}$ . Then:

(i) All eigenvalues of $\mathcal{B}$ are real.

(ii) Eigenvectors corresponding to distinct eigenvalues are orthogonal.

(iii) The non-zero eigenvalues form a countable sequence $|\lambda_1| \geq |\lambda_2| \geq \cdots$ with $\lambda_n \to 0$ .

(iv) There exists an orthonormal system $\{e_n\}$ of eigenvectors such that

$\mathcal{B}f = \sum_{n=1}^\infty \lambda_n\, \langle f, e_n \rangle\, e_n \quad \forall\, f \in \mathcal{H}.$

Self-adjointness and compactness together guarantee that the spectral theory is as "clean" as in the finite-dimensional case: real eigenvalues, orthogonal eigenvectors, and a completely explicit spectral expansion. The only infinite-dimensional feature is that eigenvalues can accumulate only at zero.

Proof

Eigenvalues are real

Since $\mathcal{B} = \mathcal{B}^*$ :

$\lambda\|f\|^2 = \langle \mathcal{B}f, f \rangle = \langle f, \mathcal{B}f \rangle = \bar\lambda\|f\|^2.$

For $f \neq 0$ : $\lambda = \bar\lambda$ , so $\lambda \in \mathbb{R}$ . $\blacksquare$

Eigenvectors for distinct eigenvalues are orthogonal

Let $\mathcal{B}e_m = \lambda_m e_m$ and $\mathcal{B}e_n = \lambda_n e_n$ with $\lambda_m \neq \lambda_n$ :

$\lambda_m \langle e_m, e_n \rangle = \langle \mathcal{B}e_m, e_n \rangle = \langle e_m, \mathcal{B}e_n \rangle = \lambda_n \langle e_m, e_n \rangle.$

So $(\lambda_m - \lambda_n)\langle e_m, e_n \rangle = 0$ . Since $\lambda_m \neq \lambda_n$ : $\langle e_m, e_n \rangle = 0$ . $\blacksquare$

Existence of the dominant eigenvalue via compactness

One shows $\|\mathcal{B}\|_{\mathrm{op}} = \sup_{\|f\|=1} |\langle \mathcal{B}f, f \rangle|$ . Since $\mathcal{B}$ is compact, the supremum is attained at some $e_1$ with $\mathcal{B}e_1 = \lambda_1 e_1$ and $|\lambda_1| = \|\mathcal{B}\|_{\mathrm{op}}$ . Restricting to $\{e_1\}^\perp$ and repeating inductively yields the full sequence $\{\lambda_n, e_n\}$ , with $\lambda_n \to 0$ by compactness. $\blacksquare$

,

Definition:
Singular System of a Compact Operator

Let $\mathcal{A}: \mathcal{X} \to \mathcal{Y}$ be a compact linear operator between Hilbert spaces. The singular system of $\mathcal{A}$ is a triple $\{(\sigma_n, v_n, u_n)\}_{n=1}^\infty$ where:

$\sigma_n > 0$ are the singular values (the non-zero square roots of the eigenvalues of $\mathcal{A}^*\mathcal{A}$ ), ordered $\sigma_1 \geq \sigma_2 \geq \cdots \to 0$ .
$\{v_n\}$ are the right singular functions in $\mathcal{X}$ (eigenfunctions of $\mathcal{A}^*\mathcal{A}$ : $\mathcal{A}^*\mathcal{A}\,v_n = \sigma_n^2\,v_n$ ).
$\{u_n\}$ are the left singular functions in $\mathcal{Y}$ (eigenfunctions of $\mathcal{A}\mathcal{A}^*$ : $\mathcal{A}\mathcal{A}^*\,u_n = \sigma_n^2\,u_n$ ).

They satisfy the cross-relations $\mathcal{A}\,v_n = \sigma_n\,u_n$ and $\mathcal{A}^*u_n = \sigma_n\,v_n$ , and the SVD expansion

$\mathcal{A}\,f = \sum_{n=1}^\infty \sigma_n\, \langle f, v_n \rangle_{\mathcal{X}}\,u_n \quad \forall\, f \in \mathcal{X}.$

This is the exact infinite-dimensional generalisation of the matrix SVD $\mathbf{A} = \mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^H$ . The key difference: in finite dimensions there are finitely many singular values, while a compact operator has $\sigma_n \to 0$ . This decay to zero is the mathematical signature of ill-posedness.

,

Example: SVD of Compact Operators — Connection to the Matrix SVD

Illustrate the SVD of a compact operator with the simple example of the finite-rank operator $\mathcal{A}: L^2([0,1]) \to \mathbb{C}^M$ defined by $(\mathcal{A}f)_m = \int_0^1 \phi_m(x)\,f(x)\,dx$ with $\phi_1, \ldots, \phi_M$ an orthonormal set. Find the singular system.

Solution

The operator is finite-rank

$\mathcal{A}f = \sum_{m=1}^M \langle f, \phi_m \rangle \mathbf{e}_m$ where $\mathbf{e}_m$ is the $m$ -th standard basis vector of $\mathbb{C}^M$ . The range has dimension at most $M$ , so $\mathcal{A}$ is finite-rank, hence compact.

Find $\mathcal{A}^*\mathcal{A}$

For $\mathbf{y} \in \mathbb{C}^M$ : $\mathcal{A}^*\mathbf{y} = \sum_{m=1}^M y_m\,\phi_m$ . Therefore $\mathcal{A}^*\mathcal{A}f = \sum_{m=1}^M \langle f, \phi_m \rangle \phi_m$ — the orthogonal projection onto $\mathrm{span}\{\phi_1,\ldots,\phi_M\}$ .

Identify the singular system

The eigenfunctions of $\mathcal{A}^*\mathcal{A}$ are the $\phi_m$ themselves, each with eigenvalue $1$ . So $\sigma_m = 1$ , $v_m = \phi_m$ , $u_m = \mathbf{e}_m$ for $m = 1, \ldots, M$ . All other singular values are zero (the null space of $\mathcal{A}$ is $\mathrm{span}\{\phi_m\}^\perp$ , the orthogonal complement of the $\phi_m$ ).

Example: Eigenvalue Decay of a Gaussian Convolution Operator

For the convolution operator with kernel $K(r - r') = e^{-|r - r'|^2 / (2\sigma^2)}$ on $\mathbb{R}$ , determine the singular value decay and explain why deblurring is severely ill-posed.

Solution

Fourier diagonalisation

Convolution operators are diagonalised by the Fourier transform. The kernel $K(r - r')$ has Fourier transform $\hat{K}(\xi) = \sigma\sqrt{2\pi}\,e^{-2\pi^2\sigma^2\xi^2}$ . Each Fourier mode $e^{j2\pi\xi r}$ is an eigenfunction with eigenvalue $\hat{K}(\xi)$ .

Eigenvalue decay

Ordering eigenvalues by decreasing magnitude and indexing by $n$ , the decay rate is

$\lambda_n \sim e^{-c\,n^2}$

for a constant $c > 0$ depending on $\sigma$ . This super-exponential (faster-than-any-polynomial) decay means that reconstructing the high-frequency content of the scene requires dividing by exponentially small numbers — even tiny noise is catastrophically amplified.

Imaging implication

For a radar system with aperture function $K$ , the singular values of the forward operator decay at the same rate as $\hat{K}(\xi)$ . A Gaussian aperture gives super-exponential decay. Even for a band-limited aperture (rectangular $\hat{K}$ ), the decay is eventually super-exponential due to the analytic continuation argument. This is the mathematical reason for the resolution limit.

Theorem: Picard Condition for Solvability

Let $\{(\sigma_n, v_n, u_n)\}$ be the singular system of a compact operator $\mathcal{A}: \mathcal{X} \to \mathcal{Y}$ with $\sigma_n > 0$ for all $n$ . Given data $\mathbf{y} \in \mathcal{Y}$ , the equation $\mathcal{A}f = \mathbf{y}$ has a solution $f \in \mathcal{X}$ if and only if:

$\mathbf{y}$ is orthogonal to $\ker(\mathcal{A}^*)$ (the consistency condition), and
The Picard condition holds:

$\sum_{n=1}^{\infty} \frac{|\langle \mathbf{y}, u_n \rangle|^2} {\sigma_n^2} < \infty.$

When it holds, the minimum-norm solution is

$f^\dagger = \sum_{n=1}^\infty \frac{\langle \mathbf{y}, u_n \rangle}{\sigma_n}\,v_n.$

The Picard condition says that the "Fourier coefficients" of the data in the $u_n$ basis must decay faster than the singular values. When the $\sigma_n$ decay rapidly (severe ill-posedness), this is a very strong requirement: even small noise components projected onto $u_n$ with small $n$ (low singular values) would violate the condition and preclude a finite-norm solution. This is the precise mathematical statement of why ill-posed problems are unstable.

Proof

Necessity

If $\mathcal{A}f = \mathbf{y}$ has a solution $f \in \mathcal{X}$ , then using $\mathcal{A}v_n = \sigma_n u_n$ :

$\langle \mathbf{y}, u_n \rangle = \langle \mathcal{A}f, u_n \rangle = \langle f, \mathcal{A}^*u_n \rangle = \sigma_n \langle f, v_n \rangle.$

So $\langle f, v_n \rangle = \langle \mathbf{y}, u_n \rangle / \sigma_n$ , and Bessel's inequality requires $\sum_n |\langle f, v_n \rangle|^2 \leq \|f\|^2 < \infty$ , which gives the Picard condition.

Sufficiency and minimum-norm solution

If the Picard condition holds, define $f^\dagger = \sum_n \frac{\langle \mathbf{y}, u_n \rangle}{\sigma_n} v_n$ . Parseval's identity and the Picard condition ensure $f^\dagger \in \mathcal{X}$ . One verifies $\mathcal{A}f^\dagger = \mathbf{y}$ using the cross-relation $\mathcal{A}v_n = \sigma_n u_n$ . The minimum-norm property follows from the fact that $f^\dagger \in \overline{\mathrm{Range}(\mathcal{A}^*)} = \ker(\mathcal{A})^\perp$ . $\blacksquare$

,

Singular Value Decay of Imaging Operators

The singular values $\sigma_n$ of imaging operators decay to zero. The decay rate determines the degree of ill-posedness: polynomial decay means "mildly ill-posed" (deconvolution), super-exponential means "severely ill-posed" (analytic continuation, back-scattering).

Observe: (a) the Picard condition requires data coefficients $|\langle \mathbf{y}, u_n \rangle|$ to decay faster than $\sigma_n$ ; (b) noise — which has roughly constant spectral power — violates the Picard condition beyond index $n^*$ where $\sigma_{n^*} \sim \|\mathbf{w}\|$ .

Parameters

Operator type

Grid size

N

64

The Picard Plot — Diagnosing Ill-Posedness

The Picard plot shows three curves on a log scale: (1) singular values $\sigma_n$ (blue, decaying), (2) exact data coefficients $|\langle \mathcal{A}f, u_n \rangle|$ (green, should decay faster than $\sigma_n$ ), (3) noisy data coefficients $|\langle \mathbf{y}, u_n \rangle|$ (red, eventually dominated by noise level $\sigma$ ).

The index $n^*$ where the red curve crosses the noise floor determines the achievable reconstruction bandwidth. Regularisation effectively truncates the SVD at $n^*$ .

Parameters

SNR (dB)20

Singular value decay exponent1.5

Example: Eigenvalue Decay — Mildly vs. Severely Ill-Posed

Compare the singular value decay for (a) the integration operator (mildly ill-posed), (b) the Laplace transform operator (moderately ill-posed), and (c) the analytic continuation operator (severely ill-posed). Classify each according to how many terms satisfy the Picard condition at noise level $\delta$ .

Solution

Integration operator (mildly ill-posed)

The integration operator $(\mathcal{A}f)(t) = \int_0^t f(s)\,ds$ has singular values decaying as $\sigma_n = O(1/n)$ — polynomial decay of order 1. At noise level $\delta$ : approximately $n^* \sim 1/\delta$ modes satisfy the Picard condition, so the regularised reconstruction has $O(1/n^*)$ bandwidth.

Laplace transform (moderately ill-posed)

The Laplace transform on $[0, 1]$ has singular values decaying as $\sigma_n = O(e^{-c\sqrt{n}})$ — sub-exponential decay. At noise level $\delta$ : approximately $n^* \sim (\log(1/\delta))^2$ modes are usable. The number of usable modes grows very slowly with improving SNR.

Analytic continuation (severely ill-posed)

Recovering a function on $[-1, 0]$ from values on $[0, 1]$ has singular values decaying as $\sigma_n = O(e^{-cn})$ — exponential decay. At noise level $\delta$ : only $n^* \sim \log(1/\delta)$ modes are usable. Even at $\text{SNR} = 60$ dB ( $\delta = 10^{-3}$ ), only about 7 modes are recoverable — catastrophically few.

Eigenvalue Decay: Mildly vs. Severely Ill-Posed

Compare singular value decay rates for different operator types. The classification (mildly / moderately / severely ill-posed) determines the achievable number of reliably recoverable modes at a given noise level and, equivalently, the spatial resolution attainable in the reconstructed image.

Parameters

Operator class

SNR (dB)30

Historical Note: The SVD in Inverse Problems — Schmidt, Picard, and Tikhonov

1907–1963

The singular value decomposition for integral operators was developed by Erhard Schmidt (1907) and independently by Émile Picard. Schmidt's work on bilinear forms of square-integrable kernels produced what we now recognise as the infinite-dimensional SVD. Picard applied it to derive the solvability condition for Fredholm integral equations of the first kind — now called the Picard condition (1910).

The connection to stability and regularisation was made explicit only in the 1950s–60s. Andrei Tikhonov's 1963 paper on regularisation of ill-posed problems provided the key idea: rather than solving $\mathcal{A}f = \mathbf{y}$ exactly (unstable), minimise $\|\mathcal{A}f - \mathbf{y}\|^2 + \alpha\|f\|^2$ (stable). In the SVD basis this amounts to replacing $1/\sigma_n$ with $\sigma_n/(\sigma_n^2 + \alpha)$ — a smooth dampening of small singular values rather than the abrupt truncation of Picard.

For RF imaging the SVD of the forward operator was first computed numerically by David Slepian (1961) for the 1-D diffraction problem using prolate spheroidal wave functions — the singular functions that optimally concentrate spatial bandwidth. This computation revealed, for the first time, the precise number of degrees of freedom in a diffraction-limited image.

⚠️Engineering Note

Practical SVD Computation for Imaging Operators

The theoretical SVD of a compact operator is defined on an infinite-dimensional space, but in practice we work with the discrete sensing matrix $\mathbf{A} \in \mathbb{C}^{M \times N}$ .

For realistic problems (e.g., 3-D SAR at 1 mm resolution: $N \sim 10^6$ voxels, $M \sim 10^5$ measurements), storing $\mathbf{A}$ explicitly is infeasible ( $10^{11}$ entries). Instead, use:

Randomised SVD (Halko et al., 2011): computes the top- $k$ singular vectors in $O(MN k)$ using random projections. Accuracy within $\sigma_{k+1}$ of exact. Suitable for $k \leq 1000$ .
Lanczos bidiagonalisation (Paige–Saunders LSQR): matrix-free, requires only $\mathcal{A}\mathbf{v}$ and $\mathcal{A}^H\mathbf{u}$ multiplications. Standard in large-scale LASSO solvers.
Fourier-domain methods: when $\mathcal{A}$ is a convolution, its SVD is the DFT — no explicit computation needed.

Practical Constraints

•
Storing the full $M \times N$ matrix requires $O(MN)$ memory; for $M = N = 10^4$ this is 1.6 GB in float32 — feasible. For $N = 10^6$ this is 80 TB — infeasible without matrix-free methods.
•
Randomised SVD accuracy degrades for slowly decaying singular values (mild ill-posedness); works well for exponentially decaying cases (severe ill-posedness) since few singular values matter.
•
CUDA-accelerated NUFFT reduces the $\mathcal{A}$ multiplication from $O(MN)$ to $O(M + N\log N)$ when the acquisition geometry is near-uniform.

Common Mistake: SVD $\neq$ Eigendecomposition for Non-Self-Adjoint Operators

Mistake:

Confusing the SVD of $\mathcal{A}$ with the eigendecomposition of $\mathcal{A}$ , or applying the spectral theorem (for self-adjoint operators) to a general $\mathcal{A}$ .

Correction:

The spectral theorem applies to compact self-adjoint operators. For a general compact operator $\mathcal{A}$ :

The SVD of $\mathcal{A}$ uses TWO sets of orthonormal functions ( $\{v_n\}$ in the input space, $\{u_n\}$ in the output space).
The eigendecomposition uses ONE set. It exists only if $\mathcal{A}$ is self-adjoint (or at least normal: $\mathcal{A}\mathcal{A}^* = \mathcal{A}^*\mathcal{A}$ ).
The forward operator $\mathcal{A}: L^2(\Omega) \to \mathbb{C}^M$ maps between different spaces — it cannot even have eigenvectors in the usual sense.

The correct object is the SVD of $\mathcal{A}$ , or equivalently the spectral decomposition of $\mathcal{A}^*\mathcal{A}$ (a self-adjoint operator on the input space).

Why This Matters: SVD, Degrees of Freedom, and the Resolution Limit

The number of degrees of freedom (DoF) of an imaging system is the effective rank of the sensing operator — the number of singular values that are non-negligible compared to the noise level $\sigma$ :

$\mathrm{DoF} = \#\bigl\{n : \sigma_n > \sigma\bigr\}.$

For a monostatic radar with aperture $D$ and wavelength $\lambda$ illuminating a scene at range $R$ , the number of DoF in the cross-range direction is approximately $D^2 / (\lambda R)$ — the ratio of the aperture area to the diffraction-limited resolution cell area. This is the fundamental Nyquist rate for imaging, analogous to the sampling theorem for 1-D signals.

The singular functions $\{v_n\}$ are the prolate spheroidal wave functions (PSWFs) — discovered by Slepian (1961) precisely by computing the SVD of the diffraction operator. The PSWFs achieve the maximum spatial concentration within the aperture footprint, making them the natural basis for band-limited imaging.

See full treatment in Chapter 7

Quick Check

The Picard condition for the equation $\mathcal{A}f = \mathbf{y}$ to have a solution requires ___.

$\sum_n |\langle \mathbf{y}, u_n \rangle|^2 / \sigma_n^2 < \infty$

$\sum_n |\langle \mathbf{y}, u_n \rangle|^2 \cdot \sigma_n^2 < \infty$

All singular values $\sigma_n$ are bounded below by a positive constant.

The operator $\mathcal{A}$ is self-adjoint.

Correction:

\sum_n |\langle \mathbf{y}, u_n \rangle|^2 / \sigma_n^2 < \infty

The Picard condition states that the data coefficients $|\langle \mathbf{y}, u_n \rangle|$ must decay fast enough relative to the singular values $\sigma_n$ that the series $\sum_n |\langle \mathbf{y}, u_n \rangle|^2 / \sigma_n^2$ converges. This is the necessary and sufficient condition for the pseudoinverse series $f^\dagger = \sum_n (\langle \mathbf{y}, u_n \rangle / \sigma_n) v_n$ to define a square-integrable function.

Quick Check

A compact operator with exponentially decaying singular values is classified as ___.

well-posed

mildly ill-posed

moderately ill-posed

severely ill-posed

Correction:

severely ill-posed

Exponential decay $\sigma_n \sim e^{-cn}$ is the signature of severely ill-posed problems. Even at $\text{SNR} = 60$ dB, only $\sim \log(10^3)/c$ modes satisfy the Picard condition — a handful. Analytic continuation and inverse scattering (beyond Born approximation) are in this class.

Singular system

The triple $\{(\sigma_n, v_n, u_n)\}$ of singular values and singular functions of a compact operator $\mathcal{A}$ . Provides the complete spectral characterisation of $\mathcal{A}$ and the pseudoinverse $\mathcal{A}^\dagger$ .

Picard condition

The condition $\sum_n |\langle \mathbf{y}, u_n \rangle|^2 / \sigma_n^2 < \infty$ necessary and sufficient for the equation $\mathcal{A}f = \mathbf{y}$ to have a solution. Noise always violates the Picard condition eventually, motivating regularisation.

Key Takeaway

Four core messages of this section:

(1) The singular system $\{(\sigma_n, v_n, u_n)\}$ provides the complete spectral picture of a compact operator: which scene modes are well-observed ( $\sigma_n$ large) and which are invisible ( $\sigma_n \approx 0$ ).

(2) Singular values decay to zero for compact operators. The decay rate classifies ill-posedness: polynomial = mild, sub-exponential = moderate, exponential = severe.

(3) The Picard condition says: the data coefficients must decay faster than the singular values for the inverse problem to be solvable. Noise violates this condition at high frequencies — the fundamental reason all imaging problems require regularisation.

(4) The number of degrees of freedom $\mathrm{DoF} \sim \#\{n : \sigma_n > \sigma\}$ is the effective rank at a given SNR and determines the achievable spatial resolution. This connects SVD theory directly to the resolution limit of any RF imaging system.

Singular Value Decomposition for Operators

The Spectral Decomposition — Why It Determines Reconstruction

Theorem: Spectral Theorem for Compact Self-Adjoint Operators

Eigenvalues are real

Eigenvectors for distinct eigenvalues are orthogonal

Existence of the dominant eigenvalue via compactness

Definition: Singular System of a Compact Operator

Example: SVD of Compact Operators — Connection to the Matrix SVD

The operator is finite-rank

Find $\mathcal{A}^*\mathcal{A}$

Identify the singular system

Example: Eigenvalue Decay of a Gaussian Convolution Operator

Fourier diagonalisation

Eigenvalue decay

Imaging implication

Theorem: Picard Condition for Solvability

Necessity

Sufficiency and minimum-norm solution

Singular Value Decay of Imaging Operators

Parameters

The Picard Plot — Diagnosing Ill-Posedness

Parameters

Example: Eigenvalue Decay — Mildly vs. Severely Ill-Posed

Integration operator (mildly ill-posed)

Laplace transform (moderately ill-posed)

Analytic continuation (severely ill-posed)

Eigenvalue Decay: Mildly vs. Severely Ill-Posed

Parameters

Historical Note: The SVD in Inverse Problems — Schmidt, Picard, and Tikhonov

Practical SVD Computation for Imaging Operators

Common Mistake: SVD ≠\neq= Eigendecomposition for Non-Self-Adjoint Operators

Why This Matters: SVD, Degrees of Freedom, and the Resolution Limit

Quick Check

Quick Check

Singular system

Picard condition

Key Takeaway

Definition:
Singular System of a Compact Operator

Common Mistake: SVD $\neq$ Eigendecomposition for Non-Self-Adjoint Operators