Ferkans — Interactive Telecom Tutor

Why Infinite-Dimensional Spaces for RF Imaging?

The fundamental task in RF imaging is reconstruction: given noisy measurements

$\mathbf{y} = \mathbf{A}\,\mathbf{c} + \mathbf{w},$

we seek to recover the scene $c(\mathbf{r})$ — a reflectivity map, a permittivity profile, or a scattering distribution defined at every point $\mathbf{r} \in \Omega$ .

This is fundamentally different from the finite-dimensional linear algebra of Telecom Ch~1, where all signals live in $\mathbb{C}^n$ . A scene function $c(\mathbf{r})$ assigns a complex value to every point in a continuous domain — it lives in an infinite-dimensional space. To judge the quality of an estimate $\hat{c}$ , we need $\|\hat{c} - c\|$ , and the choice of norm is far from cosmetic:

Minimising $\|\cdot\|_{L^2}$ yields smooth, energy-optimal reconstructions.
Minimising $\|\cdot\|_{L^1}$ promotes sparse, edge-preserving solutions.
Sobolev norms $\|\cdot\|_{H^s}$ encode prior smoothness.

The mathematical framework of this section — normed spaces, completeness, Banach spaces — is the bedrock on which every reconstruction algorithm rests.

Historical Note: Stefan Banach and the Birth of Functional Analysis

1920–1932

Stefan Banach (1892–1945), working in Lwów (then Poland), created the theory of complete normed spaces in his 1920 doctoral thesis and the landmark 1932 monograph Théorie des opérations linéaires. The monograph, written in French to reach the widest European audience, collected the Hahn–Banach theorem, the open mapping theorem, and the uniform boundedness principle — results developed in parallel by Banach, Hahn, and Steinhaus — into a unified theory.

Banach's spaces provided the language in which John von Neumann simultaneously developed quantum mechanics (Hilbert spaces) and in which Norbert Wiener grounded signal processing. The $L^p$ spaces had been defined by Riesz in 1910; Banach's insight was that completeness — not just a norm — was the essential structural property.

Definition:
Normed Space

A normed space $(\mathcal{V}, \|\cdot\|)$ is a vector space $\mathcal{V}$ over a field $\mathbb{F} \in \{\mathbb{R}, \mathbb{C}\}$ together with a function $\|\cdot\| \colon \mathcal{V} \to [0,\infty)$ — called a norm — satisfying for all $f, g \in \mathcal{V}$ and $\alpha \in \mathbb{F}$ :

#	Axiom	Statement
N1	Positive definiteness	$\\|f\\| = 0 \iff f = 0$
N2	Absolute homogeneity	$\\|\alpha f\\| = \|\alpha\|\,\\|f\\|$
N3	Triangle inequality	$\\|f + g\\| \leq \\|f\\| + \\|g\\|$

Every norm induces a metric $d(f, g) = \|f - g\|$ , so a normed space is automatically a metric space and topological notions (convergence, continuity, open/closed sets) are inherited.

The nontrivial content of N1 is the converse direction: $\|f\| = 0$ implies $f = 0$ . This rules out "pseudo-norms" that vanish on nonzero elements.

Definition:
$L^p$ Spaces

Let $\Omega \subseteq \mathbb{R}^d$ be a measurable set. For $1 \leq p < \infty$ , the Lebesgue space $L^p(\Omega)$ consists of all equivalence classes of measurable functions $f \colon \Omega \to \mathbb{C}$ for which the following norm is finite:

$\|f\|_{L^p} = \Bigl(\int_\Omega |f(\mathbf{r})|^p \, d\mathbf{r}\Bigr)^{1/p}.$

For $p = \infty$ :

$\|f\|_{L^\infty} = \operatorname{ess\,sup}_{\mathbf{r} \in \Omega}\, |f(\mathbf{r})|,$

and $L^\infty(\Omega)$ consists of all measurable functions with $\|f\|_{L^\infty} < \infty$ (essentially bounded functions).

Two functions are identified if they differ only on a set of measure zero: $f = g$ a.e.

$L^2(\Omega)$ occupies a privileged position: it is the only $L^p$ space (for $p \neq 2$ ) that is also a Hilbert space (see DHilbert Space). It is the natural home for finite-energy signals and the setting where least-squares reconstruction is most naturally formulated.

Example: $L^p$ Norms of a Rectangular Pulse

Let $f \colon \mathbb{R}^2 \to \mathbb{R}$ be the indicator function of the unit square:

$f(\mathbf{r}) = \mathbf{1}_{[0,1]^2}(\mathbf{r}) = \begin{cases} 1 & \mathbf{r} \in [0,1]^2, \\ 0 & \text{otherwise}. \end{cases}$

Compute $\|f\|_{L^1}$ , $\|f\|_{L^2}$ , and $\|f\|_{L^\infty}$ .

Solution

$L^1$ norm (total mass)

$\|f\|_{L^1} = \int_{\mathbb{R}^2} |f(\mathbf{r})|\,d\mathbf{r} = \int_{[0,1]^2} 1\,d\mathbf{r} = 1.$ $

$L^2$ norm (energy)

$\|f\|_{L^2} = \Bigl(\int_{[0,1]^2} 1^2\,d\mathbf{r}\Bigr)^{1/2} = 1.$ $The two coincide here because$ |f|^2 = |f|$ for a 0-1 valued function.

$L^\infty$ norm (peak value)

$\|f\|_{L^\infty} = \operatorname{ess\,sup}_{\mathbf{r}} |f(\mathbf{r})| = 1.$ $

Interpretation for scaled pulses

For $g(\mathbf{r}) = A\,\mathbf{1}_{[0,L]^2}(\mathbf{r})$ one gets $\|g\|_{L^1} = |A|L^2$ , $\|g\|_{L^2} = |A|L$ , $\|g\|_{L^\infty} = |A|$ — three genuinely different "sizes." This illustrates why the choice of norm shapes reconstruction objectives.

Definition:
Sequence Spaces $\ell^p$

The sequence space $\ell^p$ is the discrete analogue of $L^p$ :

$\ell^p = \Bigl\{\mathbf{x} = (x_1, x_2, \ldots) : \|\mathbf{x}\|_{\ell^p} = \Bigl(\sum_{k=1}^{\infty} |x_k|^p\Bigr)^{1/p} < \infty\Bigr\}, \quad 1 \leq p < \infty.$

For $p = \infty$ : $\ell^\infty = \{\mathbf{x} : \sup_k |x_k| < \infty\}$ .

For finite sequences of length $N$ , all $\ell^p$ norms are well-defined and the space is $\mathbb{C}^N$ with the $\|\cdot\|_p$ norm.

When we discretise a scene on a grid of $N$ voxels, the infinite-dimensional problem in $L^p(\Omega)$ becomes a finite-dimensional problem in $\mathbb{C}^N$ with the $\ell^p$ norm. This bridge between continuous theory and discrete computation is central to every practical imaging algorithm.

Definition:
Cauchy Sequence

A sequence $\{f_n\}_{n=1}^\infty$ in a normed space $(\mathcal{V}, \|\cdot\|)$ is a Cauchy sequence if for every $\varepsilon > 0$ there exists $N \in \mathbb{N}$ such that

$\|f_m - f_n\| < \varepsilon \qquad \text{for all } m, n \geq N.$

Every convergent sequence is Cauchy. The converse need not hold in an arbitrary normed space (the rationals $\mathbb{Q}$ provide a counterexample).

Definition:
Banach Space

A normed space $(\mathcal{V}, \|\cdot\|)$ is a Banach space if every Cauchy sequence in $\mathcal{V}$ converges to an element of $\mathcal{V}$ :

$\{f_n\} \text{ Cauchy in } \mathcal{V} \implies \exists\, f \in \mathcal{V} : \|f_n - f\| \to 0.$

Completeness is not a luxury — it is essential for the convergence guarantees of iterative reconstruction algorithms. Banach's fixed-point theorem (the cornerstone of ISTA, gradient descent, and proximal algorithms) requires completeness as a hypothesis. Without it, an iterative sequence could satisfy $\|f_{m} - f_{n}\| \to 0$ yet have no limit within the space.

,

Theorem: $L^p$ Spaces are Banach Spaces

For $1 \leq p \leq \infty$ , the space $L^p(\Omega)$ equipped with the $\|\cdot\|_{L^p}$ norm is a Banach space (the Riesz–Fischer theorem).

Extract a "rapidly convergent" subsequence from a Cauchy sequence, build a candidate limit via dominated or monotone convergence, then show the entire original sequence converges to that limit. The argument exploits the completeness of $\mathbb{R}$ pointwise (a.e.) and then promotes this to $L^p$ convergence.

Show Hint

Extract a subsequence $\{f_{n_k}\}$ with $\|f_{n_{k+1}} - f_{n_k}\|_p < 2^{-k}$ .

Form $G_K = \sum_{k=1}^{K}|f_{n_{k+1}} - f_{n_k}|$ and bound its $L^p$ norm by 1.

Apply monotone convergence to get $G(\mathbf{r}) < \infty$ a.e., defining a candidate limit.

Proof

Extract a rapidly convergent subsequence

Let $\{f_n\}$ be Cauchy in $L^p(\Omega)$ . For each $k \in \mathbb{N}$ choose $n_k$ such that $\|f_m - f_n\|_p < 2^{-k}$ for all $m, n \geq n_k$ , with $n_1 < n_2 < \cdots$ inductively:

$\|f_{n_{k+1}} - f_{n_k}\|_p < 2^{-k}.$

Show partial sums converge a.e.

Define $G_K(\mathbf{r}) = \sum_{k=1}^{K} |f_{n_{k+1}}(\mathbf{r}) - f_{n_k}(\mathbf{r})|$ . By Minkowski's inequality, $\|G_K\|_p \leq \sum_{k=1}^{K} 2^{-k} < 1$ . As $K \to \infty$ , $G_K \nearrow G$ pointwise, and monotone convergence gives $\|G\|_p \leq 1 < \infty$ , so $G(\mathbf{r}) < \infty$ a.e.

Identify the limit and conclude $L^p$ convergence

Since $G < \infty$ a.e., the telescoping series $f(\mathbf{r}) = f_{n_1}(\mathbf{r}) + \sum_{k=1}^{\infty} (f_{n_{k+1}} - f_{n_k})(\mathbf{r})$ converges absolutely a.e., defining a measurable $f$ with $f_{n_k} \to f$ a.e.

Since $|f_{n_k} - f|^p \to 0$ a.e. and $|f_{n_k} - f|^p \leq (2G)^p \in L^1$ , dominated convergence gives $\|f_{n_k} - f\|_p \to 0$ .

For the full sequence: for $n \geq n_k$ , $\|f_n - f\|_p \leq \|f_n - f_{n_k}\|_p + \|f_{n_k} - f\|_p \to 0$ . Hence $L^p(\Omega)$ is complete. $\blacksquare$

$L^p$ Unit Balls in $\mathbb{R}^2$

The unit ball $\{\mathbf{x} \in \mathbb{R}^2 : \|\mathbf{x}\|_p \leq 1\}$ changes shape dramatically with $p$ . At $p = 1$ : a diamond whose corners on the axes explain why $\ell^1$ minimisation promotes sparsity. At $p = 2$ : the familiar Euclidean circle. As $p \to \infty$ : a square $[-1,1]^2$ .

For $p < 1$ the "ball" is no longer convex — $\|\cdot\|_p$ fails the triangle inequality and is only a quasi-norm. Observe how the pinching makes the coordinate axes even stronger attractors, linking sub- $\ell^1$ norms to compressed sensing.

Parameters

p

2

Norm order

Definition:
Equivalence of Norms

Two norms $\|\cdot\|_a$ and $\|\cdot\|_b$ on a vector space $\mathcal{V}$ are equivalent if there exist constants $c, C > 0$ such that

$c\,\|f\|_a \leq \|f\|_b \leq C\,\|f\|_a \qquad \forall\, f \in \mathcal{V}.$

Equivalent norms define the same topology (same convergent sequences, same continuous maps, same open/closed sets).

Theorem: All Norms on Finite-Dimensional Spaces are Equivalent

Let $\mathcal{V}$ be a finite-dimensional vector space over $\mathbb{R}$ or $\mathbb{C}$ . Then any two norms on $\mathcal{V}$ are equivalent.

The unit sphere of any norm on $\mathbb{C}^n$ is compact (Heine–Borel). A continuous function on a compact set attains its minimum (which is positive, since the norm is positive definite) and maximum, supplying the equivalence constants.

Show Hint

Show every norm on $\mathbb{C}^n$ is Lipschitz continuous w.r.t. $\|\cdot\|_2$ .

Use compactness of the $\|\cdot\|_2$ unit sphere to attain inf and sup.

Proof

Continuity of an arbitrary norm w.r.t. $\|\cdot\|_2$

Let $\|\cdot\|$ be any norm on $\mathbb{C}^n$ and let $\{\mathbf{e}_k\}$ be the standard basis. For $\mathbf{x} = \sum_k x_k \mathbf{e}_k$ :

$\|\mathbf{x}\| \leq \sum_{k} |x_k|\,\|\mathbf{e}_k\| \leq \Bigl(\sum_k \|\mathbf{e}_k\|^2\Bigr)^{1/2} \|\mathbf{x}\|_2 = M\,\|\mathbf{x}\|_2$

by Cauchy–Schwarz. Thus $\|\cdot\|$ is Lipschitz (hence continuous) with respect to $\|\cdot\|_2$ .

Extract equivalence constants via compactness

The unit sphere $S = \{\mathbf{x} : \|\mathbf{x}\|_2 = 1\}$ is compact. The continuous function $\mathbf{x} \mapsto \|\mathbf{x}\|$ attains its minimum $m > 0$ and maximum $M < \infty$ on $S$ . For any nonzero $\mathbf{x}$ , normalising by $\|\mathbf{x}\|_2$ gives

$m\,\|\mathbf{x}\|_2 \leq \|\mathbf{x}\| \leq M\,\|\mathbf{x}\|_2.$

Common Mistake: Norms Are NOT Equivalent in Infinite Dimensions

Mistake:

Assuming that convergence in one $L^p$ norm implies convergence in another, as holds in $\mathbb{C}^n$ .

Correction:

In infinite-dimensional spaces different $L^p$ norms give genuinely different notions of convergence.

Example. On $\Omega = [0,1]$ , let $f_n(x) = n \cdot \mathbf{1}_{[0,\,1/n]}(x)$ (a tall thin spike).

$\|f_n\|_{L^1} = n \cdot n^{-1} = 1$ for all $n$ — does NOT converge to $0$ .
$\|f_n\|_{L^2} = \sqrt{n} \to \infty$ — diverges.
$f_n(x) \to 0$ pointwise for $x > 0$ , yet neither $L^p$ norm converges to zero.

The key point: convergence rates and even convergence itself depend on which norm you use. An algorithm converging in $L^2$ (minimising MSE) may behave very differently from one converging in $L^1$ (promoting sparsity and sharp edges).

Why This Matters: Norms and Image Reconstruction Quality

The choice of norm in

$\hat{c} = \arg\min_{c}\, \bigl\|\mathbf{y} - \mathbf{A}\,\mathbf{c}\bigr\|^2 + \lambda\,\mathcal{R}(c)$

directly determines the character of the solution via the regulariser $\mathcal{R}$ :

$L^2$ (Tikhonov): $\mathcal{R} = \|\cdot\|_{L^2}^2$ — penalises large values, produces smooth reconstructions, leads to linear systems.
$L^1$ (sparsity): $\mathcal{R} = \|\cdot\|_{L^1}$ — promotes exactly zero pixels, the basis of compressed sensing for radar.
Total variation: $\mathcal{R} = \|\nabla\cdot\|_{L^1}$ — promotes piecewise-constant images with sharp edges, ideal for SAR urban scenes.

The completeness of the underlying space guarantees that minimisers exist and iterative algorithms converge — without it, none of these guarantees hold. We return to this in Chapter 2 (Regularisation Theory).

See full treatment in Chapter 2

Quick Check

Which $L^p$ space is also a Hilbert space?

$L^1(\Omega)$

$L^2(\Omega)$

$L^\infty(\Omega)$

All $L^p$ spaces are Hilbert spaces

Correction:

L^2(\Omega)

$L^2(\Omega)$ admits the inner product $\langle f, g \rangle = \int_\Omega f\,\bar{g}\,d\mathbf{r}$ which induces the $L^2$ norm. No other $L^p$ ( $p \neq 2$ ) has this property — their norms fail the parallelogram law $\|f+g\|^2 + \|f-g\|^2 = 2\|f\|^2 + 2\|g\|^2$ .

Quick Check

In $\mathbb{R}^2$ , the $\ell^1$ unit ball $\{\mathbf{x} : \|\mathbf{x}\|_1 \leq 1\}$ is a ___.

Circle

Square with sides parallel to the axes

Diamond (rhombus) with vertices on the axes

Ellipse

Correction:

Diamond (rhombus) with vertices on the axes

The constraint $|x_1| + |x_2| \leq 1$ describes a square rotated $45°$ , with vertices at $(\pm 1, 0)$ and $(0, \pm 1)$ . The corners on the coordinate axes are why $\ell^1$ minimisation promotes sparsity: when a linear objective is tangent to this ball, it generically touches at a vertex — a sparse point.

Quick Check

True or False: All norms on $L^2([0,1])$ are equivalent.

True

False

Correction:

False

Norm equivalence holds only in finite-dimensional spaces. $L^2([0,1])$ is infinite-dimensional, so it admits inequivalent norms. For instance, the $L^2$ and $L^1$ norms on $L^2([0,1])$ are not equivalent: the sequence $f_n = n\,\mathbf{1}_{[0,1/n^2]}$ converges in $L^1$ but diverges in $L^2$ .

Normed space

A vector space $\mathcal{V}$ equipped with a norm $\|\cdot\| \colon \mathcal{V} \to [0,\infty)$ satisfying positive definiteness, absolute homogeneity, and the triangle inequality. See DNormed Space.

Banach space

A normed space that is complete: every Cauchy sequence converges to an element of the space. Named after Stefan Banach. See DBanach Space.

$L^p$ space

The Lebesgue space of measurable functions whose $p$ -th power is integrable ( $1 \leq p < \infty$ ), or essentially bounded ( $p = \infty$ ). $L^p(\Omega)$ is a Banach space for all $p \in [1,\infty]$ ; only $L^2$ is also a Hilbert space. See $L^p$ $L^{p}$ Spaces" data-ref-type="definition">D $L^p$ Spaces.

Completeness

A metric or normed space is complete if every Cauchy sequence in the space converges to a limit that belongs to the space. Completeness ensures "limits stay inside" and is essential for well-posedness of iterative algorithms. See DBanach Space.

Related: Banach Space, Cauchy Sequence

Key Takeaway

Four core messages of this section:

(1) Normed spaces generalise $\mathbb{C}^n$ to infinite dimensions. Imaging scenes are functions, not finite vectors, requiring $L^p(\Omega)$ spaces with norms that measure "size" in problem-specific ways.

(2) Banach spaces are the right setting for iterative algorithms. Completeness guarantees that convergent iterative sequences have a limit within the space — without it algorithms could "converge to nothing."

(3) The $L^p$ family is foundational: $L^2$ for energy-based and MSE reconstruction, $L^1$ for sparsity-promoting and edge-preserving solutions, Sobolev $H^s$ for smooth prior models.

(4) Unlike finite dimensions, the choice of norm is consequential. In $\mathbb{C}^n$ all norms are equivalent. In infinite-dimensional function spaces they are not — the norm determines the topology, the notion of convergence, and ultimately the character of reconstruction.

⚠️Engineering Note

From Infinite Dimensions to Finite Grids

In practice, every imaging algorithm operates on a discretised grid. Replacing $L^2(\Omega)$ by $\mathbb{C}^N$ (a grid of $N$ voxels) introduces discretisation error: the continuous forward operator $\mathcal{A}$ is approximated by a finite matrix $\mathbf{A} \in \mathbb{C}^{M \times N}$ .

The theorems of this section (completeness, Parseval's identity, the projection theorem) have exact finite-dimensional counterparts, so the transition is clean. However, the grid resolution $N$ must be chosen to satisfy the sampling theorem for the maximum spatial frequency supported by the aperture — under-sampling introduces aliasing that no regularisation can remove.

Practical Constraints

•
Grid spacing $\Delta r \leq \lambda / (2 \mathrm{NA})$ where $\mathrm{NA}$ is the numerical aperture — this is the Nyquist criterion for imaging.
•
For 3-D scenes at 10 GHz with 10 cm aperture, a $\lambda/4 = 7.5$ mm grid gives $N \sim 10^6$ voxels — GPU memory becomes the binding constraint.
•
Coarser grids speed up computation but blur the PSF and reduce effective resolution; fine grids require $O(N^2)$ storage for the full sensing matrix $\mathbf{A}$ .

From Finite to Infinite Dimensions

Why Infinite-Dimensional Spaces for RF Imaging?

Historical Note: Stefan Banach and the Birth of Functional Analysis

Definition: Normed Space

Definition: LpL^pLp Spaces

Example: LpL^pLp Norms of a Rectangular Pulse

$L^1$ norm (total mass)

$L^2$ norm (energy)

$L^\infty$ norm (peak value)

Interpretation for scaled pulses

Definition: Sequence Spaces ℓp\ell^pℓp

Definition: Cauchy Sequence

Definition: Banach Space

Theorem: LpL^pLp Spaces are Banach Spaces

Extract a rapidly convergent subsequence

Show partial sums converge a.e.

Identify the limit and conclude $L^p$ convergence

LpL^pLp Unit Balls in R2\mathbb{R}^2R2

Parameters

Definition: Equivalence of Norms

Theorem: All Norms on Finite-Dimensional Spaces are Equivalent

Continuity of an arbitrary norm w.r.t. $\|\cdot\|_2$

Extract equivalence constants via compactness

Common Mistake: Norms Are NOT Equivalent in Infinite Dimensions

Why This Matters: Norms and Image Reconstruction Quality

Quick Check

Quick Check

Quick Check

Normed space

Banach space

LpL^pLp space

Completeness

Key Takeaway

From Infinite Dimensions to Finite Grids

Definition:
Normed Space

Definition:
$L^p$ Spaces

Example: $L^p$ Norms of a Rectangular Pulse

Definition:
Sequence Spaces $\ell^p$

Definition:
Cauchy Sequence

Definition:
Banach Space

Theorem: $L^p$ Spaces are Banach Spaces

$L^p$ Unit Balls in $\mathbb{R}^2$

Definition:
Equivalence of Norms

$L^p$ space