Ferkans — Interactive Telecom Tutor

From Norms to Inner Products — Why Geometry Matters

Hilbert spaces add geometric structure — angles, projections, orthogonality — to Banach spaces. For imaging this is decisive: projecting data onto subspaces is the foundation of matched filtering, least-squares estimation, and Fourier analysis of scattered fields.

The inner product enables:

Matched filtering. Detecting a known waveform in noise by projecting the received signal onto the waveform template: $\hat{s} = \langle y, h \rangle$ where $h$ is the expected impulse response.
Minimum-norm reconstruction. Finding the element of the range of $\mathbf{A}^{H}$ closest to the observed data — the pseudoinverse solution.
Fourier analysis of scattered fields. Expanding the scene's reflectivity $c$ in an orthonormal basis of complex exponentials to study $k$ -space coverage (Ch~7).

Definition:
Inner Product Space

An inner product on a vector space $\mathcal{V}$ over $\mathbb{F} \in \{\mathbb{R}, \mathbb{C}\}$ is a function $\langle \cdot, \cdot \rangle : \mathcal{V} \times \mathcal{V} \to \mathbb{F}$ satisfying, for all $f, g, h \in \mathcal{V}$ and $\alpha, \beta \in \mathbb{F}$ :

Conjugate symmetry. $\langle f, g \rangle = \overline{\langle g, f \rangle}$ .
Linearity in the first argument. $\langle \alpha f + \beta g, h \rangle = \alpha \langle f, h \rangle + \beta \langle g, h \rangle$ .
Positive definiteness. $\langle f, f \rangle \geq 0$ , with equality iff $f = 0$ .

A vector space equipped with an inner product is an inner product space. The induced norm is $\|f\| = \sqrt{\langle f, f \rangle}$ .

Convention. We follow the physics/engineering convention: linearity in the first argument, conjugate-linearity in the second. Some mathematics texts adopt the opposite. For $L^2(\Omega)$ :

$\langle f, g \rangle = \int_\Omega f(\mathbf{r})\,\overline{g(\mathbf{r})} \, d\mathbf{r}.$

Definition:
Hilbert Space

A Hilbert space is an inner product space that is complete with respect to the norm induced by its inner product. Equivalently, a Hilbert space is a Banach space whose norm is induced by an inner product.

Key examples:

$L^2(\Omega)$ with $\langle f, g \rangle = \int_\Omega f\,\bar{g}\,d\mathbf{r}$ .
$\ell^2$ (square-summable sequences) with $\langle \mathbf{x}, \mathbf{y} \rangle = \sum_{n=1}^\infty x_n\,\bar{y}_n$ .
$\mathbb{C}^n$ with $\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{y}^H \mathbf{x}$ .

$L^p$ for $p \neq 2$ is Banach but not Hilbert: the $L^p$ norm fails the parallelogram law

$\|f + g\|^2 + \|f - g\|^2 = 2\|f\|^2 + 2\|g\|^2$

whenever $p \neq 2$ . The parallelogram law is necessary and sufficient for a norm to arise from an inner product.

,

Theorem: Cauchy–Schwarz Inequality

For any $f, g$ in an inner product space $\mathcal{H}$ :

$|\langle f, g \rangle| \leq \|f\|\,\|g\|,$

with equality if and only if $f$ and $g$ are linearly dependent.

Show Hint

If $g = 0$ both sides are zero. For $g \neq 0$ , set $\alpha = \langle f, g \rangle / \|g\|^2$ and consider $\|f - \alpha g\|^2 \geq 0$ .

Expand the squared norm and collect terms in $|\langle f, g \rangle|^2$ .

Proof

Non-negativity of the residual

If $g = 0$ both sides vanish. For $g \neq 0$ , set $\alpha = \langle f, g \rangle / \|g\|^2$ . Then by positive definiteness:

$0 \leq \|f - \alpha g\|^2.$

Expand and rearrange

$\begin{aligned} 0 &\leq \|f\|^2 - \alpha\,\overline{\langle f, g\rangle} - \bar\alpha\,\langle f, g\rangle + |\alpha|^2\,\|g\|^2 \\ &= \|f\|^2 - \frac{|\langle f, g \rangle|^2}{\|g\|^2}. \end{aligned}$ $Rearranging gives$ |\langle f, g \rangle|^2 \leq |f|^2,|g|^2 $. Equality holds iff$ |f - \alpha g| = 0 $, i.e.,$ f = \alpha g $.$ \blacksquare$

Definition:
Orthogonality and Orthogonal Complement

Two elements $f, g$ of an inner product space are orthogonal, written $f \perp g$ , if $\langle f, g \rangle = 0$ .

The orthogonal complement of a subset $S \subseteq \mathcal{H}$ is

$S^\perp = \bigl\{f \in \mathcal{H} : \langle f, g \rangle = 0 \;\text{for all } g \in S\bigr\}.$

$S^\perp$ is always a closed subspace, even if $S$ itself is not.

Theorem: Orthogonal Projection Theorem

Let $\mathcal{M}$ be a closed subspace of a Hilbert space $\mathcal{H}$ . For every $f \in \mathcal{H}$ , there exists a unique $\hat{f} \in \mathcal{M}$ — the orthogonal projection — such that

$\hat{f} = \arg\min_{g \in \mathcal{M}} \|f - g\|.$

The minimiser is characterised by the orthogonality condition

$\langle f - \hat{f},\, g \rangle = 0 \quad \forall\, g \in \mathcal{M},$

and $\mathcal{H} = \mathcal{M} \oplus \mathcal{M}^\perp$ .

Proof

Existence via infimum and completeness

Let $d = \inf_{g \in \mathcal{M}} \|f - g\|$ and let $\{g_n\}$ be a minimising sequence with $\|f - g_n\| \to d$ . The parallelogram law gives

$\|g_m - g_n\|^2 = 2\|f - g_m\|^2 + 2\|f - g_n\|^2 - \|2f - g_m - g_n\|^2.$

Since $(g_m + g_n)/2 \in \mathcal{M}$ , the last term is $\geq 4d^2$ , so $\|g_m - g_n\|^2 \to 0$ . By completeness of the closed subspace $\mathcal{M}$ , $g_n \to \hat{f} \in \mathcal{M}$ with $\|f - \hat{f}\| = d$ .

Uniqueness via the parallelogram law

If $\hat{f}_1, \hat{f}_2$ both achieve the infimum, the parallelogram law gives $\|\hat{f}_1 - \hat{f}_2\|^2 \leq 4d^2 - 4d^2 = 0$ , so $\hat{f}_1 = \hat{f}_2$ .

Orthogonality via a variational argument

For any $g \in \mathcal{M}$ and $\epsilon \in \mathbb{R}$ , $\hat{f} + \epsilon g \in \mathcal{M}$ , so

$0 \leq \|f - \hat{f} - \epsilon g\|^2 - \|f - \hat{f}\|^2 = -2\epsilon\,\operatorname{Re}\langle f - \hat{f}, g \rangle + \epsilon^2\|g\|^2.$

Dividing by $\epsilon > 0$ and taking $\epsilon \downarrow 0$ : $\operatorname{Re}\langle f - \hat{f}, g \rangle \leq 0$ . Repeating with $\epsilon < 0$ gives equality; replacing $g$ by $jg$ kills the imaginary part. Hence $\langle f - \hat{f}, g \rangle = 0$ for all $g \in \mathcal{M}$ . $\blacksquare$

,

Orthogonal Projection in $L^2$

Project a function onto the span of the first $n$ Fourier basis functions. The residual (red) is orthogonal to the projection (blue): their inner product is zero up to numerical precision. Increasing $n$ reduces the residual energy — Parseval's identity in action.

Parameters

Number of Fourier basis functions

n

5

Historical Note: David Hilbert and the Geometry of Function Spaces

1900–1930

David Hilbert (1862–1943) introduced what we now call Hilbert spaces in his 1904–1910 investigations of integral equations — specifically, the Fredholm integral equation arising in potential theory. His key insight was that the space of square-integrable functions, equipped with an $L^2$ inner product, behaves geometrically like finite-dimensional Euclidean space: projections, orthogonal decompositions, and the Pythagorean theorem all generalise.

The term "Hilbert space" was coined by John von Neumann in 1927 while developing the mathematical foundations of quantum mechanics, where the state of a quantum system is a unit vector in an abstract Hilbert space. Von Neumann's axiomatisation (1927–1929) established the abstract definition still used today.

For RF imaging, the pivotal connection is Hilbert's study of compact integral operators — the same mathematical object that models the forward operator mapping a scene reflectivity to measured fields.

Definition:
Orthonormal Basis (ONB)

A countable set $\{e_n\}_{n=1}^\infty$ in a separable Hilbert space $\mathcal{H}$ is an orthonormal basis (ONB) if:

Orthonormality. $\langle e_m, e_n \rangle = \delta_{mn}$ .
Completeness (totality). $\overline{\operatorname{span}\{e_n : n \geq 1\}} = \mathcal{H}$ — the only element orthogonal to every $e_n$ is zero.

Theorem: Parseval's Identity

For any orthonormal basis $\{e_n\}$ of a Hilbert space $\mathcal{H}$ and any $f \in \mathcal{H}$ :

Expansion. $f = \sum_{n=1}^{\infty} \langle f, e_n \rangle\, e_n$ (series converges in the norm of $\mathcal{H}$ ).
Parseval's identity. $\|f\|^2 = \sum_{n=1}^{\infty} |\langle f, e_n \rangle|^2$ .

Example: Fourier Basis as an ONB of $L^2([0,1])$

Show that $\{e^{j 2\pi n x}\}_{n \in \mathbb{Z}}$ is an orthonormal basis for $L^2([0,1])$ . Compute the Fourier coefficients of the rectangular pulse $f(x) = \mathbf{1}_{[0, 1/2]}(x)$ and verify Parseval's identity.

Solution

Orthonormality

For $m, n \in \mathbb{Z}$ : $\langle e^{j2\pi mx}, e^{j2\pi nx} \rangle = \int_0^1 e^{j2\pi(m-n)x}\,dx = \delta_{mn}.$

Completeness (sketch)

By the Stone–Weierstrass theorem, trigonometric polynomials are dense in $C([0,1])$ , which is dense in $L^2([0,1])$ .

Fourier coefficients of the rectangular pulse

$c_n = \langle f, e^{j2\pi nx}\rangle = \int_0^{1/2} e^{-j2\pi nx}\,dx = \begin{cases} \tfrac{1}{2}, & n = 0, \\ \dfrac{1 - e^{-j\pi n}}{j2\pi n}, & n \neq 0. \end{cases}$ $

Verify Parseval's identity

The energy of $f$ is $\|f\|^2 = \int_0^{1/2} 1\,dx = \tfrac{1}{2}$ . One verifies $\sum_{n \in \mathbb{Z}} |c_n|^2 = \tfrac{1}{4} + \tfrac{1}{4} = \tfrac{1}{2}$ . $\checkmark$

Parseval's Identity: Energy Convergence

Partial sums $\sum_{|n| \leq N} |\langle f, e_n \rangle|^2$ converge to $\|f\|^2$ as $N \to \infty$ . The Gibbs phenomenon is visible in the reconstruction (overshoot at discontinuities), but the energy converges monotonically.

Parameters

Signal type

Why This Matters: Hilbert Spaces in RF Imaging

$L^2(\Omega)$ is the natural space for scene functions such as reflectivity $c(\mathbf{r})$ . The key connections:

Projection theorem $\to$ minimum-norm solutions. Underdetermined imaging problems (fewer measurements $M$ than unknowns $N$ ) have infinitely many solutions. The projection theorem gives the unique minimum-norm solution — the reconstruction with smallest $L^2$ energy consistent with the data.
Parseval's identity $\to$ spatial–spectral duality. The energy of $c$ can be computed in either domain. This is the foundation of diffraction tomography (Ch~7): each scattered-field measurement samples one Fourier coefficient of the scene on the Ewald sphere.
Orthogonal decomposition $\to$ resolution analysis. The null space $\ker(\mathbf{A})$ is the subspace of scenes invisible to the sensor. Decomposing $L^2(\Omega)$ into $\ker(\mathbf{A})^\perp$ (the visible component) and $\ker(\mathbf{A})$ (the invisible component) is a projection — the fundamental tool for characterising resolution and the limits of reconstruction.

See full treatment in Chapter 7

Quick Check

Which property distinguishes a Hilbert space from a general Banach space?

Existence of an inner product satisfying the parallelogram law.

Completeness (every Cauchy sequence converges).

Finite dimensionality.

The triangle inequality.

Correction:

Existence of an inner product satisfying the parallelogram law.

A Banach space is a complete normed space. A Hilbert space is a Banach space whose norm is induced by an inner product. The parallelogram law $\|f+g\|^2 + \|f-g\|^2 = 2\|f\|^2 + 2\|g\|^2$ is necessary and sufficient for a norm to come from an inner product.

Quick Check

The orthogonal projection of $f$ onto a closed subspace $\mathcal{M} \subset \mathcal{H}$ minimises ___.

The distance $\|f - g\|$ (norm of the residual) over $g \in \mathcal{M}$ .

The inner product $\langle f, g \rangle$ over $g \in \mathcal{M}$ .

The angle between $f$ and $g$ .

The dimension of $\mathcal{M}$ .

Correction:

The distance

\|f - g\|

(norm of the residual) over

g \in \mathcal{M}

.

The projection theorem guarantees that the orthogonal projection is the unique element of $\mathcal{M}$ closest to $f$ in norm — the best approximation from $\mathcal{M}$ .

Hilbert space

A complete inner product space. The prototypical example is $L^2(\Omega)$ . Adds geometric notions of angle, projection, and orthogonality to Banach spaces.

Inner product

A function $\langle \cdot, \cdot \rangle : \mathcal{V} \times \mathcal{V} \to \mathbb{F}$ satisfying conjugate symmetry, linearity in the first argument, and positive definiteness. Equips a vector space with notions of length, angle, and orthogonality.

Orthogonal projection

The unique closest element in a closed subspace $\mathcal{M}$ to a given $f$ . Characterised by the residual $f - \hat{f}$ being orthogonal to every element of $\mathcal{M}$ .

Orthonormal basis

A countable set $\{e_n\}$ in a separable Hilbert space satisfying $\langle e_m, e_n \rangle = \delta_{mn}$ and whose span is dense. Every element has the expansion $f = \sum_n \langle f, e_n \rangle e_n$ .

Key Takeaway

Four core messages of this section:

(1) Hilbert spaces add the geometry of angles and projections to complete normed spaces. The inner product is the essential new ingredient.

(2) The projection theorem guarantees a unique best approximation in any closed subspace, characterised by orthogonality of the residual. This underpins matched filtering, MMSE estimation, and Tikhonov regularisation.

(3) Parseval's identity enables spectral analysis: the energy of a scene can be computed equivalently from its expansion coefficients.

(4) All of this extends finite-dimensional linear algebra to the function spaces needed for imaging. $L^2(\Omega)$ is the workhorse Hilbert space for scene reflectivity and scattered fields.

Hilbert Spaces and Inner Products

From Norms to Inner Products — Why Geometry Matters

Definition: Inner Product Space

Definition: Hilbert Space

Theorem: Cauchy–Schwarz Inequality

Non-negativity of the residual

Expand and rearrange

Definition: Orthogonality and Orthogonal Complement

Theorem: Orthogonal Projection Theorem

Existence via infimum and completeness

Uniqueness via the parallelogram law

Orthogonality via a variational argument

Orthogonal Projection in L2L^2L2

Parameters

Historical Note: David Hilbert and the Geometry of Function Spaces

Definition: Orthonormal Basis (ONB)

Theorem: Parseval's Identity

Example: Fourier Basis as an ONB of L2([0,1])L^2([0,1])L2([0,1])

Orthonormality

Completeness (sketch)

Fourier coefficients of the rectangular pulse

Verify Parseval's identity

Parseval's Identity: Energy Convergence

Parameters

Why This Matters: Hilbert Spaces in RF Imaging

Quick Check

Quick Check

Hilbert space

Inner product

Orthogonal projection

Orthonormal basis

Key Takeaway

Definition:
Inner Product Space

Definition:
Hilbert Space

Definition:
Orthogonality and Orthogonal Complement

Orthogonal Projection in $L^2$

Definition:
Orthonormal Basis (ONB)

Example: Fourier Basis as an ONB of $L^2([0,1])$