Exercises

$\|f + g\|^2 = \langle f+g,\, f+g \rangle = \|f\|^2 + \langle f,g\rangle + \langle g,f\rangle + \|g\|^2 = \|f\|^2 + 2\operatorname{Re}\langle f,g\rangle + \|g\|^2.$$

Since $\operatorname{Re}\langle f,g\rangle \leq |\langle f,g\rangle|$, the Cauchy–Schwarz inequality gives $$\operatorname{Re}\langle f,g\rangle \leq |\langle f,g\rangle| \leq \|f\|\,\|g\|.$$ Substituting: $$\|f+g\|^2 \leq \|f\|^2 + 2\|f\|\,\|g\| + \|g\|^2 = \bigl(\|f\| + \|g\|\bigr)^2.$$

Both sides are non-negative, so taking square roots preserves the inequality: $$\|f + g\| \leq \|f\| + \|g\|. \quad\square$$

Since $f(x) \geq 0$ on $[-1,1]$: $$\|f\|_1 = \int_{-1}^{1} (1 - |x|)\,dx = 2\int_0^1 (1 - x)\,dx = 2\left[x - \frac{x^2}{2}\right]_0^1 = 2 \cdot \frac{1}{2} = 1.$$

$\|f\|_2^2 = \int_{-1}^{1} (1 - |x|)^2\,dx = 2\int_0^1 (1-x)^2\,dx = 2\left[-\frac{(1-x)^3}{3}\right]_0^1 = \frac{2}{3}.$$Therefore$|f|_2 = \sqrt{2/3}$.

The function $f(x) = 1 - |x|$ is continuous on $[-1,1]$ with maximum at $x = 0$: $$\|f\|_\infty = \max_{x \in [-1,1]} |f(x)| = f(0) = 1.$$

We have $\|f\|_1 = 1$, $\|f\|_2 = \sqrt{2/3} \approx 0.816$, and $\|f\|_\infty = 1$. On a bounded domain, $\|f\|_p$ is generally a decreasing function of $p$ (up to normalization by the domain measure), consistent with the general inclusion $L^q \subset L^p$ for $q \geq p$.

Using the standard inner product on $\mathbb{C}^M$: $$\langle \mathbf{A}\mathbf{x},\, \mathbf{y} \rangle_{\mathbb{C}^M} = \mathbf{y}^H (\mathbf{A}\mathbf{x}) = (\mathbf{y}^H \mathbf{A})\,\mathbf{x}.$$

Note that $\mathbf{y}^H \mathbf{A} = (\mathbf{A}^H \mathbf{y})^H$, so $$\mathbf{y}^H \mathbf{A}\,\mathbf{x} = (\mathbf{A}^H \mathbf{y})^H \mathbf{x} = \langle \mathbf{x},\, \mathbf{A}^H \mathbf{y} \rangle_{\mathbb{C}^N}.$$

Since the identity $\langle \mathbf{A}\mathbf{x}, \mathbf{y}\rangle = \langle \mathbf{x}, \mathbf{A}^H\mathbf{y}\rangle$ holds for all $\mathbf{x} \in \mathbb{C}^N$ and $\mathbf{y} \in \mathbb{C}^M$, the adjoint of $\mathbf{A}$ is $\mathbf{A}^* = \mathbf{A}^H$. $\square$

$\langle 1, \cos(2\pi x)\rangle = \int_0^1 \cos(2\pi x)\,dx = \frac{\sin(2\pi x)}{2\pi}\Big|_0^1 = 0.$$So${1, \cos(2\pi x)}$are orthogonal. Their norms are$|1| = 1$and$|\cos(2\pi x)| = 1/\sqrt{2}$, giving the orthonormal basis${e_1, e_2} = {1,, \sqrt{2}\cos(2\pi x)}$.

$\langle f, e_1 \rangle = \int_0^1 x \cdot 1\,dx = \frac{1}{2}.KATEXPLACEHOLDER0END\langle f, e_2 \rangle = \sqrt{2}\int_0^1 x\cos(2\pi x)\,dx.$$Integration by parts:$\int_0^1 x\cos(2\pi x),dx = \left[\frac{x\sin(2\pi x)}{2\pi}\right]_0^1 - \int_0^1 \frac{\sin(2\pi x)}{2\pi},dx = 0 + \frac{\cos(2\pi x)}{4\pi^2}\Big|_0^1 = 0.$Therefore$\langle f, e_2 \rangle = 0$.

$P_V f = \langle f, e_1\rangle\, e_1 + \langle f, e_2\rangle\, e_2 = \frac{1}{2} \cdot 1 + 0 = \frac{1}{2}.$$The orthogonal projection of$f(x) = x$onto$V = \operatorname{span}{1, \cos(2\pi x)}$is the constant function$1/2$.

The residual is $f - P_V f = x - 1/2$. We check: $$\langle x - 1/2,\, 1\rangle = \int_0^1 (x - 1/2)\,dx = 0, \qquad \langle x - 1/2,\, \cos(2\pi x)\rangle = 0$$ (the second follows from the computation above). The residual is orthogonal to $V$, confirming our projection. $\square$

Let $\{\mathbf{x}^{(k)}\}$ be Cauchy in $\ell^2$. For each fixed $n$: $$|x_n^{(j)} - x_n^{(k)}|^2 \leq \sum_{m=1}^\infty |x_m^{(j)} - x_m^{(k)}|^2 = \|\mathbf{x}^{(j)} - \mathbf{x}^{(k)}\|_{\ell^2}^2 \to 0.$$ So $\{x_n^{(k)}\}_{k=1}^\infty$ is Cauchy in $\mathbb{C}$ and converges to some $x_n^* \in \mathbb{C}$. Define $\mathbf{x}^* = (x_1^*, x_2^*, \ldots)$.

Given $\varepsilon > 0$, choose $K$ such that $\|\mathbf{x}^{(j)} - \mathbf{x}^{(k)}\|_{\ell^2} < \varepsilon$ for all $j, k \geq K$. For any finite $N$: $$\sum_{n=1}^N |x_n^{(j)} - x_n^{(k)}|^2 \leq \varepsilon^2.$$ Letting $j \to \infty$ (limits commute with finite sums): $$\sum_{n=1}^N |x_n^* - x_n^{(k)}|^2 \leq \varepsilon^2 \quad \text{for all } k \geq K \text{ and all } N.$$ Now let $N \to \infty$: $\|\mathbf{x}^* - \mathbf{x}^{(k)}\|_{\ell^2}^2 \leq \varepsilon^2$, so $\|\mathbf{x}^* - \mathbf{x}^{(k)}\|_{\ell^2} \leq \varepsilon$.

By the triangle inequality: $$\|\mathbf{x}^*\|_{\ell^2} \leq \|\mathbf{x}^* - \mathbf{x}^{(K)}\|_{\ell^2} + \|\mathbf{x}^{(K)}\|_{\ell^2} \leq \varepsilon + \|\mathbf{x}^{(K)}\|_{\ell^2} < \infty.$$ Therefore $\mathbf{x}^* \in \ell^2$, and $\mathbf{x}^{(k)} \to \mathbf{x}^*$ in $\ell^2$ norm. $\square$

We can write $\mathcal{A}f = \langle f, \mathbf{1}\rangle_{L^2}$ where $\mathbf{1}(x) = 1$ is the constant function.

By Cauchy–Schwarz: $$|\mathcal{A}f| = |\langle f, \mathbf{1}\rangle| \leq \|f\|_{L^2}\,\|\mathbf{1}\|_{L^2} = \|f\|_{L^2} \cdot 1.$$ Therefore $\|\mathcal{A}\| \leq 1$.

Take $f = \mathbf{1}$, which has $\|f\|_{L^2} = 1$. Then $$|\mathcal{A}f| = \left|\int_0^1 1\,dx\right| = 1 = \|f\|_{L^2}.$$ The supremum is achieved, so $\|\mathcal{A}\| = 1$.

More generally, for the Riesz functional $\mathcal{A}f = \langle f, g\rangle$, the Riesz representation theorem guarantees $\|\mathcal{A}\| = \|g\|$, achieved by $f = g/\|g\|$. Our case is $g = \mathbf{1}$.

The sequence $\{e_n\}$ is bounded ($\|e_n\| = 1$) but has no convergent subsequence (since $\|e_j - e_k\| = \sqrt{2}$ for $j \neq k$).

If $\mathcal{D}$ is compact, $\{\mathcal{D}e_n\} = \{d_n e_n\}$ must have a convergent subsequence. But $\|d_j e_j - d_k e_k\|^2 = |d_j|^2 + |d_k|^2$ for $j \neq k$. For a subsequence $\{d_{n_k} e_{n_k}\}$ to converge, we need $|d_{n_k}| \to 0$.

If $d_n \not\to 0$, there exists $\varepsilon > 0$ and a subsequence with $|d_{n_k}| \geq \varepsilon$. Then no sub-subsequence of $\{d_{n_k} e_{n_k}\}$ can converge, contradicting compactness.

Define $\mathcal{D}_N$ by $\mathcal{D}_N e_n = d_n e_n$ for $n \leq N$ and $\mathcal{D}_N e_n = 0$ for $n > N$. Each $\mathcal{D}_N$ has finite rank $\leq N$, hence is compact.

For any $f = \sum_n c_n e_n$ with $\|f\| = 1$: $$\|(\mathcal{D} - \mathcal{D}_N)f\|^2 = \sum_{n > N} |d_n|^2 |c_n|^2 \leq \sup_{n > N} |d_n|^2 \sum_{n > N} |c_n|^2 \leq \sup_{n > N} |d_n|^2.$$ Therefore $\|\mathcal{D} - \mathcal{D}_N\| \leq \sup_{n > N} |d_n| \to 0$ since $d_n \to 0$.

Since $\mathcal{D}$ is the operator-norm limit of finite-rank operators, and the compact operators form a closed subspace of the bounded operators, $\mathcal{D}$ is compact. $\square$

$(\mathcal{A}e_n)(x) = \int_0^1 K(x-y)\,e^{2\pi i n y}\,dy.KATEXPLACEHOLDER0END= \int_0^1 K(t)\,e^{2\pi i n (x-t)}\,dt = e^{2\pi i n x} \int_0^1 K(t)\,e^{-2\pi i n t}\,dt.$$

$(\mathcal{A}e_n)(x) = \hat{K}(n)\,e_n(x)$$where$\hat{K}(n) = \int_0^1 K(t) e^{-2\pi i n t},dt$is the$n$-th Fourier coefficient of$K$. Therefore$e_n$is an eigenfunction with eigenvalue$\lambda_n = \hat{K}(n)$.

Since $\{e_n\}_{n \in \mathbb{Z}}$ forms an orthonormal basis for $L^2([0,1])$, the operator $\mathcal{A}$ is diagonalized by the Fourier basis: $$\mathcal{A}f = \sum_{n \in \mathbb{Z}} \hat{K}(n)\,\hat{f}(n)\,e_n.$$ The operator is compact if and only if $\hat{K}(n) \to 0$ as $|n| \to \infty$ (by Exercise 7).

This is the continuous analog of the discrete result: circulant matrices are diagonalized by the DFT matrix. If $\mathbf{A}$ is an $N \times N$ circulant matrix with first column $\mathbf{k}$, then $\mathbf{A} = \mathbf{F}^H \operatorname{diag}(\mathbf{F}\mathbf{k})\,\mathbf{F}$ where $\mathbf{F}$ is the DFT matrix.

The Picard condition requires: $$\sum_{n=1}^\infty \frac{|\langle y, u_n \rangle|^2}{\sigma_n^2} < \infty.$$ Substituting $|\langle y, u_n\rangle| = n^{-3}$ and $\sigma_n = n^{-2}$: $$\sum_{n=1}^\infty \frac{n^{-6}}{n^{-4}} = \sum_{n=1}^\infty n^{-2} = \frac{\pi^2}{6} < \infty.$$ The Picard condition is satisfied.

Since the $\{v_n\}$ are orthonormal: $$\|g^\dagger\|^2 = \sum_{n=1}^\infty \frac{|\langle y, u_n\rangle|^2}{\sigma_n^2} = \sum_{n=1}^\infty n^{-2} = \frac{\pi^2}{6} \approx 1.645.$$

If $|\langle y, u_n\rangle| = n^{-2}$ instead, the Picard sum becomes: $$\sum_{n=1}^\infty \frac{n^{-4}}{n^{-4}} = \sum_{n=1}^\infty 1 = \infty.$$ The Picard condition fails — the data coefficients decay at the same rate as the singular values, so the generalized inverse solution has infinite norm. This is precisely the ill-posedness that regularization must address.

For $q < \infty$, let $r = q/p \geq 1$ and $r' = r/(r-1) = q/(q-p)$. Apply Hölder's inequality to $|f|^p \cdot 1$: $$\int_\Omega |f|^p\,dx = \int_\Omega |f|^p \cdot 1\,dx \leq \left(\int_\Omega |f|^{pr}\,dx\right)^{1/r} \left(\int_\Omega 1^{r'}\,dx\right)^{1/r'}.$$

Since $pr = q$: $$\|f\|_p^p \leq \|f\|_q^p \cdot |\Omega|^{1/r'} = \|f\|_q^p \cdot |\Omega|^{(q-p)/q}.$$ Taking the $p$-th root: $$\|f\|_p \leq \|f\|_q \cdot |\Omega|^{(q-p)/(pq)} = |\Omega|^{1/p - 1/q}\,\|f\|_q.$$

$\|f\|_p^p = \int_\Omega |f|^p\,dx \leq \|f\|_\infty^p \int_\Omega dx = \|f\|_\infty^p\,|\Omega|.$$Therefore$|f|p \leq |\Omega|^{1/p},|f|\infty$, matching the formula with$1/q = 0$.

For $f = c$ (constant), $\|f\|_p = |c|\,|\Omega|^{1/p}$ and $\|f\|_q = |c|\,|\Omega|^{1/q}$, so $$\frac{\|f\|_p}{\|f\|_q} = |\Omega|^{1/p - 1/q},$$ achieving equality. The embedding constant is sharp. $\square$

For all $f \in \mathcal{H}_1$ and $g \in \mathcal{H}_2$: $$\langle \mathcal{A}f, g\rangle_{\mathcal{H}_2} = \langle \langle f, v\rangle u,\, g\rangle = \langle f, v\rangle\,\langle u, g\rangle = \langle f,\, \overline{\langle u, g\rangle}\, v\rangle = \langle f,\, \langle g, u\rangle\, v\rangle_{\mathcal{H}_1}.$$ Therefore $\mathcal{A}^* g = \langle g, u\rangle\, v$.

$\mathcal{A}^*\mathcal{A}f = \mathcal{A}^*(\langle f,v\rangle u) = \langle f,v\rangle \langle u,u\rangle\, v = \langle f,v\rangle\,v$$since$|u| = 1$. So$\mathcal{A}^\mathcal{A}$is the orthogonal projection onto$\operatorname{span}{v}$. Similarly,$\mathcal{A}\mathcal{A}^ g = \langle g,u\rangle u$is the projection onto$\operatorname{span}{u}$.

Since $\mathcal{A}^*\mathcal{A}\,v = v$, the eigenvalue is $\sigma_1^2 = 1$, so $\sigma_1 = 1$.

• Right singular vector: $v_1 = v$
• Left singular vector: $u_1 = u$
• SVD: $\mathcal{A} = \sigma_1\, u_1 \otimes v_1^* = u \otimes v^*$

where $(u \otimes v^*)f = \langle f, v\rangle u$.

For $\mathcal{A}f = \sigma\langle f,v\rangle u$, the same analysis gives $\mathcal{A}^*\mathcal{A}f = \sigma^2\langle f,v\rangle v$, so the singular value is $\sigma_1 = \sigma$, with the same singular vectors. The SVD is $\mathcal{A} = \sigma\, u \otimes v^*$.

This is the operator-theoretic analog of the matrix SVD for a rank-one matrix $\mathbf{A} = \sigma\, \mathbf{u}\mathbf{v}^H$. $\square$

For smooth $f$ (the general case follows by density), the mean value theorem for integrals gives a point $a \in [0,1]$ such that $f(a) = \int_0^1 f(t)\,dt$. For any $x \in [0,1]$: $$f(x) = f(a) + \int_a^x f'(t)\,dt.$$

First term: By Cauchy–Schwarz on $[0,1]$: $$|f(a)| = \left|\int_0^1 f(t)\,dt\right| \leq \|f\|_{L^2}\,\|1\|_{L^2} = \|f\|_{L^2}.$$

Second term: By Cauchy–Schwarz: $$\left|\int_a^x f'(t)\,dt\right| \leq \sqrt{|x - a|}\,\|f'\|_{L^2} \leq \|f'\|_{L^2}.$$

$|f(x)| \leq \|f\|_{L^2} + \|f'\|_{L^2} \leq \sqrt{2}\,\sqrt{\|f\|_{L^2}^2 + \|f'\|_{L^2}^2} = \sqrt{2}\,\|f\|_{H^1}$$where the last step uses the Cauchy–Schwarz inequality$(a + b)^2 \leq 2(a^2 + b^2)$. Taking the supremum over$x$:$|f|\infty \leq \sqrt{2},|f|{H^1}$.

The representation $f(x) = f(a) + \int_a^x f'(t)\,dt$ shows $f$ is absolutely continuous (hence continuous), since $f' \in L^2 \subset L^1$ on bounded domains. Therefore every $H^1([0,1])$ function has a continuous representative, establishing the embedding $H^1([0,1]) \hookrightarrow C([0,1])$. $\square$

Remark: This embedding is specific to 1D. In $d$ dimensions, $H^1(\Omega) \hookrightarrow C(\Omega)$ requires $1 > d/2$, i.e., $d = 1$ only.

Let $\{e_n\}$ be any orthonormal basis of $L^2(\Omega')$. By definition: $$\|\mathcal{A}\|_{\text{HS}}^2 = \sum_n \|\mathcal{A}e_n\|_{L^2(\Omega)}^2 = \sum_n \int_\Omega \left|\int_{\Omega'} K(\mathbf{r}, \mathbf{r}')\,e_n(\mathbf{r}')\,d\mathbf{r}'\right|^2 d\mathbf{r}.$$

For fixed $\mathbf{r}$, the function $\mathbf{r}' \mapsto K(\mathbf{r}, \mathbf{r}')$ lies in $L^2(\Omega')$. Its Fourier coefficients in the basis $\{e_n\}$ are $c_n(\mathbf{r}) = \int_{\Omega'} K(\mathbf{r}, \mathbf{r}') \overline{e_n(\mathbf{r}')}\,d\mathbf{r}'$. By Parseval: $$\sum_n |c_n(\mathbf{r})|^2 = \int_{\Omega'} |K(\mathbf{r}, \mathbf{r}')|^2\,d\mathbf{r}'.$$

Note: $|\langle K(\mathbf{r},\cdot), e_n\rangle|^2 = |\int K(\mathbf{r},\mathbf{r}')e_n(\mathbf{r}')d\mathbf{r}'|^2$ since we can absorb the conjugate into the kernel. Integrating over $\mathbf{r}$: $$\|\mathcal{A}\|_{\text{HS}}^2 = \int_\Omega \int_{\Omega'} |K(\mathbf{r}, \mathbf{r}')|^2\,d\mathbf{r}'\,d\mathbf{r}.$$