Exercises

$\hat{\mathbf{c}}^{\text{BP}} = \mathbf{A}^{H} \mathbf{y} = \mathbf{A}^{H}(\mathbf{A}\mathbf{c} + \mathbf{w}) = \underbrace{\mathbf{A}^{H}\mathbf{A}}_{\mathbf{G}}\,\mathbf{c} + \underbrace{\mathbf{A}^{H}\mathbf{w}}_{\tilde{\mathbf{w}}}.$$

For white noise $\mathbf{w} \sim \mathcal{CN}(\mathbf{0}, \sigma^2\mathbf{I}_M)$:

$$\mathbb{E}[\tilde{\mathbf{w}}\tilde{\mathbf{w}}^H] = \mathbf{A}^{H} \cdot \sigma^2\mathbf{I} \cdot \mathbf{A} = \sigma^2\,\mathbf{G}.$$

The back-projected noise is coloured with the same correlation structure as the PSF (Gram matrix). This is the key difficulty: noise and sidelobe artefacts share the same spatial pattern.

Apply $\mathbf{A}$ to the DC output: $$\mathbf{A}\,\text{DC}(\hat{\mathbf{c}}) = \mathbf{A}\hat{\mathbf{c}} - \mathbf{A}\mathbf{A}^{H}(\mathbf{A}\hat{\mathbf{c}} - \mathbf{y}) = \mathbf{A}\hat{\mathbf{c}} - (\mathbf{A}\hat{\mathbf{c}} - \mathbf{y}) = \mathbf{y}.$$

Let $\mathbf{z} = \text{DC}(\hat{\mathbf{c}})$. We showed $\mathbf{A}\mathbf{z} = \mathbf{y}$. Therefore: $$\text{DC}(\mathbf{z}) = \mathbf{z} - \mathbf{A}^{H}(\mathbf{A}\mathbf{z} - \mathbf{y}) = \mathbf{z} - \mathbf{A}^{H}(\mathbf{y} - \mathbf{y}) = \mathbf{z}. \quad\square$$

For any fixed $y$, the MSE decomposes as $\mathbb{E}[\|f(y) - c\|^2 \mid y] = \|f(y) - \mathbb{E}[c \mid y]\|^2 + \operatorname{Var}(c \mid y)$. The second term is irreducible, so $f^*(y) = \mathbb{E}[c \mid y]$.

$P(c = 1 \mid y) = \frac{e^{-(y-1)^2/(2\sigma^2)}}{e^{-(y-1)^2/(2\sigma^2)} + e^{-(y+1)^2/(2\sigma^2)}} = \frac{1}{1 + e^{-2y/\sigma^2}}.$$Therefore$f^(y) = \mathbb{E}[c \mid y] = P(c=1|y) - P(c=-1|y) = \tanh(y/\sigma^2)$. For$|y| \ll \sigma$(low SNR):$f^(y) \approx 0$, demonstrating the blurring effect — the network outputs a value that never occurs in the true scene.

$G_{ii} = \|\mathbf{a}_i\|^2 = \sum_{m=1}^M |A_{mi}|^2$. Each $|A_{mi}|^2$ has mean $\mathbb{E}[|A_{mi}|^2] = 1/M$ for $A_{mi} \sim \mathcal{CN}(0, 1/M)$. By linearity, $\mathbb{E}[G_{ii}] = M \cdot (1/M) = 1$.

For $i \neq j$: $G_{ij} = \sum_{m=1}^M \overline{A_{mi}} A_{mj}$. Since $A_{mi}$ and $A_{mj}$ are independent for $i \neq j$: $\mathbb{E}[\overline{A_{mi}} A_{mj}] = \overline{\mathbb{E}[A_{mi}]} \cdot \mathbb{E}[A_{mj}] = 0$. Hence $\mathbb{E}[G_{ij}] = 0$.

$\operatorname{Var}(G_{ij}) = \sum_{m=1}^M \mathbb{E}[|\overline{A_{mi}} A_{mj}|^2] = M \cdot \mathbb{E}[|A_{mi}|^2]\mathbb{E}[|A_{mj}|^2] = M \cdot \frac{1}{M} \cdot \frac{1}{M} = \frac{1}{M}$.

As $M \to \infty$: off-diagonal entries $\to 0$ in probability, so $\mathbf{G} \to \mathbf{I}$. The MF image satisfies $\hat{\mathbf{c}}^{\text{BP}} \approx \mathbf{c} + \text{white noise}$ — the U-Net task reduces to simple denoising. $\square$

Normalise so $\mathbf{F}^H\mathbf{F} = \mathbf{I}$ (unitary DFT). Then: $\mathbf{A}\mathbf{A}^{H} = \mathbf{P}_\Omega\mathbf{F}\mathbf{F}^H\mathbf{P}_\Omega^H = \mathbf{P}_\Omega\mathbf{I}\mathbf{P}_\Omega^H = \mathbf{P}_\Omega\mathbf{P}_\Omega^H = \mathbf{I}_M$.

$\text{DC}(\hat{\mathbf{c}}) = \hat{\mathbf{c}} - \mathbf{F}^H\mathbf{P}_\Omega^H(\mathbf{P}_\Omega\mathbf{F}\hat{\mathbf{c}} - \mathbf{y}).$$

Let $\hat{X} = \mathbf{F}\hat{\mathbf{c}}$ (DFT of network estimate). The DC layer sets: $$[\mathbf{F}\,\text{DC}(\hat{\mathbf{c}})]_k = \begin{cases} y_k & k \in \Omega \\ \hat{X}_k & k \notin \Omega. \end{cases}$$ Acquired k-space samples are replaced by the measurements; unacquired locations are filled by the network. The inverse DFT then gives the final image. $\square$

The objective is $\frac{1}{2}\|\mathbf{A}\mathbf{c} - \mathbf{y}\|^2 + \frac{\lambda_k}{2}\|\mathbf{c} - \mathbf{z}_k\|^2$. Taking the gradient and setting to zero: $$\mathbf{A}^{H}(\mathbf{A}\mathbf{c} - \mathbf{y}) + \lambda_k(\mathbf{c} - \mathbf{z}_k) = \mathbf{0}$$ $$(\mathbf{A}^{H}\mathbf{A} + \lambda_k\mathbf{I})\mathbf{c} = \mathbf{A}^{H}\mathbf{y} + \lambda_k\mathbf{z}_k.$$ Solving: $\hat{\mathbf{c}}_k = (\mathbf{A}^{H}\mathbf{A} + \lambda_k\mathbf{I})^{-1}(\mathbf{A}^{H}\mathbf{y} + \lambda_k\mathbf{z}_k)$. $\square$

$\lambda_k \to 0$: $\hat{\mathbf{c}}_k \to (\mathbf{A}^{H}\mathbf{A})^\dagger\mathbf{A}^{H}\mathbf{y}$ (pseudoinverse — pure data fidelity, ignores denoiser).

$\lambda_k \to \infty$: Divide by $\lambda_k$: $\hat{\mathbf{c}}_k \to \mathbf{z}_k$ (pure denoiser output, ignores measurements).

The step size $\lambda_k$ controls the data-vs-prior balance.

The VGG feature extractor $\phi_\ell$ maps images to feature maps via convolutions and ReLU activations. This mapping is not injective. Add a high-frequency perturbation $\mathbf{p}$ with spatial frequency above VGG's sensitivity (e.g., alternating $\pm\epsilon$ checkerboard). For small $\epsilon$, the pooling layers in VGG average out this pattern: $\phi_\ell(\hat{\mathbf{c}} + \mathbf{p}) \approx \phi_\ell(\hat{\mathbf{c}})$ for all selected layers.

Set $\hat{\mathbf{c}} = \mathbf{c} + \mathbf{p}$ where $\mathbf{p}$ is a high-frequency checkerboard pattern invisible to VGG. Then $\mathcal{L}_{\text{perc}}(\hat{\mathbf{c}}, \mathbf{c}) = 0$ but $\hat{\mathbf{c}} \neq \mathbf{c}$. The perceptual loss is a pseudo-metric (violates identity of indiscernibles).

This means perceptual loss alone cannot guarantee pixel-level fidelity. High-frequency target signatures (e.g., the precise location of a point reflector to sub-pixel accuracy) may be corrupted without any perceptual loss penalty. In RF imaging, always combine perceptual loss with a pixel-wise term (MSE or $L^1$). $\square$

$\|f_\theta(\hat{\mathbf{c}}^{\text{BP}}_{2}) - \mathbf{c}\| \leq \underbrace{\|f_\theta(\hat{\mathbf{c}}^{\text{BP}}_{2}) - f_\theta(\hat{\mathbf{c}}^{\text{BP}}_{1})\|}_{\text{transfer error}} + \underbrace{\|f_\theta(\hat{\mathbf{c}}^{\text{BP}}_{1}) - \mathbf{c}\|}_{\text{training error}}.$$

By Lipschitz continuity of $f_\theta$: $$\|f_\theta(\hat{\mathbf{c}}^{\text{BP}}_{2}) - f_\theta(\hat{\mathbf{c}}^{\text{BP}}_{1})\| \leq L_f \|\hat{\mathbf{c}}^{\text{BP}}_{2} - \hat{\mathbf{c}}^{\text{BP}}_{1}\| = L_f \|(\mathbf{G}_2 - \mathbf{G}_1)\mathbf{c} + (\tilde{\mathbf{w}}_2 - \tilde{\mathbf{w}}_1)\|.$$

$\|f_\theta(\hat{\mathbf{c}}^{\text{BP}}_{2}) - \mathbf{c}\| \leq L_f\bigl(\|\mathbf{G}_1 - \mathbf{G}_2\|\cdot\|\mathbf{c}\| + \|\tilde{\mathbf{w}}_2 - \tilde{\mathbf{w}}_1\|\bigr) + \text{training error}.$$The transfer error is controlled by$|\mathbf{G}_1 - \mathbf{G}_2|$. For random matrices (where$\mathbf{G} \approx \mathbf{I}$for any geometry), this bound is tight. For physical operators with different geometries, the bound can be large, confirming that MF→U-Net requires retraining when the sensing operator changes significantly.$\square$

$[\hat{\mathbf{c}}^{\text{BP}}]_j = [\mathbf{G}\mathbf{c}]_j + [\tilde{\mathbf{w}}]_j = G_{ji_0} c + [\tilde{\mathbf{w}}]_j.$

The sidelobe is $G_{ji_0} c$ — it exists even for $j \neq i_0$.

The back-projected noise has covariance $[\operatorname{Cov}(\tilde{\mathbf{w}})]_{jk} = \sigma^2 G_{jk}$. The signal at $j$ due to the point target has energy $|G_{ji_0}|^2|c|^2$. The covariance between noise at pixel $j$ and the sidelobe is: $$\operatorname{Cov}([\tilde{\mathbf{w}}]_j,\; G_{ji_0}c) = \sigma^2 G_{jj} \cdot G_{ji_0} \neq 0$$ whenever $G_{jj} \neq 0$ and $G_{ji_0} \neq 0$.

More precisely, the noise at pixel $j$ is $[\tilde{\mathbf{w}}]_j = \sum_m A_{mj}^* n_m$. The sidelobe contribution from target $i_0$ to pixel $j$ is $\sum_m A_{mj}^* A_{mi_0} c$. These share the same factor $A_{mj}^*$, making them statistically dependent.

Given only $[\hat{\mathbf{c}}^{\text{BP}}]_j$, the U-Net observes the sum $G_{ji_0} c + [\tilde{\mathbf{w}}]_j$. These two terms are correlated because they both involve $\mathbf{A}^{H}$. A feature at pixel $j$ could be: (a) a real target with amplitude $\sim G_{ji_0}c$, or (b) noise that looks like a target. Without additional information about the sensing geometry and the true scene at $i_0$, the U-Net cannot differentiate these cases from $\hat{\mathbf{c}}^{\text{BP}}$ alone.

This is the sidelobe corruption problem: structured sensing operators create correlated noise and signal components that are statistically inseparable from a single MF image. $\square$

The CG step solves $\hat{\mathbf{c}}_{k+1} = \arg\min_{\mathbf{c}} \frac{1}{2}\|\mathbf{A}\mathbf{c}-\mathbf{y}\|^2 + \frac{\mu}{2}\|\mathbf{c} - \mathbf{z}_k\|^2$ where $\mathbf{z}_k = \text{prox}_{\lambda R}(\hat{\mathbf{c}}_k)$.

This is equivalent to: $$\hat{\mathbf{c}}_{k+1} = \text{prox}_{\frac{1}{\mu}\|\mathbf{A}\cdot-\mathbf{y}\|^2/2}(\mathbf{z}_k) = \text{prox}_{\frac{1}{2\mu}\|\mathbf{A}\cdot-\mathbf{y}\|^2}(\text{prox}_{\lambda R}(\hat{\mathbf{c}}_k)).$$

This is a proximal-proximal splitting (alternating proximal algorithm). Convergence to the minimiser of $\frac{1}{2}\|\mathbf{A}\mathbf{c}-\mathbf{y}\|^2 + \lambda R(\mathbf{c})$ holds when:

• $R$ is convex and lower semicontinuous.
• The regularisation weight satisfies $\lambda/\mu \leq 1/L$ where $L = \|\mathbf{A}\|^2$ (spectral norm squared).
• The step size $\mu$ is bounded above by twice the smallest eigenvalue of $\mathbf{A}^{H}\mathbf{A}$ in the relevant subspace. $\square$

At the input resolution, two $3\times 3$ convolutions give $r_0 = 2 + 3 = 5$ pixels.

At level $\ell$ (after $\ell$ downsampling steps), each pixel covers $2^\ell$ pixels at the original scale. Two $3\times 3$ convolutions at level $\ell$ add $2 \times 2 \times 2^\ell = 4 \cdot 2^\ell$ pixels to the receptive field: $$r_\ell = r_{\ell-1} + 4 \cdot 2^\ell.$$

$r_L = r_0 + \sum_{\ell=1}^L 4 \cdot 2^\ell = 5 + 4 \cdot 2(2^L - 1) = 4 \cdot 2^{L+1} - 3.$$For$L = 4$:$r_4 = 4 \cdot 32 - 3 = 125$pixels. **SAR implication:** SAR PSF sidelobes from the Doppler ambiguity can extend hundreds of resolution cells in the cross-range direction. A 4-level U-Net with 125-pixel receptive field will miss long-range sidelobes entirely. This requires either deeper U-Nets (larger$L$), dilated convolutions, or explicit PSF-conditioned architectures that handle long-range correlations analytically.$\square$

Given $\hat{\mathbf{c}}^{\text{BP}} = \mathbf{G}\mathbf{c} + \tilde{\mathbf{w}}$ with $\operatorname{Cov}(\tilde{\mathbf{w}}) = \sigma^2\mathbf{G}$ and $\mathbb{E}[\mathbf{c}\mathbf{c}^{H}] = \sigma_c^2\mathbf{I}$, the Wiener filter is:

$$\hat{\mathbf{c}}^{\text{blind}} = \sigma_c^2\mathbf{G}^H (\sigma_c^2\mathbf{G}\mathbf{G}^H + \sigma^2\mathbf{G})^{-1} \hat{\mathbf{c}}^{\text{BP}}.$$

MSE is $\operatorname{tr}(\sigma_c^2\mathbf{I} - \sigma_c^2\mathbf{G}^H(\sigma_c^2\mathbf{G}\mathbf{G}^H + \sigma^2\mathbf{G})^{-1}\sigma_c^2\mathbf{G})$.

When $\mathbf{G}$ is provided, the per-pixel effective SNR $\text{SNR}_i = \sigma_c^2 G_{ii}/\sigma^2$ is known. For a diagonal $\mathbf{G}$ (shift-invariant approximation), the informed Wiener filter decomposes per-pixel: $$\hat{c}_i^{\text{informed}} = \frac{\sigma_c^2 G_{ii}}{\sigma_c^2 G_{ii}^2 + \sigma^2 G_{ii}} [\hat{\mathbf{c}}^{\text{BP}}]_i = \frac{\sigma_c^2 G_{ii}}{\sigma_c^2 G_{ii} + \sigma^2} \cdot \frac{[\hat{\mathbf{c}}^{\text{BP}}]_i}{G_{ii}}.$$ MSE is $\sum_i \frac{\sigma_c^2 \sigma^2}{\sigma_c^2 G_{ii} + \sigma^2}$.

For the blind estimator, the Wiener weights are the same for all pixels (assuming uniform $\mathbf{G}$). For the informed estimator, pixels with large $G_{ii}$ (high SNR) are relied upon more; pixels with small $G_{ii}$ (low SNR, i.e., dark side of the beam) are trusted less. When $\mathbf{G}$ has non-constant diagonal (physically structured operators), the informed estimator achieves strictly lower MSE. $\square$

Let $\mathbf{G} = \mathbf{U}\boldsymbol{\Lambda}\mathbf{U}^H$ with eigenvalues $g_i$. In this basis, the MoDL update is scalar per mode $i$: $$\hat{c}_i = \frac{g_i y_i' + \lambda d_i c_{0,i}'}{g_i + \lambda},$$ where $y_i' = [\mathbf{U}^H\mathbf{A}^{H}\mathbf{y}]_i$, $d_i$ is the $i$-th diagonal of $\mathbf{U}^H\mathbf{D}\mathbf{U}$, and $c_{0,i}' = [\mathbf{U}^H\mathbf{c}_{0}]_i$.

The MSE for mode $i$ is: $$\text{MSE}_i(\lambda) = \left|\frac{g_i + \lambda d_i}{g_i + \lambda} - 1\right|^2 \sigma_c^2 + \frac{g_i^2 \sigma^2 + \lambda^2 d_i^2 \sigma_\epsilon^2}{(g_i + \lambda)^2}.$$

Differentiating $\text{MSE}_i(\lambda)$ with respect to $\lambda$ and setting to zero gives a mode-specific optimal $\lambda_i^*$. For a single global $\lambda^*$, minimise the total MSE $\sum_i \text{MSE}_i(\lambda)$ — a scalar optimisation problem that depends on the spectrum of $\mathbf{G}$, the denoiser $\mathbf{D}$, and the noise levels $\sigma^2, \sigma_\epsilon^2$.

The result confirms that learning $\lambda_k$ is strictly better than fixing it: the optimal value depends on the spectral content of the current estimate, which changes across MoDL iterations. $\square$

For both $\mathbf{c} = (1,0)^\top$ and $\mathbf{c} = (0,1)^\top$: $y = c_1 + c_2 + n = 1 + n \sim \mathcal{N}(1, \sigma^2)$. The likelihood $p(y \mid \mathbf{c})$ is identical for both scenes. By Bayes' theorem, $p(\mathbf{c} \mid y) = p(\mathbf{c}) = 1/2$ for each.

• MSE: $\hat{\mathbf{c}} = \mathbb{E}[\mathbf{c} \mid y] = \frac{1}{2}(1,0)^\top + \frac{1}{2}(0,1)^\top = (0.5, 0.5)^\top$.
• GAN: Samples from $p(\mathbf{c} \mid y)$, outputting $(1,0)^\top$ or $(0,1)^\top$ with probability $1/2$ each.

The GAN output always places a target at exactly one pixel — but may choose the wrong pixel 50% of the time. This constitutes a false positive in target localisation — not a false detection overall, but a false position assignment that could cause a tracker to follow a hallucinated trajectory.

The MSE output $(0.5, 0.5)^\top$ is less decisive: a detection threshold of $>0.7$ would declare no target at either pixel (correct), whereas the GAN would always trigger a detection at one of two locations (50% wrong).

For radar systems operating under the Neyman-Pearson framework, the MSE estimator with threshold control is safer than the GAN estimator whose false alarm rate is uncontrolled. $\square$

For a new geometry $\boldsymbol{\alpha}^*$, find the nearest training geometry $\boldsymbol{\alpha}_{k^*} = \arg\min_k \|\mathbf{G}(\boldsymbol{\alpha}^*) - \mathbf{G}(\boldsymbol{\alpha}_k)\|$. By the triangle inequality: $$\text{Err}(\boldsymbol{\alpha}^*) \leq \text{Err}(\boldsymbol{\alpha}_{k^*}) + L_f \|\mathbf{G}(\boldsymbol{\alpha}^*) - \mathbf{G}(\boldsymbol{\alpha}_{k^*})\| \cdot \text{const}.$$

By the PAC-Bayes theorem, with probability $\geq 1 - \delta$ over $T$ training samples from geometry $\boldsymbol{\alpha}_{k^*}$: $$\mathbb{E}[\text{Err}(\boldsymbol{\alpha}_{k^*})] \leq \hat{\text{Err}}_T(\boldsymbol{\alpha}_{k^*}) + \sqrt{\frac{P\log P + \log(T/\delta)}{2T}}.$$

$\mathbb{E}[\text{Err}(\boldsymbol{\alpha}^*)] \leq \underbrace{\hat{\text{Err}}_T}_{\text{train error}} + \underbrace{\sqrt{\frac{P\log P + \log(T/\delta)}{2T}}}_{\text{complexity}} + \underbrace{L_f \cdot \delta_G}_{\text{geometry gap}},$$where$\delta_G = |\mathbf{G}(\boldsymbol{\alpha}^) - \mathbf{G}(\boldsymbol{\alpha}_{k^})|$is the Gram matrix gap to the nearest training geometry. **Implication:** Geometry generalisation requires covering the geometry space${\boldsymbol{\alpha}}$densely enough so that$\delta_G$is small, even for physics-informed networks. Domain randomisation (training on many geometries) reduces$\delta_G$at the cost of larger complexity.$\square$