Exercises

Each correspondence gives one scalar equation: $\tilde{\mathbf{x}}_2^\mathsf{T}\mathbf{F}\,\tilde{\mathbf{x}}_1 = 0$. This is linear in the 9 entries of $\mathbf{F}$.

$\mathbf{F}$ has 9 entries, but it is defined up to scale (only the direction in the 9D space matters), giving 8 DOF. The rank-2 constraint ($\det(\mathbf{F}) = 0$) reduces this to 7. The 8-point algorithm ignores the rank constraint and uses 8 correspondences to solve for $\mathbf{F}$ linearly (up to scale), then enforces rank 2 by SVD truncation.

$\mathbf{E}^\mathsf{T}\mathbf{t} = \mathbf{R}^\mathsf{T}[\mathbf{t}]_\times^\mathsf{T}\mathbf{t} = -\mathbf{R}^\mathsf{T}[\mathbf{t}]_\times\mathbf{t} = -\mathbf{R}^\mathsf{T}(\mathbf{t} \times \mathbf{t}) = \mathbf{0}$.

The epipole $\mathbf{e}_2 = \mathbf{t}$ (the projection of camera 1's centre onto image 2) lies in the left null space of $\mathbf{E}$. $\blacksquare$

Let camera parameters be $\mathbf{c} \in \mathbb{R}^{6n_c}$ and point parameters $\mathbf{p} \in \mathbb{R}^{3n_p}$. The Hessian partitions as:

$$\mathbf{H} = \begin{bmatrix} \mathbf{U} & \mathbf{W} \\ \mathbf{W}^\mathsf{T} & \mathbf{V} \end{bmatrix},$$

where $\mathbf{U} \in \mathbb{R}^{6n_c \times 6n_c}$ (camera-camera), $\mathbf{V} \in \mathbb{R}^{3n_p \times 3n_p}$ (point-point, block-diagonal with $3 \times 3$ blocks), and $\mathbf{W} \in \mathbb{R}^{6n_c \times 3n_p}$ (camera-point).

Eliminating points: $(\mathbf{U} - \mathbf{W}\mathbf{V}^{-1}\mathbf{W}^\mathsf{T})\,\delta\mathbf{c} = \mathbf{b}'$. Since $\mathbf{V}$ is block-diagonal, $\mathbf{V}^{-1}$ costs $O(n_p)$ (invert each $3 \times 3$ block). The reduced system is $6n_c \times 6n_c$, which for $n_c = 100$ cameras is $600 \times 600$ — easily solvable. Eliminating cameras first would give a $3n_p \times 3n_p$ dense system, far larger.

For a directional light: $L_i(\mathbf{p}, \boldsymbol{\omega}_i) = E_L\,\delta(\boldsymbol{\omega}_i - \boldsymbol{\omega}_L)$. Substituting into the rendering equation:

$$L_o(\mathbf{p}, \boldsymbol{\omega}_o) = \frac{\rho}{\pi}\,E_L\,\max(\boldsymbol{\omega}_L \cdot \mathbf{n}, 0).$$

This is the classic Lambert's cosine law: the reflected radiance is proportional to the cosine of the incidence angle, independent of the viewing direction $\boldsymbol{\omega}_o$.

$\int_{t_i}^{t_{i+1}} T(t)\,\sigma_i\,\mathbf{c}_i\,dt = T_i\,\mathbf{c}_i\,\sigma_i \int_0^{\delta_i} e^{-\sigma_i\tau}\,d\tau = T_i\,\mathbf{c}_i\,(1 - e^{-\sigma_i\delta_i}).$$

$T_{i+1} = T_i\,e^{-\sigma_i\delta_i}$, with $T_1 = 1$. Therefore $T_i = \exp(-\sum_{j=1}^{i-1}\sigma_j\delta_j)$.

$\hat{C} = \sum_{i=1}^{N} T_i\,(1 - e^{-\sigma_i\delta_i})\,\mathbf{c}_i,$$which is the standard NeRF discrete rendering formula. Defining$\alpha_i = 1 - e^{-\sigma_i\delta_i}$, this becomes the familiar alpha-compositing:$\hat{C} = \sum_i T_i,\alpha_i,\mathbf{c}_i$.$\blacksquare$

For Tx $i$, Rx $j$, frequency $f_k$: $y_{i,j,k} = \sum_{q=1}^{Q} [\mathbf{A}]_{(i,j,k),q}\,c_q + w_{i,j,k}$, where $[\mathbf{A}]_{(i,j,k),q} = \kappa_{k}^{2}\,G(\mathbf{r}_{j}, \mathbf{p}_{q})\,G(\mathbf{p}_{q}, \mathbf{s}_{i})\,\Delta V$.

Stacking all measurements: $\mathbf{y} = \mathbf{A}\mathbf{c} + \mathbf{w}$, where $\mathbf{A} \in \mathbb{C}^{M \times Q}$ with $M = N_t \times N_r \times N_f$ rows and $Q$ columns. This is a linear system in $\mathbf{c}$. $\blacksquare$

$\frac{\partial \mathcal{L}}{\partial \theta_m} = \frac{\partial \mathcal{L}}{\partial \mathbf{I}}\frac{\partial \mathbf{I}}{\partial \theta_m}$ for each component $\theta_m$ of $\theta$.

From $\mathbf{Z}\mathbf{I} = \mathbf{V}$: $\mathbf{Z}\frac{\partial \mathbf{I}}{\partial \theta_m} = -\frac{\partial \mathbf{Z}}{\partial \theta_m}\mathbf{I}$, so $\frac{\partial \mathbf{I}}{\partial \theta_m} = -\mathbf{Z}^{-1}\frac{\partial \mathbf{Z}}{\partial \theta_m}\mathbf{I}$.

Substituting: $\frac{\partial \mathcal{L}}{\partial \theta_m} = -\frac{\partial \mathcal{L}}{\partial \mathbf{I}}\mathbf{Z}^{-1}\frac{\partial \mathbf{Z}}{\partial \theta_m}\mathbf{I} = -\boldsymbol{\lambda}^\mathsf{T}\frac{\partial \mathbf{Z}}{\partial \theta_m}\mathbf{I}$,

where $\boldsymbol{\lambda}$ solves the single adjoint system $\mathbf{Z}^\mathsf{T}\boldsymbol{\lambda} = (\partial \mathcal{L}/\partial \mathbf{I})^\mathsf{T}$. This solve is independent of $m$, so all $|\theta|$ gradient components cost only one adjoint solve. $\blacksquare$

1. Wavelength vs. feature size: Optical $\lambda \sim 500$ nm $\ll$ scene features; RF $\lambda \sim 1$ mm to 10 cm $\sim$ features. Ray optics valid in optical, not in RF.

2. Coherence: Optical rendering sums intensities ($|E_1|^2 + |E_2|^2$); RF must sum complex fields ($|E_1 + E_2|^2$), producing interference.

3. BRDF vs. scattering cross-section: The BRDF assumes smooth surfaces at the wavelength scale; at RF wavelengths, surfaces have sub-wavelength structure requiring wave-theoretic scattering models.

$\Delta x_{\text{radar}} = R\,\Delta\phi = 50 \times 5\pi/180 \approx 4.36$ m.

$\Delta x_{\text{camera}} = R \times 0.05\pi/180 \approx 0.044$ m $= 4.4$ cm.

The camera has $\sim 100\times$ better angular resolution than the radar. This demonstrates the complementarity: radar provides range and velocity, camera provides angular detail.

$\log f(y_1, y_2 | \theta) = \log f(y_1 | \theta) + \log f(y_2 | \theta)$.

$\mathbf{J}(\Theta; Y_1, Y_2) = -\mathbb{E}\left[\nabla^2_\theta \log f(Y_1, Y_2 | \theta)\right] = -\mathbb{E}\left[\nabla^2_\theta \log f(Y_1 | \theta)\right] - \mathbb{E}\left[\nabla^2_\theta \log f(Y_2 | \theta)\right]$.

$= \mathbf{J}(\Theta; Y_1) + \mathbf{J}(\Theta; Y_2)$.

The independence ensures no cross-terms appear. $\blacksquare$

The 3D point is $\mathbf{P} = d\,\mathbf{K}^{-1}[u, v, 1]^\mathsf{T}$. The BEV coordinates are $(X, Y)$ from $\mathbf{P}$.

$\Delta X = \Delta d \cdot (u - c_x)/f$, $\Delta Y = \Delta d$. The along-range error equals $\Delta d$ directly; the cross-range error scales with the pixel's off-axis angle.

For $d = 30$ m, $\Delta d = 1$ m, $u - c_x = 400$ pixels, $f = 800$ pixels: $\Delta X = 1 \times 400/800 = 0.5$ m, $\Delta Y = 1$ m. This motivates fusing with radar, which provides accurate range directly.

At collocation point $\mathbf{r}_j = (x_j, y_j)$:

$$R_j = \frac{\partial^2 u_\theta}{\partial x^2}\bigg|_{\mathbf{r}_j} + \frac{\partial^2 u_\theta}{\partial y^2}\bigg|_{\mathbf{r}_j} + \kappa^{2}\,u_\theta(\mathbf{r}_j) + s(\mathbf{r}_j).$$

1. First call: grad_x = autograd(u, x) gives $\partial u/\partial x$.
2. Second call: grad_xx = autograd(grad_x, x) gives $\partial^2 u/\partial x^2$.
3. Similarly for $\partial^2 u/\partial y^2$.
4. Each call costs $O(P)$ (one backward pass per scalar output).
5. Total cost: $4 \times O(P)$ per collocation point.

1. FFT: $d_v$ channels $\times$ $O(N^2 \log N)$ = $O(d_v N^2 \log N)$.
2. Filter: $k_{\max}^2$ modes $\times$ $d_v \times d_v$ matrix multiply = $O(k_{\max}^2 d_v^2)$.
3. IFFT: $O(d_v N^2 \log N)$.
4. Local linear: $O(d_v^2 N^2)$.
5. Total: $O(d_v N^2 \log N + k_{\max}^2 d_v^2 + d_v^2 N^2)$.

A $3 \times 3$ convolution: $O(9\,d_v^2\,N^2)$. For $k_{\max} = 16$, $d_v = 32$, $N = 128$: FNO $\approx 2.4$M FLOPs (FFT + filter), CNN $\approx 150$M FLOPs. The FNO is more efficient because it captures long-range interactions via the Fourier domain without large kernel sizes.

$K_m(R_\alpha\mathbf{r}) = R_m(r)\,e^{jm(\phi+\alpha)} = e^{jm\alpha}\,K_m(\mathbf{r})$.

Let $f_\alpha(\mathbf{r}) = f(R_{-\alpha}\mathbf{r})$ be the rotated input. Then:

$$(K_m * f_\alpha)(\mathbf{r}) = \int K_m(\mathbf{r} - \mathbf{r}')\,f(R_{-\alpha}\mathbf{r}')\,d\mathbf{r}'.$$

Substituting $\mathbf{r}'' = R_{-\alpha}\mathbf{r}'$:

$$= \int K_m(\mathbf{r} - R_\alpha\mathbf{r}'')\,f(\mathbf{r}'')\,d\mathbf{r}'' = e^{jm\alpha}\int K_m(R_{-\alpha}\mathbf{r} - \mathbf{r}'')\,f(\mathbf{r}'')\,d\mathbf{r}''.$$

$= e^{jm\alpha}\,(K_m * f)(R_{-\alpha}\mathbf{r}).$$The output is the original convolution evaluated at the rotated coordinate, times$e^{jm\alpha}$. This is exact$SO(2)$equivariance: rotation of the input rotates the output and multiplies the$m$-th harmonic coefficient by$e^{jm\alpha}$.$\blacksquare$

The FNO filter $\mathbf{R}_\theta$ acts on the first $k_{\max} = 12$ Fourier modes. On a $64 \times 64$ grid, the FFT has modes $k = 0, \ldots, 31$. On a $128 \times 128$ grid, modes $k = 0, \ldots, 63$. The FNO applies $\mathbf{R}_\theta$ to modes $k = 0, \ldots, 11$ (same weights) and zeros modes $k \geq 12$.

The transfer is accurate when:

1. The operator $\mathcal{G}^\dagger$ is well-approximated by its first $k_{\max}$ Fourier modes (smooth operator).
2. The input $\epsilon_r$ on the finer grid does not introduce significant energy in modes $k > 12$ (smooth inputs).
3. The local linear layer $\mathbf{W}v(\mathbf{r})$ generalises (it acts pointwise, so resolution-independent by construction).

For inputs with sharp features (edges, point scatterers) at scales below $\lambda_{\min} \sim 1/k_{\max}$, the higher modes carry significant energy. The FNO misses these, producing smoothed outputs. Remedy: increase $k_{\max}$ or use a hybrid FNO + physics-based correction.

Data loss: $\mathcal{L}_d \sim (1/400)\sum|E_\theta - E^{\text{meas}}|^2 \sim |E|^2$. PDE loss: $\mathcal{L}_p \sim (\lambda/5000)\sum|\nabla^2 u + \kappa^{2}\epsilon u|^2 \sim \lambda\,\kappa^{4}\,|E|^2$.

For $\mathcal{L}_d \sim \mathcal{L}_p$: $\lambda \sim 1/\kappa^{4} \sim (\lambda/(2\pi))^4$. At 3 GHz ($\lambda = 0.1$ m): $\lambda \sim (0.016)^4 \approx 6.6 \times 10^{-8}$.

Start with $\lambda = 0$ (pure data fitting) and increase to the target value over the first 30% of training. This curriculum avoids the PDE constraint dominating early training when the network outputs are far from the solution.

1. Independent encoders: $\mathbf{z}_r = E_r(\mathbf{y}_{\text{radar}})$, $\mathbf{z}_c = E_c(\mathbf{y}_{\text{camera}})$, $\mathbf{z}_l = E_l(\mathbf{y}_{\text{LiDAR}})$.
2. Masked attention fusion: $\mathbf{z} = \text{Attention}(\mathbf{z}_r, \mathbf{z}_c, \mathbf{z}_l; \mathbf{m})$ where $\mathbf{m} \in \{0,1\}^3$ indicates available modalities.
3. Detection head operates on $\mathbf{z}$.

During training, randomly mask each modality with probability $p = 0.2$. This forces the network to learn to exploit whatever modalities are available.

Under optimal Bayesian fusion (which attention approximates): $I(\Theta; \mathbf{z}_r, \mathbf{z}_c, \mathbf{z}_l) \geq I(\Theta; \mathbf{z}_r, \mathbf{z}_c) \geq I(\Theta; \mathbf{z}_r)$ by the chain rule and non-negativity of mutual information. The learned fusion approximates this optimal combination, and modality dropout during training ensures the approximation is tight for all subsets.

For a ReLU MLP, the NTK eigenvalues $\lambda_k$ scale as $\lambda_k \sim k^{-(2L+1)}$ for network depth $L$ and Fourier mode $k$. For a 4-layer MLP: $\lambda_k \sim k^{-9}$.

Under gradient descent with learning rate $\eta$, the error in mode $k$ decays as $\epsilon_k(t) \sim e^{-\eta\lambda_k t}$. For mode $k = 10$ relative to mode $k = 1$: $\lambda_{10}/\lambda_1 \sim 10^{-9}$. Mode 10 converges $10^9$ times slower than mode 1.

Fourier feature encoding modifies the NTK to have $\lambda_k \sim \text{const}$ for $k \leq k_{\max}$ (controlled by the encoding bandwidth $\sigma$), eliminating the spectral bias for the targeted frequency range.

$\mathbf{y} = \mathbf{A}(\{\mathbf{s}_{i}\}, \{\mathbf{r}_{j}\})\,\mathbf{c} + \mathbf{w}$, where $[\mathbf{A}]_{(i,j,k),q} = \kappa_{k}^{2}\,G(\mathbf{r}_{j}, \mathbf{p}_{q})\,G(\mathbf{p}_{q}, \mathbf{s}_{i})\,\Delta V$.

$\min_{\mathbf{c}, \{\mathbf{s}_{i}\}, \{\mathbf{r}_{j}\}} \|\mathbf{y} - \mathbf{A}\mathbf{c}\|^2 + \lambda_c\|\mathbf{c}\|_1 + \lambda_p\,\mathcal{R}_{\text{pos}},$$where$\mathcal{R}_{\text{pos}}$ regularises positions (e.g., proximity to nominal values from GPS/blueprint).

1. Fix positions, solve for $\mathbf{c}$ (LASSO/ADMM).
2. Fix $\mathbf{c}$, update positions via gradient descent (differentiating $\mathbf{A}$ w.r.t. $\mathbf{s}_{i}, \mathbf{r}_{j}$).
3. Iterate.

This is the RF analog of bundle adjustment: jointly estimating the "scene" (reflectivity) and "camera poses" (sensor positions) from measurements.

For $a \in H^s([0,1])$, the truncation to modes $|k| \leq k_{\max}$ satisfies $\|a - a_{k_{\max}}\|_{L^2} \leq C\,k_{\max}^{-s}\,\|a\|_{H^s}$.

The mapping $\hat{a} = (\hat{a}_0, \ldots, \hat{a}_{k_{\max}}) \mapsto (\hat{u}_0, \ldots, \hat{u}_{k_{\max}})$ is a continuous function $\mathbb{R}^{2k_{\max}+1} \to \mathbb{R}^{2k_{\max}+1}$. By the universal approximation theorem, there exists a neural network $\Phi_\theta$ with $\|\Phi_\theta(\hat{a}) - \hat{u}^{\text{true}}\| < \epsilon'$.

$\|\mathcal{G}_\theta(a) - \mathcal{G}^\dagger(a)\|_{L^2} \leq \underbrace{\|P_{k_{\max}}\mathcal{G}^\dagger(a) - \mathcal{G}^\dagger(a)\|}_{O(k_{\max}^{-s})} + \underbrace{\|\Phi_\theta(\hat{a}) - P_{k_{\max}}\mathcal{G}^\dagger(a)\|}_{< \epsilon'}.$$Choosing$k_{\max}$large enough and$\Phi_\theta$sufficiently expressive, the total error is$< \epsilon$.$\blacksquare$