Chapter Summary

Chapter 28 Summary: Multi-View Geometry and Differentiable Rendering

Key Points

• 1.

  Multi-view geometry provides the mathematical foundation for 3D reconstruction from multiple observations. The epipolar constraint $\tilde{\mathbf{x}}_2^\mathsf{T}\mathbf{F}\,\tilde{\mathbf{x}}_1 = 0$ reduces stereo correspondence from a 2D search to a 1D search, and bundle adjustment jointly optimises scene structure and camera poses. These concepts transfer to RF: Tx-Rx pairs are "cameras," range measurements replace pixel disparities, and autofocus is the RF analog of bundle adjustment.

• 2.

  Differentiable rendering enables gradient-based inverse rendering: recovering geometry, materials, and lighting from images. The rendering equation (Kajiya, 1986) is the optical forward model; volume rendering (NeRF), soft rasterisation (SoftRas), and Gaussian splatting (3DGS) provide differentiable approximations that handle visibility discontinuities smoothly.

• 3.

  RF rendering differs fundamentally from optical rendering: wavelength-scale features cause diffraction and interference, requiring wave-based (not ray-based) transport models. The Born approximation makes the RF forward model linear in scene parameters — a simplification with no optical analog. Differentiable RF renderers (adjoint MoM, Sionna RT, neural fields + physics) enable the same analysis-through-synthesis approach.

• 4.

  Multi-modal fusion combines radar (velocity, all-weather), camera (resolution, semantics), and LiDAR (3D geometry) for robust perception. Mid-level fusion with BEV representations is the current best practice. The CommIT group's per-sensor back-projection + learned fusion is well-suited to distributed RF imaging where phase incoherence makes joint processing fragile.

• 5.

  Physics-informed networks embed governing equations into neural architectures. PINNs enforce PDE constraints via autodiff but suffer from spectral bias. Equivariant networks build symmetries (rotation, translation) into the architecture. The Fourier Neural Operator learns resolution-independent PDE solution operators in $O(N \log N)$, enabling real-time forward modelling for iterative RF imaging reconstruction.