Exercises

$\log\pi(\theta) = -\theta^2/(2\sigma_0^2) + \text{const}$, so $\partial_\theta\log\pi = -\theta/\sigma_0^2$ and $I_P = \mathbb{E}_\pi[\theta^2]/\sigma_0^4 = 1/\sigma_0^2$. The prior vanishes smoothly at infinity, so Van Trees applies directly.

$\partial_\theta\log\pi = -\mathrm{sign}(\theta)/b$, so $(\partial_\theta\log\pi)^2 = 1/b^2$ and $I_P = 1/b^2$. The density has a kink at $0$ but is continuous and vanishes at $\pm\infty$, so the integration-by-parts boundary condition holds.

$\partial_\theta\log\pi = -2\theta/(1+\theta^2)$, so $$I_P = \int_{-\infty}^{\infty}\frac{4\theta^2}{(1+\theta^2)^2}\cdot \frac{1}{\pi(1+\theta^2)}\,d\theta = \frac{1}{2}.$$ The Cauchy vanishes at $\pm\infty$ (polynomially), so Van Trees still applies. Note that $\mathbb{E}_\pi[\theta^2] = \infty$ yet $I_P$ is finite — the prior information does not require finite prior variance.

$I_B = n/\sigma_w^2 + 1/\sigma_0^2$. Factor out $1/\sigma_w^2$: $$I_B = \frac{1}{\sigma_w^2}\left(n + \frac{\sigma_w^2}{\sigma_0^2}\right) \;=\; \frac{n_{\text{eff}}}{\sigma_w^2}, \qquad n_{\text{eff}} = n + \frac{\sigma_w^2}{\sigma_0^2}.$$

The additive constant $\sigma_w^2/\sigma_0^2$ is the ratio of the noise variance to the prior variance. When the prior is as sharp as a single noisy measurement ($\sigma_0^2 = \sigma_w^2$), it counts like one extra observation. When the prior is $k$ times sharper, it counts like $k$ extra observations. This is exactly the sense in which "a tight prior is worth $k$ samples."

The Q-function $Q(x) = \Pr(Z \geq x)$ for $Z\sim\mathcal{N}(0,1)$ is strictly decreasing in $x$. Since $h\sqrt{\gamma}$ is increasing in $h$, $P_{\min}(h)$ is strictly decreasing in $h$. Therefore $\mathcal{V}\{P_{\min}\} = P_{\min}$.

$\text{ZZB} \;=\; \int_0^L \frac{h(L-h)}{L}\,Q\!\left(h\sqrt{\gamma}\right)\,dh.$$ This matches the Bellini form for a Gaussian-shift problem with no autocorrelation sidelobes — which is expected, since a pure translation model has no ambiguity function structure other than its main lobe.

For Gaussian input on a Gaussian channel, $I(X;Y) = \tfrac{1}{2}\log(1+\gamma)$ nats.

The MMSE estimator is $\hat X = \sqrt{\gamma}Y/(1+\gamma)$ with error variance $\text{mmse}(\gamma) = 1/(1+\gamma)$.

$\dfrac{d}{d\gamma}\tfrac{1}{2}\log(1+\gamma) = \tfrac{1}{2}\cdot \tfrac{1}{1+\gamma} = \tfrac{1}{2}\text{mmse}(\gamma)$. The identity holds exactly.

$\dot{\mathbf{a}}(\theta) = j\pi\cos\theta\cdot\mathrm{diag}(0,1,\ldots,N_t-1)\,\mathbf{a}(\theta)$.

$\|\dot{\mathbf{a}}\|^2 = (\pi\cos\theta)^2 \sum_{k=0}^{N_t-1}k^2 = (\pi\cos\theta)^2\cdot N_t(N_t-1)(2N_t-1)/6 \sim (\pi\cos\theta)^2 N_t^3/3$ for large $N_t$.

With $\mathbf{R}_s = \mathbf{I}/N_t$, $\dot{\mathbf{a}}^H\mathbf{R}_s \dot{\mathbf{a}} = \|\dot{\mathbf{a}}\|^2/N_t \sim (\pi\cos\theta)^2 N_t^2/3$, and $|\mathbf{a}^H\mathbf{a}| = N_t$. Plugging in: $$\mathrm{CRB}(\theta) \propto \frac{\sigma^2}{|\alpha|^2\,T\, (\pi\cos\theta)^2 N_t^2/3\cdot N_t} \propto \frac{1}{N_t^3}.$$ The cubic gain is the classical super-resolution scaling of ULAs.

For $|\theta| > a$, $\partial_\theta\log\pi_\epsilon = -2\theta/\epsilon^2$. For $|\theta| \leq a$, the score is zero. So $I_P = \int_{|\theta|>a} \dfrac{4\theta^2}{\epsilon^4}\,\pi_\epsilon(\theta)\,d\theta$.

Substitute $\theta = a + \epsilon u/\sqrt{2a}$ in the right tail. The tail probability mass is $O(\epsilon)$ (boundary-layer width), and within that layer $\theta^2/\epsilon^4 \sim a^2/\epsilon^4$, so $I_P \sim C\,a^2\cdot \epsilon/\epsilon^4 = C\,a^2/\epsilon^3 \to \infty$ as $\epsilon\to 0$.

The prior information diverges as the prior becomes sharper at the boundary, which drives $1/I_B$ to zero and the Van Trees bound collapses. This is the mathematical signature of an ill-posed boundary condition: the smooth Van-Trees bound is not the right tool for hard-support priors, and one must instead use the Gill-Levit constrained version or the Ziv-Zakai bound, which handle bounded supports natively.

Let $p(\boldsymbol\theta,\mathbf{y}) = \pi(\boldsymbol\theta)\, p(\mathbf{y}|\boldsymbol\theta)$. The joint score is $\boldsymbol\psi = \nabla\log\pi + \nabla\log p(\mathbf{y}|\boldsymbol\theta)$. The two terms are uncorrelated (conditional score has zero mean given $\boldsymbol\theta$), so $\mathbb{E}[\boldsymbol\psi\boldsymbol\psi^T] = \mathbf{I}_P + \mathbb{E}_\pi[\mathbf{I}_F(\boldsymbol\theta)] = \mathbf{I}_B$.

For each pair of coordinates $(i,j)$, $\mathbb{E}[(\hat\theta_i - \theta_i)\psi_j] = -\mathbb{E}[\partial_{\theta_j}(\hat\theta_i - \theta_i)] = \delta_{ij}$, since $\hat\theta_i$ does not depend on $\theta_j$. The boundary term from integration by parts vanishes by the assumed decay of $\pi$. Stacking: $\mathbb{E}[(\hat{\boldsymbol\theta}-\boldsymbol\theta)\boldsymbol\psi^T] = \mathbf{I}$.

With $\mathbf{u} = \hat{\boldsymbol\theta}-\boldsymbol\theta$ and $\mathbf{v} = \boldsymbol\psi$, the matrix Cauchy-Schwarz inequality gives $$\mathbb{E}[\mathbf{u}\mathbf{u}^T] \succeq \mathbb{E}[\mathbf{u}\mathbf{v}^T]\,(\mathbb{E}[\mathbf{v}\mathbf{v}^T])^{-1}\,\mathbb{E}[\mathbf{v}\mathbf{u}^T] = \mathbf{I}\cdot\mathbf{I}_B^{-1}\cdot\mathbf{I} = \mathbf{I}_B^{-1}.$$ Equality holds iff $\mathbf{u}$ and $\mathbf{v}$ are (matrix-) proportional a.s.

At high SNR, $Q(x(h))$ is supported on $h\sim 1/(B_{\text{rms}}\sqrt{E_s/N_0}) \ll L$, so $(L-h)/L\approx 1$ and we can extend the upper limit to $\infty$: $\text{ZZB}\approx \int_0^\infty h\,Q(2\pi B_{\text{rms}} h\sqrt{E_s/(2N_0)})\,dh$.

Let $u = 2\pi B_{\text{rms}} h\sqrt{E_s/(2N_0)}$, so $h = u/(2\pi B_{\text{rms}})\cdot\sqrt{2N_0/E_s}$ and $dh = du/(2\pi B_{\text{rms}})\cdot\sqrt{2N_0/E_s}$. Hence $$\text{ZZB}\approx \frac{2N_0}{(2\pi B_{\text{rms}})^2 E_s} \int_0^\infty u\,Q(u)\,du.$$

Integration by parts gives $\int_0^\infty u\,Q(u)\,du = 1/4$. Therefore $$\text{ZZB}\approx \frac{2N_0}{(2\pi B_{\text{rms}})^2 E_s}\cdot\frac{1}{4} = \frac{N_0}{8\pi^2 B_{\text{rms}}^2 E_s},$$ which is exactly the CRLB for TOA estimation. The ZZB reduces to the CRLB in the high-SNR regime, as required.

$\text{mmse}(0) = \mathrm{Var}(X) = 1$. Near $\gamma=0$, expansion gives $\text{mmse}(\gamma)\approx 1 - \gamma + O(\gamma^2)$.

As $\gamma\to\infty$, $\tanh(\sqrt{\gamma}Y)\to\mathrm{sign}(Y)=X$ almost surely, so $\text{mmse}(\gamma)\to 0$ exponentially fast. Specifically $\text{mmse}(\gamma) \approx 4 e^{-\gamma}\cdot(1+o(1))$ as $\gamma\to\infty$ by the tail of $\tanh$.

By the I-MMSE identity, $I_{\text{BPSK}}(\gamma) = \tfrac{1}{2}\int_0^\gamma \text{mmse}(s)\,ds$. Numerically integrating the sigmoidal MMSE curve produces the BPSK capacity, saturating at $\ln 2$ nats. The sharp sigmoid transition in $\text{mmse}$ at $\gamma\sim 1$ corresponds to the steep rise of $I$ from zero to its saturation.

$\text{mmse}(0) = \mathrm{Var}(X) = \mathbb{E}[X^2] - \mathbb{E}[X]^2 = 1 - 0 = 1$.

When $\gamma$ is small, the posterior $P(B=1|Y)$ is flat across noise realisations and the estimator essentially guesses $\hat X \approx 0$, giving MMSE near $\mathrm{Var}(X) = 1$. Only when $\gamma\gtrsim 1/p$ does the likelihood ratio for $B$ exceed the noise floor — at that point, $B$ becomes reliably detectable and the posterior concentrates on the correct support. The MMSE then drops rapidly as the conditional Gaussian estimate of $G$ takes over. The plateau width scales like $1/p$ and the drop steepness like $p$, giving the characteristic L-shape of sparse inputs.

This L-shape is the single-letter version of the phase transition observed in compressed-sensing reconstruction: at measurement density below a critical threshold, the MSE is stuck at the prior variance; above threshold, it drops to a denoising-limited floor. The I-MMSE integral converts this shape into a rate-versus-SNR curve with a matching phase transition.

Maximise $J(\mathbf{R}_s) = (1-\lambda)\log(1+\mathbf{h}^H\mathbf{R}_s\mathbf{h}/\sigma_c^2) + \lambda\cdot\dot{\mathbf{a}}^H\mathbf{R}_s\dot{\mathbf{a}}$ over $\mathbf{R}_s\succeq\mathbf{0},\;\mathrm{tr}(\mathbf{R}_s)\leq P$.

At $\lambda=0$, $J$ depends only on $\mathbf{h}^H\mathbf{R}_s\mathbf{h}$. By the rank-1 optimality of power-constrained quadratic forms, $\mathbf{R}_s^\star = P\,\mathbf{h}\mathbf{h}^H/\|\mathbf{h}\|^2$. The transmitter puts all its power in the user's direction.

At $\lambda=1$, only $\dot{\mathbf{a}}^H\mathbf{R}_s\dot{\mathbf{a}}$ matters. Again rank-1 is optimal: $\mathbf{R}_s^\star = P\,\dot{\mathbf{a}}(\theta_0)\dot{\mathbf{a}}^H(\theta_0)/\|\dot{\mathbf{a}}\|^2$. Power is steered along the angle-gradient direction, which is orthogonal to the main-lobe direction $\mathbf{a}(\theta_0)$ and maximises the slope of the beampattern (optimal for angular discrimination).

For $\lambda\in(0,1)$, the optimal $\mathbf{R}_s^\star$ has rank up to $2$ (a mixture of the two rank-1 solutions). Scaling $\lambda$ from $0$ to $1$ traces out the Pareto boundary, morphing smoothly from the communication beam toward the sensing beam.

The linear MMSE estimator achieves error variance $\sigma_X^2/(1+\gamma\sigma_X^2)$ for any input with variance $\sigma_X^2$. The posterior-mean (MMSE) estimator is optimal, so its error variance is at most the LMMSE error variance: $\text{mmse}(\gamma) \leq \sigma_X^2/(1+\gamma\sigma_X^2)$.

$I(X;Y) = \tfrac{1}{2}\int_0^\gamma \text{mmse}(s)\,ds \leq \tfrac{1}{2}\int_0^\gamma \frac{\sigma_X^2}{1+s\sigma_X^2}\,ds = \tfrac{1}{2}\log(1+\gamma\sigma_X^2)$.

Equality holds iff $\text{mmse}(s) = \sigma_X^2/(1+s\sigma_X^2)$ for all $s\in[0,\gamma]$, which by the uniqueness of the LMMSE implies $X$ is jointly Gaussian with $Y$, hence $X$ itself is Gaussian. This is the I-MMSE proof of the entropy-power inequality statement that Gaussian inputs maximise mutual information at a power constraint — a slick alternative to the standard Shannon-Gelfand-Yaglom derivation.

No bandwidth means zero delay information: the delay diagonal of the FIM is zero (infinite delay CRB) and delay-angle / delay-Doppler cross-terms vanish. Angle and Doppler precision remain fine. This is the classic "Doppler-only" radar mode.

Spread bandwidth gives good delay precision; random symbols decorrelate the waveform across subcarriers, killing cross-terms (the FIM becomes nearly diagonal). This is the ideal for joint estimation: each parameter is decoupled from the others and the CRBs on all three are small.

Good delay and Doppler precision (wideband, long-duration), but the linear time-frequency coupling produces a strong delay-Doppler cross-term: a shift in $\tau$ produces the same first-order change in the template as a shift in $\nu$, so the two are ambiguous. Marginal CRBs look fine; joint CRBs (on linear combinations of $\tau$ and $\nu$) are excellent only along one diagonal of the $(\tau,\nu)$ plane and poor along the other. This is the classic LFM range-Doppler coupling.

For a uniform $\theta$ and observation $Y = \cos(\theta)+W$, the expected binary error between $\theta$ and $\theta+h$ under the Bayes detector is $P_{\min}(h) = \mathbb{E}_\theta\bigl[Q(|\cos(\theta)-\cos(\theta+h)|/(2\sigma))\bigr]$. After averaging over uniform $\theta\in[0,2\pi]$, this is a deterministic function of $h$.

With the uniform-prior form, $\text{ZZB} = \int_0^\pi h\cdot\frac{\pi-h}{\pi}\cdot\mathcal{V}\{P_{\min}(h)\}\,dh$ (using the half-interval because of $\cos$-symmetry). Valley-filling makes the integrand non-increasing; at small $h$ where $\cos$ is flat (near $h=0,\pi$) the raw $P_{\min}$ is near $1/2$, and valley-filling propagates this upward.

Because $P_{\min}(h)\leq 1/2$ for all $h$, the ZZB is bounded by $\int_0^\pi h(\pi-h)/(2\pi)\,dh = \pi^2/12\cdot 2 = \pi^2/6$. The CRLB singularity at $\theta_0=0,\pi$ reflects that a pointwise estimator is hopeless near these angles (a tiny jitter wraps around), but the averaged ZZB is bounded because the prior averages over the bad regions. This is precisely why ZZB is the standard bound for circular / phase / angle estimation with singular CRLBs.

Any estimator of a circular parameter that quotes only the CRLB is misleading at the singular points. Radar and positioning engineers dealing with angle-of-arrival near endfire ($\theta\to\pm 90^\circ$) always use ZZB for the honest accuracy bound.

By I-MMSE, $\text{mmse}(\gamma) = 2\,dI/d\gamma$. The mutual information $I(X;Y_\gamma)$ is non-decreasing in $\gamma$ (more SNR cannot remove information), and strictly increasing whenever $X$ is non-degenerate. Hence $dI/d\gamma > 0$ and $\text{mmse}(\gamma) > 0$ for all $\gamma$. This is consistent but does not directly show monotonicity of $\text{mmse}$.

Guo, Shamai and Verdu (2005, Theorem 2) prove the higher-order identity $$\frac{d}{d\gamma}\text{mmse}(\gamma) = -2\,\mathbb{E}_Y\bigl[\mathrm{Var}(X|Y)^2\bigr].$$ The right-hand side is non-positive, and zero iff $\mathrm{Var}(X|Y)=0$ almost surely, which requires $X$ to be a deterministic function of $Y$ — impossible for non-degenerate $X$ and finite $\gamma$. Hence $d\text{mmse}/d\gamma < 0$: $\text{mmse}$ is strictly decreasing.

A similar computation (Guo-Shamai-Verdu, Theorem 3) gives $$\frac{d^2}{d\gamma^2}\text{mmse}(\gamma) = 4\,\mathbb{E}_Y\bigl[\mathrm{Var}(X|Y)^3 + 3\,\mathrm{Var}(X|Y)\cdot(\mathrm{third\ moment\ term})\bigr]\geq 0,$$ with the bracket non-negative. Therefore $\text{mmse}(\gamma)$ is convex in $\gamma$.

$\text{mmse}(\gamma)$ is the slope of $2I(X;Y_\gamma)$. Convexity of $\text{mmse}$ says that $I$ is concave and strictly so — consistent with the well-known concavity of mutual information in SNR for any fixed input distribution. I-MMSE gives a clean, estimation-theoretic proof of a long-standing information-theoretic property.

With a separable prior on $(\tau,\theta)$ approximated by independent Gaussians of variance $\Delta_\tau^2,\Delta_\theta^2$, $\mathbf{I}_P = \mathrm{diag}(1/\Delta_\tau^2, 1/\Delta_\theta^2)$. The data-averaged Fisher information is $\mathbf{I}_F = \mathrm{diag}(I_\tau, I_\theta)$ (assuming negligible cross-terms for a wideband uncorrelated waveform). Thus $$\mathbf{I}_B = \mathrm{diag}(I_\tau + 1/\Delta_\tau^2, I_\theta + 1/\Delta_\theta^2).$$

$\mathrm{tr}(\mathbf{I}_B^{-1}) = \frac{1}{I_\tau + 1/\Delta_\tau^2} + \frac{1}{I_\theta + 1/\Delta_\theta^2}.$$This is always below the classical CRLB trace$\mathrm{tr}(\mathbf{I}F^{-1}) = 1/I\tau + 1/I_\theta$, with the gap shrinking as priors widen.

The prior dominates on coordinate $i$ when $1/\Delta_i^2 \gtrsim I_{F,i}$, i.e., when the prior precision exceeds the data Fisher information. For a tight delay prior of half-width $\Delta_\tau = 1/(10 B_{\text{rms}})$ (say, a previous ranging fix accurate to one tenth of the pulse width), the delay Van-Trees bound is dominated by the prior whenever $I_\tau \lesssim 100 B_{\text{rms}}^2$, i.e., at low SNR. This is the coasting regime in which the tracker is "believing the prior more than the data."

For 5G NR positioning with large bandwidth and small array, the delay dimension enters the prior-dominated regime long before the angle dimension. This explains why network-based tracking (with history-informed priors) beats single-shot CRLB predictions by large factors at low SINR.

Preserving rate requires $\mathbf{h}^H\mathbf{R}_s\mathbf{h}/\|\mathbf{h}\|^2 = P$. Power constraint requires $\mathrm{tr}(\mathbf{R}_s) = P$. Cauchy-Schwarz gives $\mathbf{h}^H\mathbf{R}_s\mathbf{h}\leq \|\mathbf{h}\|^2\mathrm{tr}(\mathbf{R}_s)$, with equality iff $\mathbf{R}_s$ is rank-1 aligned with $\mathbf{h}$. So achieving $R_{\max}$ forces rank-1 along $\mathbf{h}$ — one cannot get extra sensing for free.

One must accept a rate loss. Let $\mathbf{R}_s = \beta P \mathbf{h}\mathbf{h}^H/\|\mathbf{h}\|^2 + (1-\beta) P \dot{\mathbf{a}}\dot{\mathbf{a}}^H/\|\dot{\mathbf{a}}\|^2$ with $\beta\in[0,1]$. Then (assuming orthogonality) $R(\beta) = \log(1 + \beta P\|\mathbf{h}\|^2/\sigma_c^2) < R_{\max}$ and angular Fisher is proportional to $(1-\beta) P\|\dot{\mathbf{a}}\|^2$.

A fractional rate loss of $\Delta R = \log(1+P\|\mathbf{h}\|^2/\sigma_c^2) - \log(1+\beta P\|\mathbf{h}\|^2/\sigma_c^2)$ buys angular Fisher proportional to $(1-\beta)$. For small sensing allocations ($\beta$ near 1), $\Delta R \approx (1-\beta)\cdot \rho/(1+\rho)\log e$ with $\rho = P\|\mathbf{h}\|^2/\sigma_c^2$. At high SNR, even a small $(1-\beta)$ buys significant sensing while costing little rate — the standard "free lunch at the Pareto frontier" result.

Commercial ISAC systems typically set $\beta\in[0.7,0.9]$ in high-SNR regimes: keep most power in the data direction, allocate 10-30% to the sensing direction, and accept a ~0.5 bit/symbol rate loss for non-degenerate target estimation.

The Kobayashi-Caire-Kramer formulation maximises the joint information-distortion tradeoff: the input distribution $p(x)$ controls both the rate $I(X;Y|S)$ and the achievable distortion $D = \min_{\hat S}\mathbb{E}[d(S,\hat S(X,Y))]$. Both depend on the full distribution of $X$, not just its second moment. In contrast, the Fisher-information-based rate-CRB tradeoff depends on the input only through its sample covariance $\mathbf{R}_s$.

Any non-Gaussian input achieves the same CRB as a Gaussian with the same covariance (CRB is a second-order functional), but different non-Gaussian inputs achieve different $I(X;Y|S)$ and different $D$. So the CRB-rate frontier is strictly below the full information-distortion frontier whenever non-Gaussian inputs can do better — for instance, when the channel benefits from discrete modulation or when the distortion function is non-MSE.

They coincide asymptotically when: (i) the distortion is squared error, (ii) the channel is Gaussian with Gaussian state, (iii) the SNR is high enough that Gaussian input is near-optimal for communication, and (iv) the state prior is broad enough that the CRB is tight (no ambiguity regime). Under these conditions, both frameworks reduce to the same quadratic Pareto tradeoff.

CommIT (Kobayashi-Caire-Kramer 2018, Xiong-Liu-Cui-Yuan-Han-Caire 2023) use the information-theoretic formulation to characterise the fundamental capacity-distortion region for ISAC. The CRB-rate region is a second-order surrogate that is correct in the high-SNR Gaussian regime and misleading outside it. Research aimed at discrete-modulation ISAC designs or non-MSE distortion must use the joint bound.