Exercises

$r_{xx}(0) = 4\cos(0) = 4$. Since $\cos$ is continuous, $\lim_{\tau\to 0}r_{xx}(\tau) = 4 = r_{xx}(0)$.

The autocorrelation is continuous at $\tau = 0$, so by TCharacterization of Mean-Square Continuity, the process is m.s.-continuous.

$(2\pi f)^2 P_x(f) = \frac{4\pi^2 f^2 A}{1 + (f/B)^2}$. For large $|f|$, this behaves as $4\pi^2 A B^2$, a constant. The integral diverges.

The m.s. derivative does not exist. This is the Lorentzian PSD (corresponding to the OU process), which decays too slowly ($\sim 1/f^2$) for the derivative condition.

$P_{X'}(f) = (2\pi f)^2 \cdot \frac{1}{(1+f^2)^2} = \frac{4\pi^2 f^2}{(1+f^2)^2}$. This decays as $4\pi^2/f^2$ for large $|f|$, which is integrable.

$\int (2\pi f)^4 \cdot \frac{1}{(1+f^2)^2}\, df = \int \frac{16\pi^4 f^4}{(1+f^2)^2}\, df$. For large $|f|$, the integrand $\sim 16\pi^4$, so the integral diverges.

The first derivative exists but the second does not. The PSD decays as $1/f^4$, so only $n = 1$ derivative is possible (requires decay faster than $1/f^{2n+1}$).

$\mathbb{E}[|X'(t)|^2] = \int_{-W}^{W} (2\pi f)^2 \cdot \frac{N_0}{2}\, df = N_0 \int_0^W 4\pi^2 f^2\, df = \frac{4\pi^2 N_0 W^3}{3}.$$

The derivative power grows as $W^3$ — the wider the bandwidth, the more high-frequency energy is amplified by differentiation. This quantifies how "rough" a bandlimited process becomes as bandwidth increases.

$\mathbb{E}[X(t)X'(t+\tau)^*] = \frac{\partial}{\partial \tau}r_{xx}(\tau)$. Setting $\tau = 0$: $\mathbb{E}[X(t)X'(t)^*] = r_{xx}'(0)$.

For a real-valued WSS process, $r_{xx}(\tau) = r_{xx}(-\tau)$, so $r_{xx}'(\tau) = -r_{xx}'(-\tau)$. At $\tau = 0$: $r_{xx}'(0) = -r_{xx}'(0)$, hence $r_{xx}'(0) = 0$.

$\mathbb{E}[X(t)X'(t)] = 0$. The process and its derivative are uncorrelated at the same time instant. For Gaussian processes, they are also independent.

$X'(t) = -2\pi f_0 A\sin(2\pi f_0 t + \Theta)$. This is well-defined pointwise (and hence in m.s.) since $X(t)$ is differentiable path-by-path.

$r_{xx}(\tau) = \frac{A^2}{2}\cos(2\pi f_0\tau)$. $r_{xx}''(\tau) = -\frac{A^2}{2}(2\pi f_0)^2\cos(2\pi f_0\tau)$. $-r_{xx}''(0) = \frac{A^2}{2}(2\pi f_0)^2 < \infty$, confirming existence.

$P_x(f) = \frac{A^2}{4}[\delta(f - f_0) + \delta(f + f_0)]$. So $P_{X'}(f) = (2\pi f)^2 P_x(f) = \frac{A^2(2\pi f_0)^2}{4}[\delta(f - f_0) + \delta(f + f_0)]$.

$W = 5$ kHz, so $f_s = 2 \times 5 = 10$ kHz (10,000 samples/second). $T_s = 1/(2W) = 0.1$ ms.

The triangular PSD is the convolution of a rectangle with itself in the frequency domain. By the Fourier transform pair, $r_{xx}(\tau) = 4000\,\text{sinc}^2(4000\tau)$.

$T_s = 1/8000$ s. $r_{xx}[m] = 4000\,\text{sinc}^2(m/2)$. For $m = 0$: $r_{xx}[0] = 4000$. For odd $m$: $\text{sinc}(m/2) \neq 0$ in general. For $m = 1$: $r_{xx}[1] = 4000\,\text{sinc}^2(1/2) = 4000 \cdot (2/\pi)^2 \approx 1621$.

Unlike the flat PSD case, the triangular PSD yields correlated Nyquist-rate samples. The correlation decays but is nonzero — the non-flat PSD shape introduces correlation.

$r_{xx}(\tau) = N_0\,\text{sinc}(2W\tau)$. $T_s = 1/(4W)$, so $r_{xx}[0] = N_0$ and $r_{xx}[1] = N_0\,\text{sinc}(1/2) = N_0 \cdot \frac{2}{\pi} \approx 0.637\,N_0$.

Since the process is zero-mean, the covariance matrix equals the correlation matrix: $$\mathbf{R} = N_0 \begin{pmatrix} 1 & \frac{2}{\pi} \\ \frac{2}{\pi} & 1 \end{pmatrix}.$$ The samples are correlated due to oversampling.

Define $e_N(t) = X(t) - \hat{X}_N(t)$. By the isometry between time and frequency domains, $$\mathbb{E}[|e_N(t)|^2] = \int_{-W}^{W} P_x(f) |E_N(f, t)|^2\, df,$$ where $E_N(f, t) = e^{j2\pi ft} - \sum_{n=-N}^{N} e^{j2\pi f n T_s}\text{sinc}(t/T_s - n)$.

The cardinal series $\sum_{n=-\infty}^{\infty} e^{j2\pi f n T_s}\text{sinc}(t/T_s - n)$ converges to $e^{j2\pi ft}$ for $|f| \leq W$ (Poisson summation formula). Therefore $|E_N(f, t)|^2 \to 0$ pointwise for each $f \in [-W, W]$.

$|E_N(f, t)|^2 \leq 4$ for all $N$ (triangle inequality, since each term is bounded by 1). Since $P_x(f)$ is integrable on $[-W, W]$, dominated convergence gives $\mathbb{E}[|e_N(t)|^2] \to 0$.

$2WT = 2 \times 10^6 \times 10^{-3} = 2000$ degrees of freedom. For a flat-PSD Gaussian process, these correspond to 2000 i.i.d. Nyquist-rate samples.

$R_X(t,s) = \sigma^2[\cos(2\pi f_0 t)\cos(2\pi f_0 s) + \sin(2\pi f_0 t)\sin(2\pi f_0 s)] = \sigma^2\cos(2\pi f_0(t-s))$.

On $[0, T]$ with $T = 1/f_0$, the functions $\phi_1(t) = \sqrt{2f_0}\cos(2\pi f_0 t)$ and $\phi_2(t) = \sqrt{2f_0}\sin(2\pi f_0 t)$ are orthonormal. Computing $\int_0^T R_X(t,s)\phi_k(s)\, ds$ yields $(\sigma^2/(2f_0))\phi_k(t)$ for $k = 1, 2$.

$X(t) = Z_1\phi_1(t) + Z_2\phi_2(t)$ with $\lambda_1 = \lambda_2 = \sigma^2/(2f_0)$, $Z_1 = A/\sqrt{2f_0}$, $Z_2 = B/\sqrt{2f_0}$. This is a two-term KL expansion (the process lives in a 2D subspace).

$\sigma^2 \int_0^T e^{-\alpha|t-s|}\phi(s)\, ds = \lambda\phi(t)$. Split at $s = t$: $\sigma^2[\int_0^t e^{-\alpha(t-s)}\phi(s)\, ds + \int_t^T e^{-\alpha(s-t)}\phi(s)\, ds] = \lambda\phi(t)$.

Two differentiations yield $-2\alpha\sigma^2\phi(t) = \lambda(\phi''(t) + \alpha^2\phi(t) - 2\alpha\sigma^2\phi(t)/\lambda)$. Simplifying: $\phi''(t) + \omega_n^2\phi(t) = 0$ where $\omega_n^2 = 2\alpha\sigma^2/\lambda - \alpha^2$.

The boundary conditions are $\phi'(0) = \alpha\phi(0)$ and $\phi'(T) = -\alpha\phi(T)$. The solution $\phi(t) = A\sin(\omega_n t + \theta_n)$ requires the transcendental equation $\tan(\omega_n T) = \frac{2\alpha\omega_n}{\omega_n^2 - \alpha^2}$ to determine the eigenfrequencies $\omega_n$.

Once $\omega_n$ is found, $\lambda_n = \frac{2\alpha\sigma^2}{\alpha^2 + \omega_n^2}$. The eigenvalues decay and must be computed numerically for each $n$.

By Mercer's theorem, $R_X(t,s) = \sum_n \lambda_n \phi_n(t)\phi_n^*(s)$ with absolute and uniform convergence.

Setting $s = t$: $R_X(t,t) = \sum_n \lambda_n |\phi_n(t)|^2$. Integrating over $[0, T]$: $\int_0^T R_X(t,t)\, dt = \sum_n \lambda_n \int_0^T |\phi_n(t)|^2\, dt = \sum_n \lambda_n$, since $\{\phi_n\}$ are orthonormal.

$\int_0^T r_{xx}(t-s) \frac{e^{j2\pi f_n s}}{\sqrt{T}}\, ds = \frac{1}{\sqrt{T}} \int_0^T r_{xx}(\tau') e^{j2\pi f_n(t-\tau')}\, d\tau'$ where $\tau' = t - s$.

For large $T$, the integration domain covers most of the support of $r_{xx}(\tau')$ (assuming it decays). The integral approaches $\frac{e^{j2\pi f_n t}}{\sqrt{T}} \int_{-\infty}^{\infty} r_{xx}(\tau') e^{-j2\pi f_n \tau'}\, d\tau' = \frac{P_x(f_n)}{\sqrt{T}} e^{j2\pi f_n t}$ by the Wiener-Khinchin theorem.

This gives $\lambda_n \cdot \phi_n(t) \approx P_x(f_n) \cdot \phi_n(t)$ where $f_n = n/T$. So $\lambda_n \approx P_x(n/T)$, and the eigenfunctions are approximately complex exponentials. (The factor of $1/T$ vs. $P_x$ depends on the normalization convention.)

The KL theory guarantees that $\{Z_n\}$ are uncorrelated: $\mathbb{E}[Z_n Z_m^*] = 0$ for $n \neq m$. However, for non-Gaussian processes, uncorrelatedness does not imply independence. The $Z_n$ may have higher-order statistical dependencies. Only for Gaussian processes does uncorrelatedness imply independence (since the joint distribution is fully characterized by second-order statistics).

$2WT = 2 \times 100 \times 0.1 = 20$. The KL expansion has approximately 20 significant eigenvalues, with the remaining eigenvalues being negligibly small. This is consistent with the sampling theorem: 20 Nyquist-rate samples suffice to represent the process on $[0, 0.1]$.

The integral $\int R_X(t,s)\phi(s)\, ds = \lambda\phi(t)$ becomes $\boldsymbol{\Sigma}\mathbf{u} = \lambda\mathbf{u}$ in the discrete case, where $\mathbf{u}$ is an eigenvector and $\lambda$ is the eigenvalue.

The eigenvectors $\mathbf{u}_1, \ldots, \mathbf{u}_N$ (columns of $\mathbf{U}$) play the role of the eigenfunctions $\phi_n(t)$. The KL coefficients are $Z_n = \mathbf{u}_n^H\mathbf{X}$.

$\mathbb{E}[Z_n Z_m^*] = \mathbf{u}_n^H \boldsymbol{\Sigma} \mathbf{u}_m = \lambda_n \delta_{nm}$. This is PCA: the eigendecomposition of $\boldsymbol{\Sigma}$ is the discrete KL expansion.

The $N$-term truncation error is $\varepsilon_N = \text{tr}(\mathcal{R}) - \sum_{n=1}^N \langle \psi_n, \mathcal{R}\psi_n\rangle$, where $\mathcal{R}$ is the autocorrelation operator and $\{\psi_n\}$ is any orthonormal set. Minimizing $\varepsilon_N$ is equivalent to maximizing $\sum_{n=1}^N \langle \psi_n, \mathcal{R}\psi_n\rangle$.

By the Courant-Fischer theorem for compact self-adjoint operators, $\max_{\text{dim}(V_N)=N} \sum_{n=1}^N \langle \psi_n, \mathcal{R}\psi_n\rangle = \sum_{n=1}^N \lambda_n$, achieved when $V_N = \text{span}(\phi_1, \ldots, \phi_N)$, the top $N$ eigenfunctions.

Therefore $\varepsilon_N^{(\text{KL})} = \text{tr}(\mathcal{R}) - \sum_{n=1}^N \lambda_n = \sum_{n>N}\lambda_n$, and any other $N$-term expansion has error $\geq \varepsilon_N^{(\text{KL})}$.

The eigendecomposition $\mathbf{R} = \mathbf{U}\boldsymbol{\Lambda}\mathbf{U}^H$ expresses the channel vector in the eigenbasis: $\mathbf{h} = \mathbf{U}\mathbf{z}$ where $\mathbf{z} = \mathbf{U}^H\mathbf{h}$ has uncorrelated components with variance $\lambda_n$. This is the discrete, spatial analogue of the KL expansion.

If the eigenvalues decay rapidly ($\lambda_n \approx 0$ for $n > r$), the channel effectively lives in an $r$-dimensional subspace spanned by the dominant eigenvectors $\mathbf{U}_r = [\mathbf{u}_1, \ldots, \mathbf{u}_r]$. The pre-beamformer $\mathbf{B} = \mathbf{U}_r$ projects onto this subspace.

In JSDM, users are grouped by covariance similarity. Each group $g$ has pre-beamformer $\mathbf{B}_g$ spanning the group's dominant eigenmodes. Since different groups occupy approximately orthogonal eigenspaces, inter-group interference is small. Within each group, a second-stage beamformer handles the remaining instantaneous CSI — but in a reduced dimension $r \ll N_t$, dramatically reducing feedback requirements.