Ferkans — Interactive Telecom Tutor

The Wiener Filter as the Centerpiece of Linear Estimation

The Wiener filter is beautiful, classical, and — in its pure form — rarely used in practice. Why? Because it assumes we know the second-order statistics exactly, and because it assumes stationarity. Real systems face non-stationarity (a user walks into a tunnel, a channel changes) and uncertainty about the statistics (we do not know $P_x(f)$ and $P_z(f)$ from first principles; we have to estimate them). The Wiener filter is the theoretical ceiling against which three practical alternatives are benchmarked:

The Kalman filter (Chapter 10) generalizes Wiener to time-varying state-space models. Its steady-state converges to the causal Wiener filter when the model is time-invariant.
Adaptive filters (LMS, RLS) estimate the statistics on the fly, producing a time-varying approximation to the Wiener filter that tracks slow changes in the channel.
Model-based Wiener filters are the causal Wiener filter with the PSDs replaced by parametric models (AR or ARMA) whose parameters are estimated from data.

This closing section works through the canonical AR(1)+noise example end-to-end, makes the Kalman-Wiener bridge explicit, and sketches the transition to adaptive filtering.

Example: AR(1)+Noise: End-to-End Walkthrough

Consider $X_n = 0.8 X_{n-1} + U_n$ with $\sigma_u^2 = 1$ , observed in white noise of variance $\sigma_z^2 = 0.5$ : $Y_n = X_n + Z_n$ . Compute (a) the non-causal Wiener filter and its MMSE; (b) the spectral factors $P_y^{\pm}$ ; (c) the causal Wiener filter, in recursive form; (d) the one-step prediction MMSE for $Y$ .

Solution

(a) Non-causal Wiener and MMSE

$P_x(f) = 1/|1 - 0.8 e^{-j 2\pi f}|^2 = 1/(1.64 - 1.6 \cos(2\pi f))$ . Signal variance: $\sigma_X^2 = 1/(1 - 0.64) = 2.778$ . SNR $= 2.778/0.5 \approx 5.56$ (7.45 dB). $\check{h}_{\text{nc}}(f) = P_x(f)/(P_x(f) + 0.5)$ . Numerically (integrating): $\sigma_{\text{nc}}^2 = \int \dfrac{0.5 \cdot P_x(f)}{P_x(f) + 0.5}\,df \approx 0.353.$

(b) Spectral factorization

Apply the formulas of ESpectral Factorization of the AR(1)+Noise PSD with $a = 0.8$ , $\sigma_u^2 = 1$ , $\sigma_z^2 = 0.5$ : $N_0 = 1 + 0.5 \cdot 1.64 = 1.82$ . Solve $b^2 - (1.82/(0.5 \cdot 0.8))\,b + 1 = 0$ , i.e., $b^2 - 4.55\,b + 1 = 0$ . Roots: $b \in \{0.231, 4.331\}$ . Pick $b = 0.231$ . $\beta^2 = 0.5 \cdot 0.8 / 0.231 = 1.731$ , so $\beta = 1.316$ . Therefore $P_y^+(f) = 1.316 (1 - 0.231 e^{-j 2\pi f}) / (1 - 0.8 e^{-j 2\pi f})$ .

(c) Causal Wiener filter

Apply EClosed-Form Causal Wiener Filter for AR(1)+Noise: $K = \sigma_u^2 / (\beta^2 (1 - a b)) = 1 / (1.731 \cdot (1 - 0.8 \cdot 0.231)) = 1/(1.731 \cdot 0.815) \approx 0.709$ . The recursion is $\hat{X}_n = 0.231\, \hat{X}_{n-1} + 0.709\, Y_n$ . Numerically the causal MMSE is $\sigma_c^2 \approx 0.435$ , a ratio $\sigma_c^2/\sigma_{\text{nc}}^2 \approx 1.23$ .

(d) One-step prediction for $Y$

$\sigma_p^2 = \beta^2 = 1.731$ by Kolmogorov-Szego (the geometric mean of $P_y(f)$ is $\beta^2$ for rational factorable spectra of this form). Compare with $r_{yy}[0] = \sigma_X^2 + \sigma_z^2 = 2.778 + 0.5 = 3.278$ . The ratio $1.731/3.278 \approx 0.528$ measures the predictability of $Y$ .

🎓CommIT Contribution(2021)

Wiener Prediction for Massive MIMO Channel Aging

G. Caire, M. B. Khalilsarai — IEEE Transactions on Wireless Communications

In massive MIMO systems with user mobility the channel evolves between uplink pilot transmissions, and the base station's precoder — designed from a stale estimate — suffers a SINR loss known as channel aging. The CommIT group showed that by modeling each antenna's channel as a narrowband WSS process in time (governed by a Jakes-like Doppler spectrum), one can apply the Wiener-Kolmogorov causal prediction framework of this chapter on a per-eigenbeam basis. The resulting subspace-based predictor tracks the dominant eigenmodes of the channel covariance and predicts each one independently. The prediction horizon is limited by the Kolmogorov-Szego bound: for a user moving at 30 km/h at 3 GHz carrier, the geometric-mean of the Jakes PSD falls one to three dB below the variance, corresponding to a useful prediction horizon of roughly one channel coherence time. Beyond this horizon the predictor loses to simply assuming the channel is unchanged.

massive-mimochannel-predictionwienerchannel-aging

The Bridge to Kalman: State-Space Is the Finite-Memory Formulation

For the AR(1)+noise problem, the causal Wiener filter is a first-order recursion: $\hat{X}_n = b \hat{X}_{n-1} + K Y_n$ . This is exactly the steady-state Kalman filter for the scalar state-space model $X_{n+1} = a X_n + U_n$ , $Y_n = X_n + Z_n$ . Here is the dictionary:

Wiener quantity	Kalman quantity
AR pole $a$	State transition scalar $F$
Innovation variance $\sigma_u^2$	Process noise variance $Q$
Noise variance $\sigma_z^2$	Measurement noise variance $R$
Wiener gain $K$	Steady-state Kalman gain $K_\infty$
Closed-loop pole $b$	$F(1 - K_\infty)$

The Kalman formulation has two advantages. First, it applies to time-varying state-space models, where the Wiener filter simply does not apply. Second, it is recursive by construction — no spectral factorization is needed, because the recursive update effectively computes it on the fly through the Riccati equation. The Wiener filter, in turn, has the virtue of an explicit closed-form expression and a cleaner connection to frequency-domain intuition. Both perspectives are essential, and Chapter 10 builds the Kalman framework from scratch.

Adaptive Wiener: LMS and RLS

When the statistics $r_{yy}$ and $r_{xy}$ are unknown, we can estimate them from data. Two archetypal algorithms:

Least Mean Squares (LMS) uses a stochastic-gradient update: $\mathbf{w}_{n+1} = \mathbf{w}_n + \mu\, e_n^*\, \mathbf{y}_n$ , where $e_n = X_n - \mathbf{w}_n^H \mathbf{y}_n$ is the instantaneous error. LMS converges to the Wiener filter as $\mu \to 0$ slowly, with complexity $O(N)$ per sample for a length- $N$ filter. Convergence requires $0 < \mu < 2/\lambda_{\max}(\mathbf{R}_{yy})$ , and the steady-state excess MSE (the "misadjustment") is approximately $\mu\,\text{tr}(\mathbf{R}_{yy})/2$ .
Recursive Least Squares (RLS) solves the exponentially-weighted least-squares problem $\sum_{k \leq n} \lambda^{n-k} |X_k - \mathbf{w}^H \mathbf{y}_k|^2$ recursively via the matrix inversion lemma. RLS converges much faster — in $O(N)$ samples rather than $O(N/\mu)$ — at the cost of $O(N^2)$ operations per sample.

Both are on-line approximations to the Wiener filter. In the limit of infinite data and constant statistics, both converge to the FIR Wiener solution. Their value lies in tracking time-varying statistics — something the stationary Wiener filter cannot do.

Why This Matters: MMSE Equalization as Wiener Filtering

In a frequency-selective fading channel, the received signal is $Y(f) = H(f) X(f) + Z(f)$ in the frequency domain. The MMSE equalizer, which you will meet in Chapter 11, is exactly the Wiener filter for this problem: $W_{\text{MMSE}}(f) = H^*(f) P_x(f) / (|H(f)|^2 P_x(f) + P_z(f))$ . The zero-forcing equalizer is the high-SNR limit. For OFDM (Book Telecom, Ch 14) each subcarrier's per-tone MMSE estimator is a scalar Wiener filter applied independently — this is one of the major reasons OFDM is the dominant physical-layer architecture in 4G, 5G, and Wi-Fi.

See full treatment in Chapter 11

🔧Engineering Note

Robustness Under Statistical Mismatch

A Wiener filter designed with PSD estimates that differ from the true PSDs produces a suboptimal MSE. For signal-plus-noise problems, a standard robustness result says that if the assumed SNR is off by a factor of $\rho$ , the MSE degrades by approximately $\rho^2/(1+\rho)^2$ relative to the optimum at moderate SNR. The practical implication: Wiener filters are most useful when the statistics are reasonably stable and estimable from training data. In rapidly changing environments an adaptive filter, which uses only a short effective memory, typically outperforms a mis-designed static Wiener filter.

Practical Constraints

•
Performance degrades gracefully under statistical mismatch but requires correct qualitative structure (e.g., signal-in-noise).
•
Robust variants (e.g., minimax Wiener filters) exist for worst-case uncertainty sets.

The Wiener Family Tree — Conceptual map of the estimators derived from (or approximating) the Wiener filter. Non-causal Wiener, causal Wiener, and one-step prediction are exact classical solutions; Kalman generalizes to time-varying state-space models; LMS/RLS adaptively estimate the Wiener coefficients from data.

Key Takeaway

The Wiener filter sits at the crossroads of linear estimation. Every adaptive or model-based filter in practice — Kalman, LMS, RLS, MMSE equalizer — is a variant or approximation of it. Knowing the Wiener solution gives you the fundamental limit against which any practical algorithm must be measured.

Quick Check

In what sense does the steady-state Kalman filter equal the causal Wiener filter?

They coincide exactly when the state-space model is time-invariant and the Kalman filter has reached its steady-state gain.

They are exactly equal for any state-space model, time-varying or not.

They differ by a time-invariant all-pass filter.

Correction: