Ferkans — Interactive Telecom Tutor

Real-Time Filtering: You Cannot Look Into the Future

In any real-time system — a radio receiver, a noise-cancelling headphone, a Kalman-filter navigation unit — the estimate $\hat{X}_n$ must be computed from observations up to time $n$ . We must restrict the filter support to $\mathcal{K} = \mathbb{Z}_{\geq 0}$ . The Wiener-Hopf equation $\sum_{k \geq 0} h[k]\, r_{yy}[\ell - k] = r_{xy}[\ell], \qquad \ell \geq 0,$ is not a plain convolution: it holds only on the non-negative half-line. Taking the Fourier transform of both sides leaves the right-hand side unconstrained on $\ell < 0$ , so we cannot simply divide. The strategy is to pre-whiten the observation using the innovations representation of Section 9.3, solve the easy problem in innovations space, then map back.

Theorem: Causal Wiener Filter

Let $(X_n, Y_n)$ be jointly WSS with $P_y$ satisfying Paley-Wiener, and let $P_y(f) = P_y^+(f) P_y^-(f)$ be the spectral factorization of the observation. The MMSE causal linear estimator $\hat{X}_n = \sum_{k \geq 0} h_c[k] Y_{n-k}$ has transfer function $\boxed{\;\check{h}_c(f) = \frac{1}{P_y^+(f)}\left[\frac{P_{xy}(f)}{P_y^-(f)}\right]_{+}\;}$ where $[\cdot]_+$ is the causal projection operator: if $G(f) = \sum_{n \in \mathbb{Z}} g[n] e^{-j 2\pi f n}$ , then $[G(f)]_+ = \sum_{n \geq 0} g[n] e^{-j 2\pi f n}$ . The causal MMSE is $\sigma_c^2 = \int_{-1/2}^{1/2}\left[P_x(f) - \check{h}_c(f)\, P_{xy}^*(f)\right] df.$

Read the formula from right to left, as a three-step recipe. (1) Whiten the observation by dividing the cross-spectrum by the anti-causal factor $P_y^-(f)$ — this effectively re-expresses $P_{xy}$ in terms of the innovations $J_n$ . (2) Discard the anti-causal components of the result: that is what $[\cdot]_+$ does. (3) Color the output back by dividing by the causal factor $P_y^+(f)$ . The filter is the composition of a whitener, a causal truncator, and a recoloring stage.

Proof

Whiten the observation

Let $J_n$ be the innovations of $Y_n$ . Because $J$ is white and $\text{span}\{J_k : k \leq n\} = \text{span}\{Y_k : k \leq n\}$ , the causal MMSE estimate of $X_n$ from $\{Y_k : k \leq n\}$ equals the causal MMSE estimate from $\{J_k : k \leq n\}$ .

Solve in innovations space

Because $J$ is white, the causal Wiener-Hopf equation decouples: for each $\ell \geq 0$ , $h_J[\ell] \cdot 1 = r_{xJ}[\ell]$ , so $h_J[\ell] = r_{xJ}[\ell]$ for $\ell \geq 0$ and $h_J[\ell] = 0$ for $\ell < 0$ . Equivalently, in the frequency domain, $\check{h}_J(f) = \left[P_{xJ}(f)\right]_+$ .

Compute the cross-spectrum $P_{xJ}$

Since $J_n = Y_n$ passed through $1/P_y^+(f)$ , $P_{xJ}(f) = P_{xy}(f) \cdot \overline{1/P_y^+(f)} = P_{xy}(f)/P_y^-(f)$ , using $P_y^- = (P_y^+)^*$ .

Recolor and assemble

The total filter from $Y$ to $\hat{X}$ is the composition of the whitener $1/P_y^+(f)$ with the innovation filter $[P_{xy}/P_y^-]_+$ . Both are causal, so their composition is causal, and the frequency response is the product: $\check{h}_c(f) = (1/P_y^+(f)) \cdot [P_{xy}(f)/P_y^-(f)]_+$ .

Derive the MMSE

By TMMSE of the Wiener Estimator, $\sigma_c^2 = r_{xx}[0] - \sum_{k \geq 0} h_c[k] r_{xy}^*[k] = \int P_x(f) df - \int \check{h}_c(f) P_{xy}^*(f) df$ (Parseval). Combine into a single integral to obtain the stated formula.

,

Causal MMSE Is Never Smaller Than Non-Causal MMSE

For any jointly WSS pair $(X, Y)$ , $\sigma_{\text{nc}}^2 \leq \sigma_c^2 \leq \sigma_p^2,$ where $\sigma_p^2$ is the one-step prediction MMSE of $Y$ (Section 9.5). The first inequality reflects the fact that the causal filter is a special case of the non-causal filter (with zero taps for $k < 0$ ), so its MSE cannot be smaller. The second inequality is less obvious and requires the observation that the prediction problem is the causal Wiener problem for the specific target $X_n = Y_n$ . The gap $\sigma_c^2 - \sigma_{\text{nc}}^2$ quantifies the price of real-time causality and depends on how much future information carries useful signal about $X_n$ .

Example: Closed-Form Causal Wiener Filter for AR(1)+Noise

Using the spectral factors from ESpectral Factorization of the AR(1)+Noise PSD, derive the causal Wiener filter for the AR(1)+noise problem in closed form.

Solution

Assemble the cross-spectrum term

$P_{xy}(f) = P_x(f) = \sigma_u^2 / |1 - a e^{-j 2\pi f}|^2$ , and $P_y^-(f) = \beta (1 - b e^{+j 2\pi f}) / (1 - a e^{+j 2\pi f})$ . Dividing gives $\frac{P_{xy}(f)}{P_y^-(f)} = \frac{\sigma_u^2 / [(1 - a e^{-j2\pi f})(1 - a e^{+j2\pi f})]}{\beta(1 - b e^{+j2\pi f})/(1 - a e^{+j2\pi f})} = \frac{\sigma_u^2}{\beta (1 - a e^{-j2\pi f})(1 - b e^{+j2\pi f})}.$

Partial-fractions decomposition

Write the denominator product as $(1 - a z^{-1})(1 - b z)$ with $z = e^{j2\pi f}$ , and decompose into causal and anti-causal parts: $\frac{1}{(1 - a z^{-1})(1 - b z)} = \frac{1}{1 - a b}\left[\frac{1}{1 - a z^{-1}} + \frac{b z}{1 - b z}\right].$ The first term is causal (expands in $z^{-n}$ , $n \geq 0$ ); the second term is strictly anti-causal.

Apply the causal projector and recolor

$[P_{xy}/P_y^-]_+ = \dfrac{\sigma_u^2}{\beta(1-ab)} \cdot \dfrac{1}{1 - a e^{-j2\pi f}}$ . Now divide by $P_y^+(f) = \beta(1 - b e^{-j 2\pi f})/(1 - a e^{-j 2\pi f})$ : $\check{h}_c(f) = \frac{1}{P_y^+(f)} \cdot \frac{\sigma_u^2/(\beta(1-ab))}{1 - a e^{-j 2\pi f}} = \frac{\sigma_u^2}{\beta^2 (1-ab)} \cdot \frac{1}{1 - b e^{-j 2\pi f}}.$

Identify the recursive form

This is the transfer function of a first-order causal recursion: $\hat{X}_n = b\, \hat{X}_{n-1} + K\, Y_n, \qquad K = \frac{\sigma_u^2}{\beta^2(1 - ab)}.$ It has the exact form of a steady-state Kalman filter for this scalar state-space model — indeed, the steady-state Kalman filter is the causal Wiener filter. This is no coincidence; see Chapter 10.

Causal vs Non-Causal MMSE Ratio

For three AR(1) coefficients, the ratio $\sigma_c^2 / \sigma_{\text{nc}}^2$ as a function of SNR. The ratio is always at least 1; it measures how much the causal constraint costs. For weakly correlated signals ( $a$ small) the ratio stays near 1 because the past contains most of the information; for strongly correlated signals the gap opens at moderate SNRs.

Parameters

Min SNR (dB)-10

Max SNR (dB)30

Design Flow for the Causal Wiener Filter (Rational PSDs)

Complexity: Spectral factorization is O(d^3) for degree-d rational PSDs. Projection and recoloring are O(d).

Input: rational PSDs P_x(f), P_y(f), P_xy(f)

Output: causal Wiener filter h_c[n]

1. Check Paley-Wiener: verify int log P_y(f) df > -infty

2. Spectral factorization:

factor numerator and denominator of P_y(z)

collect all roots with |z| < 1 into P_y^+(z)

set P_y^-(z) = conj(P_y^+(1/z^*))

3. Whiten cross-spectrum:

G(f) = P_xy(f) / P_y^-(f)

4. Causal projection:

G_+(f) = [G(f)]_+ (partial fractions: keep causal terms)

5. Recolor:

H_c(f) = G_+(f) / P_y^+(f)

6. Invert DTFT to obtain h_c[n] (or keep H_c for frequency-domain use)

For non-rational PSDs, step 2 becomes a numerical integration (of $\log P_y$ ) followed by cepstral recovery of the coefficients $c_n$ . This is the standard numerical route in real codes.

Non-Causal vs Causal Wiener Filter

Aspect	Non-Causal	Causal
Support of $h[k]$	$k \in \mathbb{Z}$ (all lags)	$k \geq 0$
Transfer function	$P_{xy}(f)/P_y(f)$	$(1/P_y^+(f)) \cdot [P_{xy}(f)/P_y^-(f)]_+$
Requires spectral factorization?	No	Yes
Requires Paley-Wiener?	No (only $P_y(f) > 0$ )	Yes
MMSE	$\int (P_x - \|P_{xy}\|^2/P_y)\,df$	$\int (P_x - \check{h}_c P_{xy}^*)\,df$
Ordering	$\sigma_{\text{nc}}^2 \leq \sigma_c^2$ always	equal iff $P_{xy}/P_y^-$ is already causal
Real-time?	No (batch)	Yes
Implementation	FFT block	Recursive (IIR) or FIR truncation

Common Mistake: $[\cdot]_+$ Is Not the Same as the Real Part

Mistake:

Replacing the causal projector $[G(f)]_+$ with $\mathrm{Re}\,G(f)$ or with a zero-phase truncation.

Correction:

The causal projector keeps the Fourier coefficients $g[n]$ for $n \geq 0$ and throws away those for $n < 0$ . In the frequency domain this is not a linear-phase operation — it produces a complex function even when $G(f)$ is real. The only exact way to compute $[G(f)]_+$ is to take the inverse DTFT, multiply by the unit step, and take the DTFT back (or do partial fractions for rational $G$ ).

⚠️Engineering Note

Stability of the Causal Wiener IIR Implementation

Because the causal Wiener filter involves $1/P_y^+(f)$ , its impulse response may be IIR. The filter is guaranteed stable (all poles inside the unit circle) precisely because $P_y^+(f)$ is minimum-phase. In finite-precision arithmetic, however, poles close to the unit circle can produce numerically unstable recursions; in DSP practice one often uses a second-order cascade implementation and monitors the pole radii. For very-high-order rational PSDs, balanced-truncation model reduction can yield a lower-order filter with negligible loss in MMSE.

Practical Constraints

•
Minimum-phase spectral factor guarantees mathematical stability.
•
Numerical stability requires care when poles approach the unit circle.

Key Takeaway

The causal Wiener filter is obtained by a three-stage procedure: whiten the observation with the inverse minimum-phase factor $1/P_y^+(f)$ , project onto the causal subspace, then recolor. The resulting filter is causal, stable, and minimum-MSE over all causal linear estimators. Its recursive implementation coincides with the steady-state Kalman filter for rational spectra.

Quick Check

Which of the following is always TRUE about the causal and non-causal Wiener MMSE?

$\sigma_c^2 \geq \sigma_{\text{nc}}^2$ with equality iff $P_{xy}(f)/P_y^-(f)$ is already causal.

$\sigma_c^2 \leq \sigma_{\text{nc}}^2$ because the causal filter uses less information.

$\sigma_c^2 = \sigma_{\text{nc}}^2$ whenever the signal is white.