Ferkans — Interactive Telecom Tutor

Estimating a Signal from Noisy Observations

The matched filter answers the question: "Is the signal there?" Now we ask a different question: "What is the signal?" Given a noisy observation $Y(t) = X(t) + N(t)$ , where $X(t)$ is a WSS signal process and $N(t)$ is noise, we want to design an LTI filter that produces the best estimate of $X(t)$ in the minimum mean-square error (MMSE) sense. This is the Wiener filter — the frequency-domain solution to the LMMSE estimation problem for stationary processes. The result is elegant: the optimal filter simply weights each frequency by the fraction of total power at that frequency that comes from the signal.

,

Definition:
The Wiener Filtering Problem

Let $Y(t) = X(t) + N(t)$ , where $X(t)$ and $N(t)$ are jointly WSS, zero-mean processes that are uncorrelated: $r_{xn}(\tau) = 0$ for all $\tau$ . The Wiener filtering problem is to find the LTI filter $h(t)$ that minimizes the mean-square error $\text{MSE} = \mathbb{E}\!\left[|X(t) - \hat{X}(t)|^2\right],$ where $\hat{X}(t) = (h * Y)(t) = \int h(\tau) Y(t - \tau)\, d\tau$ is the filtered estimate.

Theorem: The Wiener-Hopf Equation

The optimal (non-causal) Wiener filter satisfies the Wiener-Hopf equation in the time domain: $r_{xy}(\tau) = \int_{-\infty}^{\infty} h_{\text{opt}}(\alpha)\, r_{yy}(\tau - \alpha)\, d\alpha = h_{\text{opt}}(\tau) * r_{yy}(\tau),$ or equivalently, $r_{xy}(\tau) = h_{\text{opt}} * r_{yy}(\tau)$ for all $\tau$ .

This is the orthogonality principle in disguise: the estimation error $X(t) - \hat{X}(t)$ must be orthogonal to the observation $Y(s)$ for all $s$ . Writing out this orthogonality condition gives a convolution equation relating the optimal filter to the input and cross-correlation functions.

Proof

Orthogonality principle

The MSE is minimized when the error $e(t) = X(t) - \hat{X}(t)$ is orthogonal to the data: $\mathbb{E}[e(t) Y^*(s)] = 0$ for all $s$ . This gives $\mathbb{E}[X(t) Y^*(s)] = \mathbb{E}[\hat{X}(t) Y^*(s)],$ i.e., $r_{xy}(t - s) = \int h_{\text{opt}}(\alpha)\, r_{yy}(t - \alpha - s)\, d\alpha$ .

Substitute $ au = t - s$

Setting $\tau = t - s$ : $r_{xy}(\tau) = \int h_{\text{opt}}(\alpha)\, r_{yy}(\tau - \alpha)\, d\alpha = (h_{\text{opt}} * r_{yy})(\tau).$ This is the Wiener-Hopf integral equation.

,

Theorem: The Non-Causal Wiener Filter

For the signal-plus-noise model $Y(t) = X(t) + N(t)$ with $X$ and $N$ uncorrelated WSS processes, the non-causal Wiener filter has frequency response $\boxed{\check{h}_{\text{opt}}(f) = \frac{P_x(f)}{P_x(f) + P_N(f)},}$ where $P_x(f)$ is the signal PSD and $P_N(f)$ is the noise PSD. The minimum MSE is $\text{MMSE} = \int_{-\infty}^{\infty} \frac{P_x(f)\, P_N(f)}{P_x(f) + P_N(f)}\, df.$

At each frequency, the Wiener filter computes a weight equal to the fraction of total power contributed by the signal. Where $\text{SNR}(f) = P_x(f)/P_N(f) \gg 1$ , the filter passes that frequency almost unchanged ( $\check{h} \approx 1$ ). Where $\text{SNR}(f) \ll 1$ , the filter suppresses that frequency ( $\check{h} \approx 0$ ). The Wiener filter is the optimal frequency-by-frequency tradeoff between signal distortion and noise suppression.

Proof

Transform the Wiener-Hopf equation

Taking the Fourier transform of $r_{xy}(\tau) = h_{\text{opt}} * r_{yy}(\tau)$ : $P_{xy}(f) = \check{h}_{\text{opt}}(f)\, P_y(f).$ Since $X$ and $N$ are uncorrelated, $P_y(f) = P_x(f) + P_N(f)$ and $P_{xy}(f) = P_x(f)$ . Solving: $\check{h}_{\text{opt}}(f) = \frac{P_x(f)}{P_x(f) + P_N(f)}.$

Minimum MSE

The error PSD is $P_e(f) = P_x(f)(1 - \check{h}_{\text{opt}}(f))$ , since the signal component not passed by the filter is lost. Substituting: $P_e(f) = P_x(f) \cdot \frac{P_N(f)}{P_x(f) + P_N(f)}.$ Integrating: $\text{MMSE} = \int P_e(f)\, df = \int \frac{P_x(f) P_N(f)}{P_x(f) + P_N(f)}\, df$ .

Alternative form

Using $\text{SNR}(f) = P_x(f)/P_N(f)$ , we can write $\check{h}_{\text{opt}}(f) = \frac{\text{SNR}(f)}{1 + \text{SNR}(f)}$ and $\text{MMSE} = \int \frac{P_x(f)}{1 + \text{SNR}(f)}\, df$ .

, ,

Example: Wiener Filter for Signal in White Noise

A WSS signal $X(t)$ with PSD $P_x(f) = \frac{A}{1 + (f/f_0)^2}$ (Lorentzian spectrum) is observed in additive white noise with PSD $P_N(f) = N_0/2$ . Find the Wiener filter frequency response and the MMSE.

Solution

Wiener filter

$\check{h}_{\text{opt}}(f) = \frac{A / (1 + (f/f_0)^2)}{A / (1 + (f/f_0)^2) + N_0/2} = \frac{1}{1 + \frac{N_0}{2A}(1 + (f/f_0)^2)}.$ $Define$ \gamma = N_0/(2A) $(inverse SNR parameter). Then$ \check{h}_{\text{opt}}(f) = \frac{1}{1 + \gamma + \gamma (f/f_0)^2} = \frac{1}{(1 + \gamma)(1 + (f/f_1)^2)} $where$ f_1 = f_0 \sqrt{(1 + \gamma)/\gamma} $. This is itself a lowpass filter with a bandwidth that shrinks as noise increases (smaller$ A/N_0$).

Limiting cases

High SNR ( $\gamma \to 0$ ): $\check{h}_{\text{opt}}(f) \to 1$ (pass everything).
Low SNR ( $\gamma \to \infty$ ): $\check{h}_{\text{opt}}(f) \to 0$ (suppress everything). The Wiener filter automatically adapts its bandwidth to the SNR.

MMSE

$\text{MMSE} = \int \frac{A \cdot N_0/2}{A + (N_0/2)(1 + (f/f_0)^2)} \cdot \frac{df}{1 + (f/f_0)^2}.$ $ This can be evaluated in closed form but the key insight is that MMSE decreases as the signal PSD becomes more concentrated (narrowband signals are easier to extract).

Example: Discrete-Time Wiener Filter for AR(1) Signal

An AR(1) signal $X_n = a X_{n-1} + W_n$ with $|a| < 1$ and $\sigma^2_{W} = 1$ is observed in additive white noise: $Y_n = X_n + V_n$ with $\sigma^2_{V} = \sigma_v^2$ . Find the DT Wiener filter.

Solution

Signal PSD

From Section 15.1, $P_x(f) = \frac{1}{|1 - a e^{-j2\pi f}|^2} = \frac{1}{1 - 2a\cos(2\pi f) + a^2}$ .

Noise PSD

$P_V(f) = \sigma_v^2$ (flat).

Wiener filter

$\check{h}_{\text{opt}}(f) = \frac{1/(1 - 2a\cos(2\pi f) + a^2)}{1/(1 - 2a\cos(2\pi f) + a^2) + \sigma_v^2} = \frac{1}{1 + \sigma_v^2(1 - 2a\cos(2\pi f) + a^2)}.$ $At high SNR ($ \sigma_v^2 \to 0 $), this approaches 1. At low SNR, the filter becomes a narrowband filter centered at$ f = 0 $(for$ a > 0 $) or$ f = 1/2 $(for$ a < 0$).

Wiener Filter Frequency Response vs. SNR

Explore how the Wiener filter $\check{h}_{\text{opt}}(f) = P_x(f)/(P_x(f) + P_N(f))$ adapts to the signal-to-noise ratio at each frequency. Adjust the signal and noise PSDs to see the filter shape change.

Parameters

f_0

(signal bandwidth, Hz)3

N_0

0.5

A

(signal power parameter)2

Wiener Denoising: Before and After

See the Wiener filter in action. A realization of a WSS signal is corrupted by noise, and the Wiener filter extracts the signal estimate. Compare input and output in both time and frequency domains.

Parameters

\text{SNR}

(dB)10

Signal bandwidth (Hz)2

Random seed42

Matched Filter vs. Wiener Filter

Property	Matched Filter	Wiener Filter
Goal	Maximize output SNR at one instant	Minimize MSE of signal estimate
Signal model	Known deterministic $s(t)$	WSS random process $X(t)$
Noise	White (PSD $N_0/2$ )	Any WSS noise $N(t)$
Optimal $\check{h}(f)$	$\check{s}^*(f) e^{-j2\\pi f t_0}$	$\P_x(f) / (\P_x(f) + P_N(f))$
Performance metric	$\\text{SNR}_{\\max} = 2E_s/\N_0$	$\\text{MMSE} = \\int \\frac{\P_x P_N}{\P_x + P_N}\\, df$
Uses signal shape?	Yes (explicitly)	Only through PSD
Application	Radar, digital communications	Speech enhancement, channel estimation

Quick Check

As the noise PSD $P_N(f) \to 0$ at all frequencies, what does the Wiener filter converge to?

The identity filter $\check{h}(f) = 1$

The matched filter

Zero (suppress everything)

Correction:

The identity filter

\check{h}(f) = 1

With no noise, the observation equals the signal, so the optimal estimate is to pass $Y(t)$ unchanged: $\check{h}(f) = P_x(f)/(P_x(f) + 0) = 1$ .

Quick Check

The Wiener filter for a narrowband signal in broadband noise has frequency response that is approximately:

A bandpass filter matching the signal bandwidth

An allpass filter

A notch filter at the signal frequency

Correction:

A bandpass filter matching the signal bandwidth

Where $P_x(f) \gg P_N(f)$ , $\check{h} \approx 1$ ; where $P_x(f) \ll P_N(f)$ , $\check{h} \approx 0$ . So the Wiener filter naturally becomes a bandpass filter matched to the signal's spectral support.

🔧Engineering Note

Wiener Filtering in Speech Enhancement

The Wiener filter is the workhorse of noise suppression in speech processing. In practice, the signal and noise PSDs are estimated from the data (e.g., using voice activity detection to identify noise-only frames). The short-time Fourier transform (STFT) provides a time-varying spectral estimate, and the Wiener filter is applied independently to each frequency bin. This simple approach — estimate $P_x(f)$ and $P_N(f)$ , then multiply by $P_x/({P_x + P_N})$ — remains competitive with far more complex algorithms.

Practical Constraints

•
PSD estimation errors cause 'musical noise' artifacts
•
Non-stationary noise requires adaptive PSD tracking

🎓CommIT Contribution(2018)

LMMSE Channel Estimation for Massive MIMO

G. Caire, K. Vu, R. Cavalcante — IEEE Transactions on Signal Processing

The Wiener filter principle extends directly to MIMO channel estimation. When the channel vector has spatial covariance $\mathbf{R}_h$ and the observations are corrupted by white noise of power $\sigma^2$ , the LMMSE estimator is $\hat{\mathbf{h}} = \mathbf{R}_h (\mathbf{R}_h + \sigma^2 \mathbf{I})^{-1} \mathbf{y}$ , which is the matrix analogue of the scalar Wiener filter $P_x/(P_x + P_N)$ . This work by the CommIT group showed how to exploit spatial covariance structure to dramatically improve channel estimation in massive MIMO systems, reducing pilot overhead and improving spectral efficiency.

LMMSEmassive-MIMOchannel-estimation

Wiener Filter

The LMMSE-optimal LTI filter for estimating a WSS signal from noisy observations. For signal-plus-noise with uncorrelated components, the frequency response is $\check{h}_{\text{opt}}(f) = P_x(f)/(P_x(f) + P_N(f))$ .

Related: Matched Filter

Key Takeaway

The non-causal Wiener filter $\check{h}_{\text{opt}}(f) = P_x(f)/(P_x(f) + P_N(f))$ is the LMMSE-optimal LTI filter for extracting a WSS signal from additive uncorrelated noise. It weights each frequency by the local signal-to-noise ratio, automatically adapting its bandwidth to the noise level. The MMSE is $\int P_x(f) P_N(f) / (P_x(f) + P_N(f))\, df$ .

The Wiener Filter

Estimating a Signal from Noisy Observations

Definition: The Wiener Filtering Problem

Theorem: The Wiener-Hopf Equation

Orthogonality principle

Substitute $ au = t - s$

Theorem: The Non-Causal Wiener Filter

Transform the Wiener-Hopf equation

Minimum MSE

Alternative form

Example: Wiener Filter for Signal in White Noise

Wiener filter

Limiting cases

MMSE

Example: Discrete-Time Wiener Filter for AR(1) Signal

Signal PSD

Noise PSD

Wiener filter

Wiener Filter Frequency Response vs. SNR

Parameters

Wiener Denoising: Before and After

Parameters

Matched Filter vs. Wiener Filter

Quick Check

Quick Check

Wiener Filtering in Speech Enhancement

LMMSE Channel Estimation for Massive MIMO

Wiener Filter

Key Takeaway

Definition:
The Wiener Filtering Problem