Ferkans — Interactive Telecom Tutor

Why Settle for Linear?

MLSE is optimal but expensive. For channels with long delay spread, the trellis becomes infeasibly large and we need a cheaper decoder. The simplest compromise is to give up on sequence estimation entirely and treat each symbol independently, using a linear filter to approximate the channel inverse. We saw the relevant tool in Chapter 9: the Wiener filter, which minimizes mean-squared error subject to a linearity constraint. Apply it here and two named equalizers emerge — the zero-forcing (ZF) equalizer, which forces the ISI to zero regardless of noise, and the minimum mean-squared-error (MMSE) equalizer, which trades ISI for noise reduction. The analytical machinery is entirely from Chapter 9; what is new is the operational question of when linear processing is good enough.

Definition:
Linear Equalizer

A linear equalizer is an LTI filter with frequency response $W(f)$ (or impulse response $\{w[k]\}$ ) applied to the received sequence $y[n]$ to produce a scalar decision statistic:

$z[n] \;=\; \sum_{k} w[k]\, y[n-k].$

The output $z[n]$ is then quantized to the nearest point of $\mathcal{A}$ to produce a symbol-by-symbol decision $\hat{x}[n] = \arg\min_{a \in \mathcal{A}} |z[n] - a|^2$ .

Linear equalizers are characterized by (i) their cost criterion (ZF, MMSE, least-squares), (ii) their implementation domain (frequency or time), and (iii) whether they operate at the symbol rate ( $T$ -spaced) or fractionally ( $T/2$ -spaced).

Definition:
Zero-Forcing Equalizer

The zero-forcing (ZF) equalizer is the linear filter whose frequency response is the inverse of the channel response:

$W_{\text{ZF}}(f) \;=\; \frac{1}{H(f)}, \qquad H(f) \neq 0.$

Its defining property: in the absence of noise, the cascade $H(f) W_{\text{ZF}}(f) = 1$ reproduces the transmitted symbol exactly — ISI is "forced to zero" at every frequency.

The ZF equalizer is ill-defined at channel nulls, and nearly so at near-nulls. The analytical pathology becomes operational pathology — noise is amplified by $|W_{\text{ZF}}(f)|^2 = 1/|H(f)|^2$ at each frequency, which diverges as $H(f) \to 0$ . This is the noise enhancement problem that dominates the rest of the section.

Definition:
MMSE Equalizer

The minimum mean-squared-error (MMSE) equalizer is the Wiener filter that minimizes $\mathbb{E}[|z[n] - x[n]|^2]$ . Assuming i.i.d. unit-variance symbols and white Gaussian noise of PSD $N_0$ , its frequency response is

$W_{\text{MMSE}}(f) \;=\; \frac{H^{*}(f)}{|H(f)|^2 + N_0}.$

At high SNR ( $N_0 \to 0$ ) the MMSE filter approaches the ZF filter. At low SNR, the regularization term $N_0$ dominates and the filter acts as a scaled matched filter $H^{*}(f)$ .

Equivalently in the time domain, $\mathbf{w}_{\text{MMSE}} = (\mathbf{H}^{H} \mathbf{H} + N_0 \mathbf{I})^{-1} \mathbf{H}^{H}\mathbf{e}_{\Delta}$ , where $\mathbf{H}$ is a convolution (Toeplitz) matrix and $\mathbf{e}_{\Delta}$ targets the symbol at delay $\Delta$ .

Theorem: MMSE Linear Equalizer via the Wiener Filter

Let $\{x[n]\}$ be a wide-sense-stationary sequence with zero mean and autocorrelation $r_{xx}[k] = \delta[k]$ , passed through the LTI ISI channel of DDiscrete-Time ISI Channel with frequency response $H(f)$ , corrupted by white complex Gaussian noise of PSD $N_0$ . The symbol-rate linear filter $W(f)$ minimizing $\mathbb{E}[|z[n] - x[n]|^2]$ is

$W_{\text{MMSE}}(f) \;=\; \frac{H^{*}(f)}{|H(f)|^2 + N_0}.$

The resulting minimum mean-squared error is

$\text{MSE}_{\min} \;=\; \int_{-1/2}^{1/2} \frac{N_0}{|H(f)|^2 + N_0}\, df.$

The Wiener filter is the orthogonal projection of $x[n]$ onto the subspace spanned by past and current observations. In the frequency domain this projection decouples per-frequency, so the optimal filter is the pointwise ratio of cross-PSD to observation PSD — exactly the MMSE expression.

Proof

Setup via orthogonality

By the orthogonality principle, the MMSE filter is characterized by $\mathbb{E}[(z[n] - x[n]) y^{*}[n-k]] = 0$ for all $k$ . Writing $z[n] = \sum_{\ell} w[\ell] y[n-\ell]$ and using joint wide-sense-stationarity gives the Wiener–Hopf equations

$r_{xy}[k] \;=\; \sum_{\ell} w[\ell]\, r_{yy}[k - \ell], \qquad k \in \mathbb{Z}.$

Compute the relevant correlations

From $y[n] = \sum_j h[j] x[n-j] + w[n]$ with unit-variance i.i.d. symbols and noise of PSD $N_0$ :

$r_{xy}[k] = \mathbb{E}[x[n] y^{*}[n-k]] = h^{*}[-k],$ $r_{yy}[k] = (h * h^{\dagger})[k] + N_0\, \delta[k],$

where $h^{\dagger}[k] = h^{*}[-k]$ .

Transform to frequency domain

Taking DTFTs, the convolution in Wiener–Hopf becomes a product:

$R_{xy}(f) = W(f)\, R_{yy}(f).$

Compute each spectrum:

$R_{xy}(f) = H^{*}(f), \qquad R_{yy}(f) = |H(f)|^2 + N_0.$

Solve for $W(f)$

Dividing gives the stated result:

$W_{\text{MMSE}}(f) = \frac{R_{xy}(f)}{R_{yy}(f)} = \frac{H^{*}(f)}{|H(f)|^2 + N_0}.$

Minimum MSE

Plugging $W_{\text{MMSE}}$ back into $\mathbb{E}[|z[n] - x[n]|^2]$ and using Parseval's theorem,

$\text{MSE}_{\min} = \int_{-1/2}^{1/2} \left(1 - \frac{|H(f)|^2}{|H(f)|^2 + N_0}\right) df = \int_{-1/2}^{1/2} \frac{N_0}{|H(f)|^2 + N_0}\, df. \;\blacksquare$

Theorem: Output SNR of the ZF Equalizer

Let the channel have frequency response $H(f)$ with $|H(f)|$ bounded away from zero on $[-1/2, 1/2]$ , and let the noise be white Gaussian with PSD $N_0$ . The output SNR of the $T$ -spaced zero-forcing equalizer is

$\text{SNR}_{\text{ZF}}^{\text{out}} \;=\; \frac{1}{N_0} \cdot \left( \int_{-1/2}^{1/2} \frac{df}{|H(f)|^2} \right)^{-1} \;\leq\; \frac{1}{N_0} \cdot \left( \int_{-1/2}^{1/2} |H(f)|^2\, df \right).$

The inequality is strict unless $|H(f)|$ is constant; the gap quantifies the loss relative to the matched-filter bound.

The ZF equalizer outputs $z[n] = x[n] + \tilde{w}[n]$ , where $\tilde{w}[n]$ has PSD $N_0/|H(f)|^2$ . Its total variance is the integral of that PSD. Where $|H(f)|$ is small, $1/|H(f)|^2$ blows up — noise is amplified. The Cauchy–Schwarz upper bound compares the arithmetic and harmonic means of $|H(f)|^2$ and shows that ZF is always at least as noisy as a system with flat channel of the same energy.

Proof

Output noise PSD

After filtering, $z[n] = x[n] + \tilde{w}[n]$ where $\tilde{w}[n]$ is the convolution of $w[n]$ with $w_{\text{ZF}}[k]$ . The noise output PSD is

$P_{\tilde{w}}(f) = N_0 \cdot |W_{\text{ZF}}(f)|^2 = \frac{N_0}{|H(f)|^2}.$

Output noise power

Integrating over one period,

$\text{Var}(\tilde{w}[n]) = \int_{-1/2}^{1/2} \frac{N_0}{|H(f)|^2}\, df = N_0 \int_{-1/2}^{1/2} \frac{df}{|H(f)|^2}.$

Output SNR

With unit-variance symbols, the output SNR is the ratio of signal power (unity) to noise power:

$\text{SNR}_{\text{ZF}}^{\text{out}} = \frac{1}{N_0 \int |H(f)|^{-2}\, df}.$

Comparison to matched-filter bound via Cauchy–Schwarz

By Cauchy–Schwarz applied to $|H(f)|$ and $1/|H(f)|$ ,

$\left(\int |H(f)|^2\, df\right) \left(\int |H(f)|^{-2}\, df\right) \;\geq\; \left(\int 1\, df\right)^2 = 1.$

Rearranging,

$\text{SNR}_{\text{ZF}}^{\text{out}} = \frac{1}{N_0 \int |H(f)|^{-2}\, df} \;\leq\; \frac{1}{N_0} \int |H(f)|^2\, df.$

Equality holds iff $|H(f)|$ is constant, i.e. the channel is all-pass in magnitude. $\blacksquare$

Key Takeaway

ZF makes the ISI vanish but inflates the noise by a factor equal to the integral of $1/|H(f)|^2$ . This factor blows up for channels with spectral nulls. MMSE regularizes that inverse and always outperforms ZF — strictly so, unless the noise is negligible.

Example: ZF and MMSE for a Two-Tap Channel

A normalized two-tap channel has taps $\mathbf{h} = [1, \alpha]^T / \sqrt{1+\alpha^2}$ with $\alpha = 0.8$ . The noise PSD is $N_0 = 0.1$ (input SNR $= 10$ dB). Derive the ZF and MMSE frequency responses, evaluate the noise enhancement at DC and at the first Nyquist null, and compute the minimum MSE achieved by the MMSE equalizer.

Solution

Channel frequency response

With $c = 1/\sqrt{1+\alpha^2} = 1/\sqrt{1.64} \approx 0.781$ ,

$H(f) = c\,(1 + \alpha\, e^{-j2\pi f}), \qquad |H(f)|^2 = c^2\, (1 + \alpha^2 + 2\alpha\cos(2\pi f)).$

At $f = 0$ : $|H(0)|^2 = c^2 (1 + \alpha)^2 = 0.781^2 \cdot 1.8^2 = 1.977$ . At $f = 1/2$ : $|H(1/2)|^2 = c^2(1 - \alpha)^2 = 0.781^2 \cdot 0.04 = 0.0244$ .

ZF frequency response and noise amplification

$|W_{\text{ZF}}(f)|^2 = \frac{1}{|H(f)|^2}.$ $At$ f = 0 $:$ |W_{\text{ZF}}|^2 = 1/1.977 = 0.506 $. At$ f = 1/2 $:$ |W_{\text{ZF}}|^2 = 1/0.0244 = 41.0 $. The ZF filter amplifies noise by a factor of about$ 16 \text{ dB}$ at the band edge — the deepest channel dip.

MMSE frequency response

$W_{\text{MMSE}}(f) = \frac{H^{*}(f)}{|H(f)|^2 + 0.1}.$ $At$ f = 0 $:$ |W_{\text{MMSE}}|^2 = 1.977 / 2.077^2 = 0.458 $. At$ f = 1/2 $:$ |W_{\text{MMSE}}|^2 = 0.0244 / 0.124^2 = 1.585 $. The MMSE filter refuses to amplify beyond about$ 2 \text{ dB} $at the band edge — the regularizer$ N_0$ caps the inversion.

Minimum MSE

With $\beta \equiv 2\alpha c^2 = 2 \cdot 0.8 \cdot 0.781^2 = 0.976$ and $\mu \equiv c^2(1+\alpha^2) + 0.1 = 1.0 + 0.1 = 1.1$ ,

$\text{MSE}_{\min} = \int_{-1/2}^{1/2} \frac{0.1}{\mu + \beta\cos(2\pi f)}\, df = \frac{0.1}{\sqrt{\mu^2 - \beta^2}}.$

Numerically $\mu^2 - \beta^2 = 1.21 - 0.953 = 0.257$ so $\text{MSE}_{\min} = 0.1/\sqrt{0.257} \approx 0.197$ . The output SINR of the MMSE equalizer is therefore $(1 - 0.197)/0.197 \approx 4.08$ , i.e. about $6.1 \text{ dB}$ — a loss of roughly $4 \text{ dB}$ relative to the AWGN baseline due to ISI.

ZF and MMSE Frequency Responses

Compare $|H(f)|$ , $|W_{\text{ZF}}(f)|$ , and $|W_{\text{MMSE}}(f)|$ for several canonical ISI channels. Note how the ZF curve rises at frequencies where the channel attenuates, while the MMSE curve flattens.

Parameters

Channel preset

SNR (dB)15

BER vs SNR for ZF, MMSE, MMSE-DFE, and the MLSE Bound

BPSK bit-error rate as a function of SNR for the four equalizer families. The matched-filter bound is a lower bound on MLSE and a useful reference point. Observe the ordering: $\text{ZF} \succ \text{MMSE} \succ \text{MMSE-DFE} \succ \text{MLSE bound} \succ \text{AWGN}$ (where $\succ$ denotes "higher BER").

Parameters

Channel preset

SNR min (dB)0

SNR max (dB)20

ZF Noise Enhancement at a Spectral Null

Animation showing how the ZF equalizer amplifies the noise PSD at a channel null, while MMSE gracefully caps the amplification.

As

\alpha \to 1

in

h = [1, -\alpha]

, the channel develops a null at

f = 0

. The ZF filter's magnitude diverges there; the MMSE filter saturates at

1/N_0

.

⚠️Engineering Note

Per-Subcarrier MMSE Is the Engine of OFDM

OFDM converts a wideband ISI channel into $N$ parallel flat-fading subchannels by inserting a cyclic prefix and performing an $N$ -point DFT at the receiver. On the $k$ -th subcarrier the received sample is $Y_k = H_k X_k + W_k$ with $H_k$ a scalar — the ISI has been diagonalized. The optimal per-subcarrier equalizer is then simply $W_k = H_k^{*} / (|H_k|^2 + N_0)$ , the scalar MMSE equalizer. This is why OFDM's receiver is so much simpler than a single-carrier equalizer: the wideband MMSE frequency response is literally evaluated on a grid of subcarriers. The price of this simplification is the cyclic-prefix overhead and the peak-to-average power ratio of the transmitted waveform.

📋 Ref: IEEE 802.11a/g/n/ac/ax, 3GPP LTE/NR

Common Mistake: Never Use ZF When $|H(f)|$ Has Nulls

Mistake:

Engineers newly introduced to channel inversion sometimes implement the ZF equalizer because "it removes the ISI exactly." For typical frequency-selective wireless channels, the resulting noise enhancement makes ZF uncompetitive — often by more than 6 dB.

Correction:

Use MMSE unless you have a specific reason not to. The MMSE filter costs one extra addition per subcarrier (the $N_0$ term in the denominator) and is always at least as good as ZF. At asymptotically high SNR they coincide; at low-to-moderate SNR, MMSE dominates.

Common Mistake: Input SNR vs Output SNR — Know Which You Are Plotting

Mistake:

When comparing equalizers on a BER-vs-SNR curve, students sometimes normalize the x-axis by the output SNR of the equalizer rather than the input SNR of the channel. This hides the equalizer's performance — every linear equalizer trivially achieves $Q(\sqrt{2 \cdot \text{output SNR}})$ .

Correction:

Always plot BER as a function of input SNR, i.e. $E_s / N_0$ at the receiver's front end. The equalizer's loss relative to AWGN is then made visible as a horizontal shift of the BER curve.

Quick Check

As the noise PSD $N_0 \to 0$ , the MMSE equalizer's frequency response approaches which of the following?

The matched filter $H^{*}(f)$

The ZF filter $1/H(f)$

A constant (frequency-flat) response

Zero

Correction:

The ZF filter

1/H(f)

At $\N_0 = 0$ , $W_{\\text{MMSE}}(f) = H^{*}(f)/|H(f)|^2 = 1/H(f) = W_{\\text{ZF}}(f)$ . At asymptotically high SNR the two filters coincide — which is precisely why ZF's noise-enhancement penalty does not disappear at high SNR unless the channel is flat.

Quick Check

For a flat channel $H(f) \equiv 1$ and noise PSD $N_0$ , the MMSE linear equalizer's minimum MSE is:

$N_0$

$N_0 / (1 + N_0)$

$1$

$0$

Correction:

N_0 / (1 + N_0)

With $|H(f)|^2 \\equiv 1$ the MSE integrand becomes $\N_0/(1 + \N_0)$ , a constant, so the integral over $[-1/2, 1/2]$ is the same constant. At high SNR this is $\\approx \N_0$ ; at low SNR it saturates near $1$ .

Block Diagram: Linear Equalizer Receiver — ISI channel, AWGN, linear equalizer $W(f)$ , and symbol-by-symbol decision device. The equalizer's job is to make $z[n]$ as close to $x[n]$ as possible in mean-squared-error sense.

Why This Matters: OFDM Pilot-Assisted Equalization

In LTE and 5G NR, pilot symbols scattered through the time–frequency grid give the receiver per-subcarrier channel estimates $\hat{H}_k$ . Data symbols on each subcarrier are then equalized by $\hat{W}_k = \hat{H}_k^{*} / (|\hat{H}_k|^2 + \hat{N_0})$ — a direct implementation of the MMSE formula derived here. The equalizer's cost per received data symbol is a single complex multiplication; this scaling to massive-bandwidth systems is impossible with a time-domain single-carrier MLSE receiver.

Zero-forcing (ZF) equalizer

The linear filter $W_{\text{ZF}}(f) = 1/H(f)$ that inverts the channel response. Makes the noise-free cascade equal to the identity but amplifies noise at frequencies where $|H(f)|$ is small.

Related: MMSE equalizer

MMSE equalizer

The linear filter $W_{\text{MMSE}}(f) = H^{*}(f)/(|H(f)|^2 + N_0)$ that minimizes mean-squared error between equalized output and transmitted symbol. A regularized channel inverse; reduces to ZF at high SNR and to a matched filter at low SNR.

Related: Zero-forcing (ZF) equalizer

Linear Equalizers — ZF and MMSE

Why Settle for Linear?

Definition: Linear Equalizer

Definition: Zero-Forcing Equalizer

Definition: MMSE Equalizer

Theorem: MMSE Linear Equalizer via the Wiener Filter

Setup via orthogonality

Compute the relevant correlations

Transform to frequency domain

Solve for $W(f)$

Minimum MSE

Theorem: Output SNR of the ZF Equalizer

Output noise PSD

Output noise power

Output SNR

Comparison to matched-filter bound via Cauchy–Schwarz

Key Takeaway

Example: ZF and MMSE for a Two-Tap Channel

Channel frequency response

ZF frequency response and noise amplification

MMSE frequency response

Minimum MSE

ZF and MMSE Frequency Responses

Parameters

BER vs SNR for ZF, MMSE, MMSE-DFE, and the MLSE Bound

Parameters

ZF Noise Enhancement at a Spectral Null

Per-Subcarrier MMSE Is the Engine of OFDM

Common Mistake: Never Use ZF When ∣H(f)∣|H(f)|∣H(f)∣ Has Nulls

Common Mistake: Input SNR vs Output SNR — Know Which You Are Plotting

Quick Check

Quick Check

Block Diagram: Linear Equalizer Receiver

Why This Matters: OFDM Pilot-Assisted Equalization

Zero-forcing (ZF) equalizer

MMSE equalizer

Definition:
Linear Equalizer

Definition:
Zero-Forcing Equalizer

Definition:
MMSE Equalizer

Common Mistake: Never Use ZF When $|H(f)|$ Has Nulls