Ferkans — Interactive Telecom Tutor

Inverting the Channel

The simplest equalization strategy is to pass the received signal through a linear filter whose transfer function approximates the inverse of the channel. Two design criteria dominate: zero-forcing (ZF), which eliminates ISI completely at the cost of amplifying noise, and minimum mean-square error (MMSE), which balances ISI suppression against noise enhancement. Understanding the trade-off between these two criteria is essential for practical equalizer design.

Definition:
Zero-Forcing (ZF) Equalizer

The zero-forcing equalizer is a linear filter with transfer function

$W_{\text{ZF}}(e^{j\omega}) = \frac{1}{H(e^{j\omega})}$

where $H(e^{j\omega})$ is the channel transfer function. In the time domain, this corresponds to a transversal (FIR) filter with $N_f$ taps $\{w_k\}$ chosen so that the cascade $h[n] * w[n]$ satisfies

$(h * w)[n] = \delta[n - d]$

where $d$ is a design delay. The ZF equalizer eliminates ISI completely but amplifies noise at frequencies where $|H(e^{j\omega})|$ is small.

In practice, the ZF equalizer is implemented as an FIR filter with a finite number of taps, so ISI is only approximately eliminated. The number of taps must be much larger than the channel memory $L$ for good ISI cancellation.

,

Definition:
Minimum Mean-Square Error (MMSE) Equalizer

The MMSE linear equalizer minimises the mean-square error between the equalizer output and the desired symbol:

$\mathbf{w}_{\text{MMSE}} = \arg\min_{\mathbf{w}} \; E\!\left[|a_{k-d} - \mathbf{w}^H \mathbf{y}_k|^2\right]$

where $\mathbf{y}_k = [y_k, y_{k-1}, \ldots, y_{k-N_f+1}]^T$ is the vector of received samples and $d$ is the decision delay.

The solution is given by the Wiener-Hopf equation:

$\mathbf{w}_{\text{MMSE}} = \mathbf{R}_{yy}^{-1}\, \mathbf{p}$

where $\mathbf{R}_{yy} = E[\mathbf{y}_k \mathbf{y}_k^H]$ is the autocorrelation matrix and $\mathbf{p} = E[\mathbf{y}_k\, a_{k-d}^*]$ is the cross-correlation vector.

In the frequency domain:

$W_{\text{MMSE}}(e^{j\omega}) = \frac{H^*(e^{j\omega})}{|H(e^{j\omega})|^2 + N_0 / E_s}$

The MMSE equalizer balances ISI suppression and noise enhancement by backing off from full channel inversion where $|H(e^{j\omega})|$ is small.

At high SNR ( $N_0/E_s \to 0$ ), the MMSE equalizer converges to the ZF equalizer. At low SNR, it approaches the matched filter $H^*(e^{j\omega})$ .

,

Theorem: Noise Enhancement of the ZF Equalizer

For a ZF equalizer operating on channel $H(e^{j\omega})$ with input noise PSD $N_0/2$ , the output noise variance is

$\sigma_{\text{ZF}}^2 = \frac{N_0}{2\pi} \int_{-\pi}^{\pi} \frac{1}{|H(e^{j\omega})|^2}\, d\omega$

This can be arbitrarily large if the channel has spectral nulls (frequencies where $|H(e^{j\omega})| \approx 0$ ). The noise enhancement factor is

$\xi_{\text{ZF}} = \frac{\sigma_{\text{ZF}}^2}{\sigma_\eta^2} = \frac{1}{2\pi} \int_{-\pi}^{\pi} \frac{1}{|H(e^{j\omega})|^2}\, d\omega$

where $\sigma_\eta^2$ is the input noise variance.

The ZF equalizer boosts all frequencies equally to flatten the overall response. At spectral nulls, this requires enormous gain, which amplifies noise without limit. This is the fundamental weakness of zero-forcing equalization.

Proof

Output noise PSD

The noise at the ZF equalizer output has PSD

$S_{\text{out}}(\omega) = \frac{N_0}{2} \cdot |W_{\text{ZF}}(e^{j\omega})|^2 = \frac{N_0}{2} \cdot \frac{1}{|H(e^{j\omega})|^2}$

Integration

The output noise variance is the integral of the PSD:

$\sigma_{\text{ZF}}^2 = \frac{1}{2\pi} \int_{-\pi}^{\pi} S_{\text{out}}(\omega)\, d\omega = \frac{N_0}{2\pi} \int_{-\pi}^{\pi} \frac{d\omega}{|H(e^{j\omega})|^2}$

If $H(e^{j\omega_0}) = 0$ for some $\omega_0$ , the integrand diverges and $\sigma_{\text{ZF}}^2 \to \infty$ . $\blacksquare$

Theorem: MMSE Equalizer Optimality

Among all linear equalizers of the form $\hat{a}_{k-d} = \mathbf{w}^H \mathbf{y}_k$ , the MMSE equalizer $\mathbf{w}_{\text{MMSE}} = \mathbf{R}_{yy}^{-1}\mathbf{p}$ achieves the minimum mean-square error

$J_{\text{MMSE}} = E_s - \mathbf{p}^H \mathbf{R}_{yy}^{-1} \mathbf{p}$

where $E_s = E[|a_k|^2]$ is the average symbol energy.

In the frequency domain, the MMSE is

$J_{\text{MMSE}} = \left[\frac{1}{2\pi} \int_{-\pi}^{\pi} \frac{d\omega}{|H(e^{j\omega})|^2 + N_0/E_s}\right]^{-1} \cdot \frac{N_0}{E_s}$

The MMSE formula reveals that the equalizer automatically trades off between suppressing ISI (making the output track $a_{k-d}$ ) and limiting noise enhancement. At spectral nulls, the MMSE equalizer does not attempt full inversion --- it accepts residual ISI rather than injecting infinite noise.

Proof

Orthogonality principle

The MMSE solution satisfies the orthogonality condition: the error $e_k = a_{k-d} - \mathbf{w}^H \mathbf{y}_k$ must be orthogonal to the data $\mathbf{y}_k$ :

$E[\mathbf{y}_k\, e_k^*] = \mathbf{0}$

Expanding: $E[\mathbf{y}_k (a_{k-d}^* - \mathbf{y}_k^H \mathbf{w})] = \mathbf{0}$

$\mathbf{p} - \mathbf{R}_{yy}\, \mathbf{w} = \mathbf{0} \implies \mathbf{w} = \mathbf{R}_{yy}^{-1}\, \mathbf{p}$

Minimum MSE value

$J_{\text{MMSE}} = E[|a_{k-d}|^2] - \mathbf{w}^H E[\mathbf{y}_k a_{k-d}^*] = E_s - \mathbf{p}^H \mathbf{R}_{yy}^{-1}\, \mathbf{p}$ $This is always less than or equal to the MSE of any other linear filter.$ \blacksquare$

,

ZF vs. MMSE Equalizer Performance

Compare BER performance of zero-forcing and MMSE linear equalizers on a multipath channel. Observe how the ZF equalizer suffers from noise enhancement, especially at high SNR when the channel has deep fades.

Parameters

Channel tap

h_1

0.5

Channel tap

h_2

0

Equalizer length

N_f

11

Max SNR (dB)30

Equalizer Frequency Response

View the frequency response of the channel and the ZF/MMSE equalizer. Notice how the MMSE equalizer avoids the large gain peaks at spectral nulls that plague the ZF equalizer.

Parameters

Channel tap

h_1

0.5

Channel tap

h_2

-0.3

SNR (dB)15

Example: ZF Equalizer for a Two-Tap Channel

A channel has impulse response $h[0] = 1,\; h[1] = 0.6$ . Design a 3-tap ZF equalizer $\mathbf{w} = [w_0, w_1, w_2]^T$ with decision delay $d = 1$ . Compute the noise enhancement factor.

Solution

Convolution constraint

We need $(h * w)[n] = \delta[n - 1]$ , which gives:

$\begin{bmatrix} h[0] & 0 & 0 \\ h[1] & h[0] & 0 \\ 0 & h[1] & h[0] \\ 0 & 0 & h[1] \end{bmatrix} \begin{bmatrix} w_0 \\ w_1 \\ w_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \\ 0 \\ 0 \end{bmatrix}$

Substituting $h[0] = 1,\; h[1] = 0.6$ :

$\begin{bmatrix} 1 & 0 & 0 \\ 0.6 & 1 & 0 \\ 0 & 0.6 & 1 \\ 0 & 0 & 0.6 \end{bmatrix} \begin{bmatrix} w_0 \\ w_1 \\ w_2 \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \\ 0 \\ 0 \end{bmatrix}$

Solve via least squares

This is overdetermined; we solve $\mathbf{H}^{H} \mathbf{H}\, \mathbf{w} = \mathbf{H}^{H} \mathbf{d}$ .

From the first row: $w_0 = 0$ .

From the second row: $0.6(0) + w_1 = 1 \implies w_1 = 1$ .

From the third row: $0.6(1) + w_2 = 0 \implies w_2 = -0.6$ .

Check fourth row: $0.6(-0.6) = -0.36 \neq 0$ .

With only 3 taps, the ZF condition is not perfectly met. The least-squares solution gives approximately $\mathbf{w} \approx [0,\; 0.735,\; -0.441]^T$ .

Noise enhancement

The noise enhancement factor is $\xi = \|\mathbf{w}\|^2 = |w_0|^2 + |w_1|^2 + |w_2|^2$ .

For the approximate solution: $\xi \approx 0 + 0.735^2 + 0.441^2 \approx 0.540 + 0.194 = 0.734$ .

With an infinite-length ZF equalizer, $\xi = \frac{1}{2\pi} \int_{-\pi}^{\pi} \frac{d\omega}{|1 + 0.6 e^{-j\omega}|^2} = \frac{1}{1 - 0.6^2} = 1.5625$ .

The 3-tap filter has less noise enhancement because it does not fully eliminate ISI. $\blacksquare$

Example: MMSE Equalizer via Wiener-Hopf

For the channel $h = [1,\; 0.5]$ with $E_s/N_0 = 10$ dB, compute the 3-tap MMSE equalizer with delay $d = 1$ .

Solution

Build the autocorrelation matrix

The channel convolution matrix for 3-tap output is

$\mathbf{H} = \begin{bmatrix} 1 & 0 \\ 0.5 & 1 \\ 0 & 0.5 \end{bmatrix}$

With $\sigma_\eta^2 = N_0 = E_s / 10$ , the autocorrelation matrix is

$\mathbf{R}_{yy} = E_s\, \mathbf{H}\mathbf{H}^{H} + N_0\, \mathbf{I}$

$= E_s \begin{bmatrix} 1 & 0.5 & 0 \\ 0.5 & 1.25 & 0.5 \\ 0 & 0.5 & 0.25 \end{bmatrix} + \frac{E_s}{10} \mathbf{I}$

$= E_s \begin{bmatrix} 1.1 & 0.5 & 0 \\ 0.5 & 1.35 & 0.5 \\ 0 & 0.5 & 0.35 \end{bmatrix}$

Cross-correlation vector

The cross-correlation vector for delay $d = 1$ picks out the second column of $\mathbf{H}$ :

$\mathbf{p} = E_s\, \mathbf{h}_1 = E_s \begin{bmatrix} 0 \\ 1 \\ 0.5 \end{bmatrix}$

Solve the Wiener-Hopf equation

$\mathbf{w}_{\text{MMSE}} = \mathbf{R}_{yy}^{-1}\, \mathbf{p}$ $Solving numerically:$ \mathbf{w}_{\text{MMSE}} \approx [-0.296,; 0.858,; 0.340]^T $The MMSE is$ J = E_s - \mathbf{p}^H \mathbf{w} \approx 0.127, E_s $. The residual MSE is about 12.7% of symbol energy.$ \blacksquare$

Quick Check

What happens to the MMSE linear equalizer as $\text{SNR} \to \infty$ ?

It converges to the matched filter $H^*(e^{j\omega})$

It converges to the zero-forcing equalizer $1/H(e^{j\omega})$

It converges to a constant gain filter

It diverges and becomes unstable

Correction:

It converges to the zero-forcing equalizer

1/H(e^{j\omega})

Correct. As $N_0/E_s \to 0$ , the MMSE frequency response $H^*/(|H|^2 + N_0/E_s) \to H^*/|H|^2 = 1/H$ , which is the ZF equalizer.

Common Mistake: ZF Equalizer at Spectral Nulls

Mistake:

Applying a ZF equalizer to a channel with spectral nulls (e.g., $H(e^{j\pi}) = 0$ ) and expecting good performance at moderate SNR.

Correction:

At spectral nulls, $|H(e^{j\omega})| = 0$ and the ZF gain $1/|H(e^{j\omega})|^2 \to \infty$ . The resulting noise enhancement can make the ZF equalizer worse than no equalization at all. Always use the MMSE criterion when the channel has deep nulls, or switch to a DFE or MLSE receiver.

ZF vs. MMSE Linear Equalizer

Property	Zero-Forcing (ZF)	MMSE
Design criterion	Eliminate all ISI	Minimise MSE = ISI + noise
Frequency response	$1 / H(e^{j\omega})$	$H^* / (\|H\|^2 + N_0/E_s)$
Noise enhancement	Severe at spectral nulls	Controlled; backs off at nulls
Residual ISI	Zero (infinite taps)	Small but nonzero
High-SNR limit	Optimal among linear EQs	Converges to ZF
Low-SNR limit	Poor (noise dominated)	Converges to matched filter
Complexity	$O(N_f)$ per symbol	$O(N_f)$ per symbol

Why This Matters: Equalization in the Spatial Domain

In MIMO systems (Chapters 15–17), the received signal at each antenna is a superposition of signals from all transmit antennas — creating inter-stream interference analogous to ISI. The same ZF, MMSE, and ML detection principles apply:

ZF receiver: $\mathbf{W} = (\mathbf{H}^{H}\mathbf{H})^{-1}\mathbf{H}^{H}$ — eliminates inter-stream interference but amplifies noise
MMSE receiver: $\mathbf{W} = (\mathbf{H}^{H}\mathbf{H} + \sigma^2\mathbf{I})^{-1}\mathbf{H}^{H}$ — same noise-ISI trade-off as the scalar MMSE equalizer
ML detection: searches over $M^{n_t}$ constellation points — same exponential complexity as MLSE

The Book MIMO (Chapters 3–5) develops these spatial equalizers in full detail.

See full treatment in MIMO Receivers

Why This Matters: OFDM as Frequency-Domain Equalization

OFDM (Orthogonal Frequency-Division Multiplexing) can be viewed as the ultimate frequency-domain equalizer. By converting a frequency-selective channel into $N$ parallel flat-fading subchannels, OFDM reduces equalization to a simple per-subcarrier scalar division --- equivalent to a single-tap ZF or MMSE equalizer on each subcarrier. This eliminates the need for complex time-domain equalization and is the reason OFDM dominates modern wireless standards (4G LTE, 5G NR, Wi-Fi).

See full treatment in Chapter 14

Zero-Forcing Equalizer

A linear equalizer designed to completely eliminate ISI by inverting the channel transfer function. Suffers from noise enhancement at spectral nulls.

Related: MMSE Equalizer, Equalization

MMSE Equalizer

A linear equalizer that minimises the mean-square error between its output and the desired symbol, balancing ISI suppression against noise enhancement.

⚠️Engineering Note

Equalizer Complexity in Real-Time Systems

In practical implementations, equalizer complexity is measured in multiply-accumulate (MAC) operations per symbol:

Linear FIR equalizer: $N_f$ complex MACs per symbol. For $N_f = 11$ at 30.72 MHz (5G NR 20 MHz BW), this is $\sim 338$ million MACs/s --- easily handled by modern DSPs.
Matrix inversion for MMSE: Computing $\mathbf{R}_{yy}^{-1}$ is $O(N_f^3)$ , but it only needs to be updated when the channel changes (once per coherence time). Levinson-Durbin exploits Toeplitz structure to reduce this to $O(N_f^2)$ .
Fixed-point implementation: 16-bit fixed-point arithmetic is sufficient for equalizers with $N_f \leq 20$ . Longer equalizers may need 32-bit accumulators to avoid numerical instability, particularly for the ZF design.
OFDM alternative: In OFDM systems, equalization reduces to $N$ complex divisions (one per subcarrier), completely avoiding the matrix inversion. This is a key reason for OFDM's dominance in modern standards.

Practical Constraints

•
FIR equalizer: $N_f$ complex MACs per symbol; Levinson-Durbin: $O(N_f^2)$ for Toeplitz inversion
•
16-bit fixed-point sufficient for $N_f \leq 20$ ; longer equalizers need 32-bit accumulators
•
OFDM reduces equalization to $N$ scalar divisions, avoiding matrix operations entirely

Key Takeaway

The MMSE equalizer is almost always preferred over ZF in practice. It automatically interpolates between matched filtering (at low SNR) and channel inversion (at high SNR), avoiding catastrophic noise enhancement at spectral nulls. The only additional information it requires is the noise variance $N_0$ .

Noise Enhancement

The amplification of noise caused by a ZF equalizer at frequencies where the channel gain is small. Quantified by the ratio of output noise power to input noise power.

Related: Zero-Forcing Equalizer

Linear Equalizers

Inverting the Channel

Definition: Zero-Forcing (ZF) Equalizer

Definition: Minimum Mean-Square Error (MMSE) Equalizer

Theorem: Noise Enhancement of the ZF Equalizer

Output noise PSD

Integration

Theorem: MMSE Equalizer Optimality

Orthogonality principle

Minimum MSE value

ZF vs. MMSE Equalizer Performance

Parameters

Equalizer Frequency Response

Parameters

Example: ZF Equalizer for a Two-Tap Channel

Convolution constraint

Solve via least squares

Noise enhancement

Example: MMSE Equalizer via Wiener-Hopf

Build the autocorrelation matrix

Cross-correlation vector

Solve the Wiener-Hopf equation

Quick Check

Common Mistake: ZF Equalizer at Spectral Nulls

ZF vs. MMSE Linear Equalizer

Why This Matters: Equalization in the Spatial Domain

Why This Matters: OFDM as Frequency-Domain Equalization

Zero-Forcing Equalizer

MMSE Equalizer

Equalizer Complexity in Real-Time Systems

Key Takeaway

Noise Enhancement

Definition:
Zero-Forcing (ZF) Equalizer

Definition:
Minimum Mean-Square Error (MMSE) Equalizer