Ferkans — Interactive Telecom Tutor

Why Move to the Frequency Domain?

The autocorrelation $r_{xx}(\tau)$ tells us how a process is correlated across time. But many engineering questions are most natural in the frequency domain: How much power does this signal carry in the band $[f_1, f_2]$ ? The power spectral density answers this question directly. It decomposes the total power $r_{xx}(0)$ across frequency, just as Parseval's theorem decomposes energy. The Wiener-Khinchin theorem is the bridge: the PSD is simply the Fourier transform of the autocorrelation.

Definition:
Power Spectral Density (Continuous-Time)

Let $X(t)$ be a random process and define the truncated Fourier transform $\check{X}_T(f) = \int_{-T/2}^{T/2} X(t)\,e^{-j2\pi ft}\,dt$ . The power spectral density of $X(t)$ is

$P_x(f) = \lim_{T \to \infty} \frac{1}{T}\,\mathbb{E}\!\left[|\check{X}_T(f)|^2\right],$

provided the limit exists. The total average power is $\mathcal{P}_x = \int_{-\infty}^{\infty} P_x(f)\,df$ .

,

Operational Interpretation of the PSD

The PSD $P_x(f)$ has units of power per Hz. The power in any frequency band $[f_1, f_2]$ is $\int_{f_1}^{f_2} P_x(f)\,df$ . This is exactly what a spectrum analyzer measures: it passes the signal through a narrow bandpass filter centered at $f$ and reports the output power, which equals $P_x(f)\,\Delta f$ for bandwidth $\Delta f$ .

Theorem: Wiener-Khinchin Theorem (Continuous-Time)

Let $X(t)$ be a WSS process with absolutely integrable autocorrelation $r_{xx}(\tau)$ . Then the PSD exists and equals the Fourier transform of the autocorrelation:

$\boxed{P_x(f) = \int_{-\infty}^{\infty} r_{xx}(\tau)\,e^{-j2\pi f\tau}\,d\tau.}$

The inverse relation is

$r_{xx}(\tau) = \int_{-\infty}^{\infty} P_x(f)\,e^{j2\pi f\tau}\,df.$

The truncated periodogram $\frac{1}{T}|\check{X}_T(f)|^2$ is a convolution of the true Fourier spectrum with a Fejér-type window that converges to a delta as $T \to \infty$ . In expectation, this window smoothing disappears and the limit becomes exactly the Fourier transform of $r_{xx}(\tau)$ .

Proof

Expand the periodogram

Write

$\frac{1}{T}\mathbb{E}\!\left[|\check{X}_T(f)|^2\right] = \frac{1}{T}\int_{-T/2}^{T/2}\!\int_{-T/2}^{T/2} \mathbb{E}[X(t)X^*(s)]\,e^{-j2\pi f(t-s)}\,dt\,ds.$

Since $X(t)$ is WSS, $\mathbb{E}[X(t)X^*(s)] = r_{xx}(t-s)$ .

Change of variables

Let $\tau = t - s$ and $u = t + s$ . The Jacobian is $1/2$ and the integration region maps to $|\tau| \leq T$ , giving

$\frac{1}{T}\mathbb{E}\!\left[|\check{X}_T(f)|^2\right] = \int_{-T}^{T} r_{xx}(\tau)\,e^{-j2\pi f\tau} \left(1 - \frac{|\tau|}{T}\right) d\tau.$

The factor $(1 - |\tau|/T)$ is the Bartlett (triangular) window.

Take the limit

As $T \to \infty$ , the window $(1 - |\tau|/T) \to 1$ for every fixed $\tau$ . By the dominated convergence theorem (using absolute integrability of $r_{xx}(\tau)$ as the dominating function), we may pass the limit inside the integral:

$P_x(f) = \int_{-\infty}^{\infty} r_{xx}(\tau)\,e^{-j2\pi f\tau}\,d\tau. \qquad \square$

,

Example: PSD of a Process with Exponential Autocorrelation

A WSS process has autocorrelation $r_{xx}(\tau) = \sigma^2 e^{-\alpha|\tau|}$ with $\alpha > 0$ . Find $P_x(f)$ and verify $\int P_x(f)\,df = r_{xx}(0) = \sigma^2$ .

Solution

Compute the Fourier transform

$P_x(f) = \int_{-\infty}^{\infty} \sigma^2 e^{-\alpha|\tau|} e^{-j2\pi f\tau}\,d\tau = \frac{2\alpha\sigma^2}{\alpha^2 + (2\pi f)^2}.$ $

This is a Lorentzian (Cauchy-shaped) spectrum.

Verify the power integral

$\int_{-\infty}^{\infty} \frac{2\alpha\sigma^2}{\alpha^2 + (2\pi f)^2}\,df = \frac{2\alpha\sigma^2}{2\pi} \cdot \frac{\pi}{\alpha} = \sigma^2 = r_{xx}(0). \quad \checkmark$ $

Interpretation

As $\alpha \to 0$ the autocorrelation approaches $\sigma^2$ (constant, i.e., a DC process), and the PSD concentrates at $f = 0$ . As $\alpha \to \infty$ the process decorrelates rapidly and the PSD flattens — approaching white noise.

$P_x(f) \leftrightarrow r_{xx}(\tau)$ Duality

Vary the autocorrelation parameters and observe how the PSD changes. A narrow autocorrelation (fast decorrelation) maps to a wide PSD, and vice versa — the time-frequency uncertainty principle at work.

Parameters

Autocorrelation shape

\alpha

(decay rate)3

\sigma^2

(power)1

Theorem: Properties of the PSD

Let $P_x(f)$ be the PSD of a WSS process $X(t)$ . Then:

Non-negativity: $P_x(f) \geq 0$ for all $f$ .
Reality: $P_x(f) \in \mathbb{R}$ for all $f$ .
Symmetry for real processes: If $X(t)$ is real-valued, $P_x(f) = P_x(-f)$ .
Power: $r_{xx}(0) = \int_{-\infty}^{\infty} P_x(f)\,df = \mathcal{P}_x$ .

Non-negativity follows because the autocorrelation is positive semi-definite, and the Fourier transform of a PSD function is non-negative. Reality follows from the Hermitian symmetry of the autocorrelation: $r_{xx}(-\tau) = r_{xx}^*(\tau)$ .

Proof

Non-negativity

The autocorrelation $r_{xx}(\tau)$ is a positive semi-definite function: for any $g(t)$ with finite energy, $\int\!\int g(t)\,r_{xx}(t-s)\,g^*(s)\,dt\,ds \geq 0$ . Taking $g(t) = e^{j2\pi ft}$ over a finite interval and passing to the limit shows $P_x(f) \geq 0$ .

Reality and symmetry

Since $r_{xx}(-\tau) = r_{xx}^*(\tau)$ (Hermitian symmetry), the Fourier transform is real: $P_x(f)^* = P_x(f)$ . For real $X(t)$ , $r_{xx}(\tau)$ is real and even, so $P_x(f)$ is also real and even.

Power identity

Setting $\tau = 0$ in the inverse Fourier relation: $r_{xx}(0) = \int P_x(f)\,df$ . But $r_{xx}(0) = \mathbb{E}[|X(t)|^2] = \mathcal{P}_x$ .

,

$P_x(f) \leftrightarrow r_{xx}(\tau)$ Duality Animation

Watch how narrowing the autocorrelation in time broadens the PSD in frequency, illustrating the Fourier duality at the heart of the Wiener-Khinchin theorem.

As

\alpha

increases,

r_{xx}(\tau)

decays faster and

P_x(f)

broadens. The area under

P_x(f)

remains

r_{xx}(0) = \sigma^2

throughout.

Theorem: PSD of Non-WSS Processes

For a random process $X(t)$ with autocorrelation $r_{xx}(t_1, t_2)$ (not necessarily WSS), the PSD is

$P_x(f) = \int_{-\infty}^{\infty} \bar{r}_{xx}(\tau)\,e^{-j2\pi f\tau}\,d\tau,$

where the time-averaged autocorrelation is

$\bar{r}_{xx}(\tau) = \lim_{T \to \infty}\frac{1}{T}\int_{-T/2}^{T/2} r_{xx}(\tau + \theta,\, \theta)\,d\theta,$

provided the limit exists and is absolutely integrable.

Even when the autocorrelation depends on both time arguments, averaging over the absolute time $\theta$ extracts a function of the lag $\tau$ alone. The Wiener-Khinchin theorem then applies to this averaged autocorrelation. For WSS processes, $r_{xx}(\tau + \theta, \theta) = r_{xx}(\tau)$ regardless of $\theta$ , so the averaging is trivial.

Proof

Expand the truncated periodogram

As in Theorem 45, write $\frac{1}{T}\mathbb{E}[|\check{X}_T(f)|^2]$ and substitute $r_{xx}(t, s)$ for $\mathbb{E}[X(t)X^*(s)]$ .

Change variables and average

With $\tau = t - s$ and $\theta = s$ , the double integral becomes

$\int_{-T}^{T} \left[\frac{1}{T}\int_{\text{valid }\theta} r_{xx}(\tau+\theta, \theta)\,d\theta\right] e^{-j2\pi f\tau}\,d\tau.$

As $T \to \infty$ the inner average converges to $\bar{r}_{xx}(\tau)$ .

Conclude

Taking the limit gives $P_x(f) = \int \bar{r}_{xx}(\tau)\,e^{-j2\pi f\tau}\,d\tau$ .

Theorem: Corollary: PSD of Wide-Sense Cyclostationary Processes

If $X(t)$ is wide-sense cyclostationary with period $T_0$ , then

$\bar{r}_{xx}(\tau) = \frac{1}{T_0}\int_0^{T_0} r_{xx}(\tau + \theta,\, \theta)\,d\theta.$

The PSD is then $P_x(f) = \int \bar{r}_{xx}(\tau)\,e^{-j2\pi f\tau}\,d\tau$ .

For a cyclostationary process, the time average over one period suffices — there is no need to send $T \to \infty$ . This is the continuous-time analog of Corollary 7 in the discrete-time case.

Proof

Periodicity simplifies the limit

Since $r_{xx}(\tau + \theta, \theta)$ is periodic in $\theta$ with period $T_0$ , the Cesàro average $\frac{1}{T}\int_{-T/2}^{T/2}(\cdot)\,d\theta$ equals $\frac{1}{T_0}\int_0^{T_0}(\cdot)\,d\theta$ in the limit.

Example: PSD of a Random Sinusoid

Let $X(t) = A\cos(2\pi f_0 t + \Phi)$ where $A$ is a constant and $\Phi \sim \text{Uniform}[0, 2\pi)$ . Find $P_x(f)$ .

Solution

Autocorrelation

We showed in the previous chapter that $r_{xx}(\tau) = \frac{A^2}{2}\cos(2\pi f_0 \tau)$ .

Fourier transform

$P_x(f) = \mathcal{F}\!\left\{\frac{A^2}{2}\cos(2\pi f_0 \tau)\right\} = \frac{A^2}{4}\bigl[\delta(f - f_0) + \delta(f + f_0)\bigr].$ $

Interpretation

All the power is concentrated at $\pm f_0$ . The total power is $\int P_x(f)\,df = A^2/2 = r_{xx}(0)$ , as expected. This is a line spectrum — the hallmark of periodic or almost-periodic processes.

Example: PSD of a Sum of Random Sinusoids

Let $X(t) = \sum_{k=1}^{K} A_k \cos(2\pi f_k t + \Phi_k)$ where the $\Phi_k$ are i.i.d. $\text{Uniform}[0, 2\pi)$ and the $f_k$ are distinct. Find $P_x(f)$ .

Solution

Use linearity and independence

Since the phases are independent and uniformly distributed, cross terms in the autocorrelation vanish: $r_{xx}(\tau) = \sum_{k=1}^{K} \frac{A_k^2}{2}\cos(2\pi f_k \tau)$ .

Fourier transform

$P_x(f) = \sum_{k=1}^{K} \frac{A_k^2}{4}\bigl[\delta(f - f_k) + \delta(f + f_k)\bigr].$ $The PSD is a sum of spectral lines. The total power is$ \sum_{k} A_k^2/2$.

Quick Check

If a WSS process has PSD $P_x(f) = \frac{4}{1 + (2\pi f)^2}$ , what is the total average power $\mathcal{P}_x$ ?

$2$

$4$

$4\pi$

$1$

Correction:

2

$\mathcal{P}_x = r_{xx}(0) = \int_{-\infty}^{\infty} \frac{4}{1+(2\pi f)^2}\,df = \frac{4}{2\pi}\cdot\pi = 2$ .

Common Mistake: PSD vs. Energy Spectral Density

Mistake:

Computing $|\check{X}(f)|^2$ (without the $1/T$ normalization and expectation) and calling the result the PSD.

Correction:

For a power signal (infinite energy, finite power per unit time) the PSD is $\lim_{T\to\infty} \frac{1}{T}\mathbb{E}[|\check{X}_T(f)|^2]$ . The quantity $|\check{X}(f)|^2$ is the energy spectral density and applies to finite-energy (deterministic) signals. Confusing the two leads to infinite or zero answers.

Common Mistake: PSD Units

Mistake:

Writing $P_x(f)$ in "watts" or "volts squared."

Correction:

The PSD has units of power per Hz (e.g., W/Hz or V $^2$ /Hz). It is a density, so only its integral over a frequency band yields power.

Historical Note: Wiener and Khinchin

1930--1934

Norbert Wiener (1894--1964) introduced the concept of the power spectrum for stationary processes in his 1930 paper on generalized harmonic analysis. Independently, Aleksandr Khinchin (1894--1959) proved the equivalence between the power spectrum and the Fourier transform of the autocorrelation in 1934. Wiener's approach was more analytical (using generalized functions), while Khinchin's was more probabilistic. The theorem bearing both their names is one of the cornerstones of spectral analysis and communications engineering.

,

Power spectral density (PSD)

The Fourier transform of the autocorrelation of a WSS process. Describes the distribution of average power across frequency. Non-negative, real, and integrates to the total power.

Line spectrum

A PSD consisting of Dirac delta functions at discrete frequencies. Arises from periodic or almost-periodic processes.

Key Takeaway

The Wiener-Khinchin theorem is the fundamental bridge between the time and frequency descriptions of a WSS process: $P_x(f)$ and $r_{xx}(\tau)$ are a Fourier transform pair. Narrow correlation in time implies broad spectral content, and vice versa.

The Wiener-Khinchin Theorem

Why Move to the Frequency Domain?

Definition: Power Spectral Density (Continuous-Time)

Operational Interpretation of the PSD

Theorem: Wiener-Khinchin Theorem (Continuous-Time)

Expand the periodogram

Change of variables

Take the limit

Example: PSD of a Process with Exponential Autocorrelation

Compute the Fourier transform

Verify the power integral

Interpretation

Px(f)↔rxx(τ)P_x(f) \leftrightarrow r_{xx}(\tau)Px​(f)↔rxx​(τ) Duality

Parameters

Theorem: Properties of the PSD

Non-negativity

Reality and symmetry

Power identity

Px(f)↔rxx(τ)P_x(f) \leftrightarrow r_{xx}(\tau)Px​(f)↔rxx​(τ) Duality Animation

Theorem: PSD of Non-WSS Processes

Expand the truncated periodogram

Change variables and average

Conclude

Theorem: Corollary: PSD of Wide-Sense Cyclostationary Processes

Periodicity simplifies the limit

Example: PSD of a Random Sinusoid

Autocorrelation

Fourier transform

Interpretation

Example: PSD of a Sum of Random Sinusoids

Use linearity and independence

Fourier transform

Quick Check

Common Mistake: PSD vs. Energy Spectral Density

Common Mistake: PSD Units

Historical Note: Wiener and Khinchin

Power spectral density (PSD)

Line spectrum

Key Takeaway

Definition:
Power Spectral Density (Continuous-Time)

$P_x(f) \leftrightarrow r_{xx}(\tau)$ Duality

$P_x(f) \leftrightarrow r_{xx}(\tau)$ Duality Animation