Ferkans — Interactive Telecom Tutor

The Discrete-Time Setting

In practice, we always work with sampled data. The discrete-time PSD is the workhorse of digital signal processing and communications: given a sequence of samples, how is the power distributed across the normalized frequency range $[-\frac{1}{2}, \frac{1}{2}]$ ? The Wiener-Khinchin theorem carries over to discrete time — the PSD is the DTFT of the autocorrelation — and the periodogram provides a finite-data estimate that connects theory to measurement.

Definition:
Power Spectral Density (Discrete-Time)

Let $X_n$ be a random process. Define the truncated DFT $\check{X}_N(f) = \sum_{n=-N}^{N} X_n\,e^{-j2\pi fn}$ . The power spectral density is

$P_x(f) = \lim_{N \to \infty} \frac{1}{2N+1}\,\mathbb{E}\!\left[|\check{X}_N(f)|^2\right], \qquad f \in [-\tfrac{1}{2},\tfrac{1}{2}].$

The total average power is $\mathcal{P}_x = \int_{-1/2}^{1/2} P_x(f)\,df$ .

Theorem: Wiener-Khinchin Theorem (Discrete-Time)

Let $X_n$ be a WSS sequence with absolutely summable autocorrelation $r_{xx}[m]$ , i.e., $\sum_{m=-\infty}^{\infty} |r_{xx}[m]| < \infty$ . Then

$\boxed{P_x(f) = \sum_{m=-\infty}^{\infty} r_{xx}[m]\,e^{-j2\pi fm}, \qquad f \in [-\tfrac{1}{2}, \tfrac{1}{2}].}$

The inverse relation is

$r_{xx}[m] = \int_{-1/2}^{1/2} P_x(f)\,e^{j2\pi fm}\,df.$

The proof follows the same pattern as the CT case: expand the periodogram in terms of the autocorrelation, and the Fejér-type window $(1 - |m|/(2N+1))$ converges to $1$ as $N \to \infty$ under absolute summability.

Proof

Expand the periodogram

$\frac{1}{2N+1}\mathbb{E}\!\left[|\check{X}_N(f)|^2\right] = \sum_{\ell=-(2N)}^{2N} r_{xx}[\ell]\left(1 - \frac{|\ell|}{2N+1}\right) e^{-j2\pi f\ell}.$ $This uses the WSS property:$ \mathbb{E}[X_n X_m^*] = r_{xx}[n-m] $. For each lag$ \ell = n - m $, there are exactly$ (2N+1 - |\ell|) $pairs$ (n,m) \in {-N,\ldots,N}^2 $with$ n - m = \ell$.

Take the limit

As $N \to \infty$ , $(1 - |\ell|/(2N+1)) \to 1$ for each fixed $\ell$ . By absolute summability of $r_{xx}[\ell]$ and dominated convergence:

$P_x(f) = \sum_{\ell=-\infty}^{\infty} r_{xx}[\ell]\,e^{-j2\pi f\ell}. \qquad \square$

,

Theorem: Properties of the Discrete-Time PSD

The DT PSD satisfies:

Periodic: $P_x(f + 1) = P_x(f)$ (period 1 in $f$ ).
Non-negative: $P_x(f) \geq 0$ for all $f$ .
Real: $P_x(f) \in \mathbb{R}$ .
Even for real processes: $P_x(f) = P_x(-f)$ .
Power: $r_{xx}[0] = \int_{-1/2}^{1/2} P_x(f)\,df$ .

Periodicity comes from the DTFT. The remaining properties mirror the CT case and follow from the positive semi-definiteness and Hermitian symmetry of $r_{xx}[m]$ .

Proof

Periodicity

$e^{-j2\pi(f+1)m} = e^{-j2\pi fm}\cdot e^{-j2\pi m} = e^{-j2\pi fm}$ since $m \in \mathbb{Z}$ . So $P_x(f+1) = P_x(f)$ .

Non-negativity and reality

Same argument as the CT case: $r_{xx}[m]$ is positive semi-definite, so its DTFT is non-negative and real.

Theorem: PSD of Non-WSS Processes (Discrete-Time)

For a DT process with autocorrelation $r_{xx}[n, m]$ (not necessarily WSS):

$P_x(f) = \sum_{m=-\infty}^{\infty} \bar{r}_{xx}[m]\,e^{-j2\pi fm},$

where $\bar{r}_{xx}[m] = \lim_{N \to \infty}\frac{1}{2N+1}\sum_{n=-N}^{N} r_{xx}[n, n-m]$ is the time-averaged autocorrelation.

This is the DT version of Theorem 43. Even when the autocorrelation depends on both time indices, time-averaging extracts a function of the lag alone, to which the Wiener-Khinchin theorem applies.

Proof

Expand and average

Write $\frac{1}{2N+1}\mathbb{E}[|\check{X}_N(f)|^2]$ using $r_{xx}[n,m]$ and change the summation variable to the lag $\ell = n - m$ . The coefficient of $e^{-j2\pi f\ell}$ is $\frac{1}{2N+1}\sum_{n} r_{xx}[n, n-\ell] \cdot (\text{window})$ . As $N \to \infty$ this converges to $\bar{r}_{xx}[\ell]$ .

Theorem: Corollary: PSD of WSC Processes (Discrete-Time)

If $X_n$ is wide-sense cyclostationary with period $T$ , then

$\bar{r}_{xx}[m] = \frac{1}{T}\sum_{n=0}^{T-1} r_{xx}[n, n-m].$

The periodicity allows replacing the infinite Cesàro average with a finite average over one period.

Proof

Finite average suffices

Since $r_{xx}[n, n-m]$ is periodic in $n$ with period $T$ , the Cesàro limit reduces to the average over one period: $\frac{1}{T}\sum_{n=0}^{T-1} r_{xx}[n, n-m]$ .

Example: PSD of an AR(1) Process

Consider the AR(1) process $X_n = a X_{n-1} + W_n$ where $|a| < 1$ and $W_n$ is white noise with variance $\sigma^2$ . Find $P_x(f)$ .

Solution

Autocorrelation

We know $r_{xx}[m] = \frac{\sigma^2}{1 - a^2}\,a^{|m|}$ .

DTFT

$P_x(f) = \sum_{m=-\infty}^{\infty} \frac{\sigma^2}{1-a^2}\,a^{|m|}\,e^{-j2\pi fm} = \frac{\sigma^2}{1 - 2a\cos(2\pi f) + a^2}.$ $

Verification

Alternatively, $X_n$ is the output of the filter $\check{h}(f) = 1/(1 - ae^{-j2\pi f})$ driven by white noise $W_n$ . So $P_x(f) = |\check{h}(f)|^2\,\sigma^2 = \sigma^2/|1 - ae^{-j2\pi f}|^2$ , which matches.

Definition:
The Periodogram

Given $N$ samples $X_0, X_1, \ldots, X_{N-1}$ of a WSS process, the periodogram is the estimator

$\hat{P_x}(f) = \frac{1}{N}\left|\sum_{n=0}^{N-1} X_n\,e^{-j2\pi fn}\right|^2.$

The periodogram is a natural finite-data approximation to the PSD: it replaces the infinite sum and the expectation in the definition with a finite sum and a single realization.

The periodogram is an asymptotically unbiased estimator of $P_x(f)$ , but it is not consistent: its variance does not decrease as $N \to \infty$ . Averaging multiple periodograms (Bartlett's method) or windowing (Welch's method) is needed for consistent estimation.

,

Periodogram of a WSS Sequence

Generate an AR(1) process and compare the periodogram (single realization) with the true PSD. Observe how the periodogram is noisy and how averaging (Bartlett's method) reduces variance.

Parameters

a

(AR coefficient)0.8

N

(samples)256

Bartlett segments1

Random seed42

DT PSD $\leftrightarrow$ Autocorrelation

Explore the DTFT relationship between $r_{xx}[m]$ and $P_x(f)$ for different process types.

Parameters

Process type

Process parameter

a

0.7

Example: PSD of an MA(1) Process

Let $X_n = W_n + b W_{n-1}$ where $W_n$ is white noise with variance $\sigma^2$ . Find $P_x(f)$ .

Solution

Autocorrelation

$r_{xx}[0] = \sigma^2(1 + b^2)$ , $r_{xx}[\pm 1] = \sigma^2\,b$ , $r_{xx}[m] = 0$ for $|m| \geq 2$ .

DTFT

$P_x(f) = \sigma^2(1 + b^2) + 2\sigma^2\,b\cos(2\pi f) = \sigma^2\,|1 + b\,e^{-j2\pi f}|^2.$ $

Interpretation

This is also $|\check{h}(f)|^2\,\sigma^2$ where $\check{h}(f) = 1 + be^{-j2\pi f}$ is the MA filter. For $b > 0$ the PSD peaks at $f = 0$ (low-pass); for $b < 0$ it peaks at $f = 1/2$ (high-pass).

Quick Check

The periodogram $\hat{P_x}(f)$ computed from $N$ samples of a WSS process is:

Asymptotically unbiased but inconsistent

Consistent and unbiased

Biased and consistent

Neither unbiased nor consistent

Correction:

Asymptotically unbiased but inconsistent

$\mathbb{E}[\hat{P_x}(f)] \to P_x(f)$ as $N \to \infty$ , but $\text{Var}[\hat{P_x}(f)]$ does not vanish. The variance stays approximately $P_x(f)^2$ regardless of $N$ .

Common Mistake: Trusting a Single Periodogram

Mistake:

Using the raw periodogram from a single data record as if it were the true PSD.

Correction:

The periodogram has high variance (approximately equal to $P_x(f)^2$ ) regardless of data length. To reduce variance, use averaging methods: Bartlett (segment-average), Welch (overlapping windowed segments), or multitaper methods. The bias-variance tradeoff is controlled by the number of segments.

🔧Engineering Note

Welch's Method in Practice

Welch's method (1967) splits the data into overlapping segments, windows each segment, computes the periodogram of each, and averages. With $K$ segments and 50% overlap, the variance is reduced by approximately a factor of $9K/11$ compared to the raw periodogram, at the cost of frequency resolution. This is the de facto standard PSD estimator in most signal processing libraries (e.g., scipy.signal.welch in Python, pwelch in MATLAB).

Historical Note: Schuster and the Periodogram

1898

The periodogram was introduced by Arthur Schuster in 1898 for detecting hidden periodicities in meteorological and geophysical data. The term "periodogram" reflects its original purpose: finding periodic components. Its statistical properties as a PSD estimator were only understood much later, when it was recognized that the periodogram is an inconsistent estimator — a surprising and initially disappointing result that motivated the development of averaged and windowed spectral estimators.

DT vs. CT PSD Comparison

Property	Continuous-Time	Discrete-Time
PSD formula	$P_x(f) = \int r_{xx}(\tau)\,e^{-j2\pi f\tau}\,d\tau$	$P_x(f) = \sum_m r_{xx}[m]\,e^{-j2\pi fm}$
Frequency range	$f \in (-\infty, \infty)$	$f \in [-1/2, 1/2]$ (periodic)
Power	$\int_{-\infty}^{\infty} P_x(f)\,df$	$\int_{-1/2}^{1/2} P_x(f)\,df$
White noise PSD	$N_0/2$ (infinite power)	$\sigma^2$ (finite power)
Periodogram	$\frac{1}{T}\|\check{X}_T(f)\|^2$	$\frac{1}{N}\|\sum_n X_n e^{-j2\pi fn}\|^2$

Periodogram

The estimator $\hat{P_x}(f) = \frac{1}{N}|\sum_n X_n e^{-j2\pi fn}|^2$ . Asymptotically unbiased but inconsistent (variance does not decrease with $N$ ).

Related: {{Ref:Gloss Psd}}

Autocorrelation (WSS)

$r_{xx}[m] = \mathbb{E}[X_n X_{n-m}^*]$ (DT) or $r_{xx}(\tau) = \mathbb{E}[X(t+\tau)X^*(t)]$ (CT). Depends only on the lag for WSS processes. Forms a Fourier pair with the PSD.

Key Takeaway

The DT Wiener-Khinchin theorem mirrors the CT version: $P_x(f)$ is the DTFT of $r_{xx}[m]$ , periodic in $f$ with period 1. The periodogram is the natural finite-data estimator but requires averaging (Bartlett, Welch) for reliable spectral estimation.

Discrete-Time PSD

The Discrete-Time Setting

Definition: Power Spectral Density (Discrete-Time)

Theorem: Wiener-Khinchin Theorem (Discrete-Time)

Expand the periodogram

Take the limit

Theorem: Properties of the Discrete-Time PSD

Periodicity

Non-negativity and reality

Theorem: PSD of Non-WSS Processes (Discrete-Time)

Expand and average

Theorem: Corollary: PSD of WSC Processes (Discrete-Time)

Finite average suffices

Example: PSD of an AR(1) Process

Autocorrelation

DTFT

Verification

Definition: The Periodogram

Periodogram of a WSS Sequence

Parameters

DT PSD ↔\leftrightarrow↔ Autocorrelation

Parameters

Example: PSD of an MA(1) Process

Autocorrelation

DTFT

Interpretation

Quick Check

Common Mistake: Trusting a Single Periodogram

Welch's Method in Practice

Historical Note: Schuster and the Periodogram

DT vs. CT PSD Comparison

Periodogram

Autocorrelation (WSS)

Key Takeaway

Definition:
Power Spectral Density (Discrete-Time)

Definition:
The Periodogram

DT PSD $\leftrightarrow$ Autocorrelation