Ferkans — Interactive Telecom Tutor

From Continuous to Discrete: Why Sampling Matters

Every digital communication system works with discrete-time samples, yet the physical world is continuous. The sampling theorem tells us when — and how — we can recover a continuous-time signal from its samples without loss. The deterministic Nyquist-Shannon theorem is well known; the point is that the same result holds for random processes in the mean-square sense. A bandlimited WSS process can be perfectly reconstructed from samples taken at rate $2W$ , where $W$ is the one-sided bandwidth. This justifies the discrete-time models we use throughout communications theory.

,

Definition:
Bandlimited Random Process

A WSS process $X(t)$ with PSD $P_x(f)$ is bandlimited to $W$ Hz if $P_x(f) = 0 \quad \text{for all } |f| > W.$ The minimum such $W$ is the bandwidth of the process. The Nyquist rate is $2W$ samples per second, and the Nyquist interval is $T_s = 1/(2W)$ .

A bandlimited process is infinitely m.s.-differentiable, because $\int (2\pi f)^{2n} P_x(f)\, df$ is automatically finite when $P_x$ has compact support.

,

Bandlimited Process

A WSS process whose PSD vanishes outside $[-W, W]$ . Such processes are infinitely m.s.-differentiable and can be perfectly reconstructed from Nyquist-rate samples.

Related: Nyquist Rate

Nyquist Rate

The minimum sampling rate $f_s = 2W$ required to perfectly reconstruct a bandlimited process with bandwidth $W$ . Sampling below this rate causes aliasing.

Related: Bandlimited Process

Theorem: Sampling Theorem for WSS Processes

Let $X(t)$ be a zero-mean WSS process that is bandlimited to $W$ Hz, i.e., $P_x(f) = 0$ for $|f| > W$ . Set $T_s = 1/(2W)$ . Then $X(t) = \text{l.i.m.}_{N\to\infty} \sum_{n=-N}^{N} X(nT_s)\,\text{sinc}\!\left(\frac{t - nT_s}{T_s}\right),$ where the convergence is in the mean-square sense for every $t$ .

Moreover, the samples $\{X(nT_s)\}$ form a WSS discrete-time process with autocorrelation $r_{xx}[m] = r_{xx}(mT_s)$ and the reconstruction is exact — no information is lost.

A bandlimited PSD occupies a finite range of frequencies. Sampling at the Nyquist rate creates spectral copies that tile the frequency axis without overlap (no aliasing). The sinc interpolation filter selects exactly one copy, recovering the original PSD. The m.s. convergence follows because the reconstruction error has zero power.

Proof

Define the reconstruction

Define $\hat{X}_N(t) = \sum_{n=-N}^{N} X(nT_s)\,\text{sinc}\bigl(\frac{t - nT_s}{T_s}\bigr)$ . We need to show $\mathbb{E}[|X(t) - \hat{X}_N(t)|^2] \to 0$ as $N \to \infty$ .

Express the error in the spectral domain

By the Wiener-Khinchin theorem and Parseval's relation, $\mathbb{E}[|X(t) - \hat{X}_N(t)|^2] = \int_{-W}^{W} P_x(f)\left|1 - \sum_{n=-N}^{N} e^{-j2\pi f n T_s}\,\text{sinc}\!\left(\frac{t - nT_s}{T_s}\right)\right|^2 df.$ We use the fact that $\text{sinc}(t/T_s - n) = T_s \int_{-W}^{W} e^{j2\pi f(t - nT_s)}\, df$ (the sinc is the inverse Fourier transform of the ideal lowpass filter).

Show the reconstruction is exact for bandlimited signals

The sum $\sum_{n} e^{-j2\pi f n T_s}\,\text{sinc}\bigl(\frac{t - nT_s}{T_s}\bigr)$ converges to $e^{j2\pi ft}$ for $|f| \leq W$ (this is the Poisson summation formula applied to the sinc basis). Therefore the integrand vanishes for all $f$ in the support of $P_x$ , giving $\mathbb{E}[|X(t) - \hat{X}_N(t)|^2] \to 0$ .

Discrete-time autocorrelation

The samples $X_n \triangleq X(nT_s)$ have autocorrelation $\mathbb{E}[X_n X_m^*] = r_{xx}((n-m)T_s) = r_{xx}[n-m]$ . This is a valid discrete-time WSS autocorrelation sequence, and the discrete-time PSD is $P_x^{(d)}(e^{j\omega}) = \frac{1}{T_s}\sum_k P_x\bigl(\frac{\omega - 2\pi k}{2\pi T_s}\bigr)$ . Since $P_x$ is bandlimited, only the $k=0$ term contributes (no aliasing).

, ,

Example: Sampling an Ideal Lowpass Process

A WSS process has PSD $P_x(f) = N_0/(2W)$ for $|f| \leq W$ and $P_x(f) = 0$ otherwise (flat bandlimited). Compute the autocorrelation of the samples at the Nyquist rate.

Solution

Compute the continuous-time autocorrelation

$r_{xx}(\tau) = \int_{-W}^{W} \frac{N_0}{2W} e^{j2\pi f\tau}\, df = N_0\,\text{sinc}(2W\tau).$ $

Sample at $T_s = 1/(2W)$

$r_{xx}[m] = r_{xx}(m T_s) = N_0\,\text{sinc}(m) = \begin{cases} N_0 & m = 0, \\ 0 & m \neq 0. \end{cases}$ $

Interpret

The Nyquist-rate samples of a flat-bandlimited process are uncorrelated! Each sample carries independent information — exactly $2W$ independent "degrees of freedom" per second. For a Gaussian process, uncorrelated implies independent, so the samples are i.i.d. $\mathcal{N}(0, N_0)$ .

,

Sampling and Reconstruction of a Bandlimited Process

Observe how a bandlimited random process can be reconstructed from its Nyquist-rate samples using sinc interpolation. Adjust the oversampling factor to see how reconstruction quality changes, and observe aliasing when the sampling rate drops below Nyquist.

Parameters

W

(Hz)2

Oversampling factor (

f_s / 2W

)1

1.0 = Nyquist rate; < 1 = undersampling (aliasing); > 1 = oversampling

Number of samples20

Random seed42

⚠️Engineering Note

Practical Sampling: Anti-Aliasing and Oversampling

The sampling theorem assumes an ideal bandlimited PSD — but real signals are never perfectly bandlimited. In practice, an anti-aliasing filter (a lowpass filter before the ADC) attenuates frequencies above $W$ , and the system oversamples by a factor of 2--4x to allow a gradual filter rolloff. In 5G NR, the useful bandwidth is $0.9 \times f_s / 2$ (90% of Nyquist), with the remaining 10% serving as a guard band. The tradeoff is clear: sharper anti-aliasing filters are more expensive and introduce more group-delay distortion; oversampling relaxes the filter requirements at the cost of higher ADC rate and data throughput.

Practical Constraints

•
ADC resolution and sampling rate trade off (sigma-delta vs. flash ADC)
•
Anti-aliasing filter order vs. group delay distortion
•
In OFDM systems, the cyclic prefix provides inherent oversampling in the time domain

,

Why This Matters: Sampling Theorem in OFDM and Discrete-Time Channel Models

The sampling theorem is the bridge between continuous-time wireless channels and the discrete-time models used in receiver design. In OFDM, the transmitted signal is bandlimited to $N \Delta f$ Hz. The receiver samples at rate $f_s = N \Delta f$ (at or above Nyquist), then applies an $N$ -point DFT to recover the frequency-domain symbols. The entire OFDM transceiver chain — from IFFT at the transmitter to FFT at the receiver — is an engineered implementation of the sampling theorem for bandlimited processes. Without this theorem, we could not justify the discrete-time system model $\mathbf{y} = \mathbf{H}\mathbf{x} + \mathbf{w}$ that underpins all of MIMO-OFDM theory.

Common Mistake: Aliasing Corrupts the Autocorrelation, Not Just the PSD

Mistake:

Thinking aliasing only affects the frequency domain — that it folds the PSD but leaves the autocorrelation "approximately correct."

Correction:

When sampling below the Nyquist rate, the discrete-time autocorrelation becomes $r_{xx}^{(\text{aliased})}[m] = \sum_k r_{xx}(mT_s + kT_s/T_s')$ where the sum accounts for spectral folding. This is not equal to $r_{xx}(mT_s)$ in general. Aliasing introduces spurious correlations between samples that do not exist in the original process, corrupting any subsequent statistical analysis (PSD estimation, Wiener filtering, detection).

Historical Note: The Long History of the Sampling Theorem

1915-1957

The sampling theorem has many fathers. E. T. Whittaker published the cardinal interpolation formula in 1915, expressing an entire function as a series of sinc functions. Harry Nyquist (1928) established the relationship between bandwidth and signaling rate for telegraph channels. Claude Shannon (1949) proved the theorem in its modern form and recognized its fundamental importance for communication theory. In the Soviet Union, V. A. Kotelnikov independently proved the same result in 1933. The theorem is variously called the Whittaker-Shannon, Nyquist-Shannon, or WKS (Whittaker-Kotelnikov-Shannon) theorem. Its extension to random processes — which we prove here — was developed by Balakrishnan (1957) and Papoulis.

, ,

Degrees of Freedom of a Bandlimited Process

A bandlimited process on $[0, T]$ has approximately $2WT$ degrees of freedom. This is because $2WT$ Nyquist-rate samples suffice to represent the process (up to edge effects). This "dimensionality count" is fundamental:

In information theory, the capacity of a band-limited AWGN channel is $C = W \log_2(1 + \text{SNR})$ bits/s — each of the $2W$ samples per second carries $\frac{1}{2}\log_2(1+\text{SNR})$ bits.
In estimation theory, the number of resolvable parameters in a bandlimited observation is bounded by $2WT$ .
In the KL expansion (Section 3), the number of significant eigenvalues is approximately $2WT$ .

,

Sinc Interpolation

The reconstruction formula $X(t) = \sum_n X(nT_s)\,\text{sinc}(t/T_s - n)$ that recovers a bandlimited process from its Nyquist-rate samples. Also called the Whittaker-Shannon interpolation formula.

Quick Check

A bandlimited process with $W = 10$ kHz is sampled at $f_s = 30$ kHz (1.5x Nyquist). Are the resulting samples uncorrelated?

Not necessarily — oversampled samples are generally correlated.

Yes — oversampling always produces uncorrelated samples.

Yes — the process is bandlimited, so samples are always uncorrelated.