Ferkans — Interactive Telecom Tutor

From Random Variables to Random Functions

In Parts I--III we studied random variables: a single measurement, or a finite collection of measurements. But the signals we encounter in communications — thermal noise at a receiver, a fading channel gain, a transmitted waveform — are functions of time, each realization producing an entire waveform. A stochastic process is the mathematical object that captures this: a random variable that takes values not in $\mathbb{R}$ or $\mathbb{R}^n$ , but in a space of functions. The challenge, and the beauty, is that we can still characterize such objects through their finite-dimensional projections.

Definition:
Stochastic Process (Definition 48)

Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space. A stochastic process is a family of random variables $\{X(t) : t \in \mathcal{T}\}$ indexed by a set $\mathcal{T}$ , where each $X(t)$ is an $\mathcal{F}$ -measurable mapping $X(t) : \Omega \to \mathbb{K}$ with $\mathbb{K} = \mathbb{R}$ or $\mathbb{C}$ .

Equivalently, a stochastic process is a function of two variables: $X : \mathcal{T} \times \Omega \to \mathbb{K}, \quad (t, \omega) \mapsto X(t, \omega).$

For fixed $t$ , $X(t, \cdot)$ is a random variable. For fixed $\omega$ , $X(\cdot, \omega)$ is a deterministic function called a sample path or realization.

Definition:
Classification by Index Set and State Space

A stochastic process $\{X(t) : t \in \mathcal{T}\}$ is classified by:

Index set $\mathcal{T}$ :
- Continuous-time (CT): $\mathcal{T} \subseteq \mathbb{R}$ (e.g., $\mathcal{T} = [0, \infty)$ or $\mathbb{R}$ ).
- Discrete-time (DT): $\mathcal{T} \subseteq \mathbb{Z}$ (e.g., $\mathcal{T} = \mathbb{Z}$ or $\{0, 1, 2, \ldots\}$ ). We write $X_n$ instead of $X(n)$ .
State space $\mathbb{K}$ :
- Continuous-valued: $\mathbb{K} = \mathbb{R}$ or $\mathbb{C}$ .
- Discrete-valued: $\mathbb{K}$ is countable (e.g., $\{0, 1\}$ ).

The Four Classes of Stochastic Processes

Combining the two classifications yields four types:

	Continuous-valued	Discrete-valued
CT	Thermal noise $N(t)$	Poisson counting process $N(t)$
DT	Sampled fading $H_n$	Binary sequence $X_n \in \{0,1\}$

In communications, we most frequently encounter CT continuous-valued processes (noise, channel fading) and DT continuous-valued processes (sampled signals, symbol sequences after modulation).

Definition:
Sample Path (Realization)

For a fixed outcome $\omega \in \Omega$ , the function $t \mapsto X(t, \omega)$ is called a sample path, realization, or trajectory of the process. The set of all sample paths is the ensemble of the process.

A single oscilloscope trace of receiver noise is one sample path. The statistical properties of the process describe the behavior across the entire ensemble of possible traces.

Sample Path Realizations

Visualize sample paths of different stochastic process types: i.i.d. sequence, moving average, and random sinusoid. Each "Generate" changes the random seed.

Parameters

Process Type

Number of Paths5

Random Seed42

Definition:
Finite-Dimensional Distributions (fdds)

The finite-dimensional distributions (fdds) of a process $\{X(t) : t \in \mathcal{T}\}$ are the collection of all joint CDFs $F_{t_1, \ldots, t_N}(x_1, \ldots, x_N) = \mathbb{P}(X(t_1) \leq x_1, \ldots, X(t_N) \leq x_N)$ for every $N \geq 1$ and every choice of time indices $t_1, \ldots, t_N \in \mathcal{T}$ .

The fdds completely characterize the probabilistic behavior of the process. In principle, knowing all fdds tells us everything about the process — but in practice, we rarely have access to distributions of order higher than two.

Theorem: Kolmogorov Consistency Conditions

A collection of joint CDFs $\{F_{t_1, \ldots, t_N}\}$ are the fdds of some stochastic process if and only if they satisfy:

Marginalization: For any permutation $\sigma$ of $\{1, \ldots, N\}$ , $F_{t_{\sigma(1)}, \ldots, t_{\sigma(N)}}(x_{\sigma(1)}, \ldots, x_{\sigma(N)}) = F_{t_1, \ldots, t_N}(x_1, \ldots, x_N).$
Compatibility: $\lim_{x_N \to \infty} F_{t_1, \ldots, t_N}(x_1, \ldots, x_N) = F_{t_1, \ldots, t_{N-1}}(x_1, \ldots, x_{N-1}).$

The first condition says that relabeling the time indices just relabels the arguments. The second says that "forgetting" the last time point by letting $x_N \to \infty$ gives back the lower-dimensional distribution. These are the minimal consistency requirements for a coherent probabilistic description.

Proof

Necessity

If the fdds come from a process on $(\Omega, \mathcal{F}, \mathbb{P})$ , both conditions follow directly from the properties of joint CDFs.

Sufficiency (Kolmogorov Extension Theorem)

This is the deep direction. Given consistent fdds, Kolmogorov's extension theorem guarantees the existence of a probability space and a process with those fdds. The proof constructs the process on the product space $\mathbb{K}^{\mathcal{T}}$ using the Daniell-Kolmogorov extension. We omit the full measure-theoretic construction, which requires the theory of projective limits.

Example: The Random Sinusoid

Let $X(t) = A \cos(2\pi \nu_0 t + \Theta)$ , where $A$ is a positive random variable and $\Theta \sim \text{Uniform}[0, 2\pi)$ is independent of $A$ . Find the mean function and the autocorrelation function of $X(t)$ .

Solution

Mean function

$\mu_X(t) = \mathbb{E}[A \cos(2\pi \nu_0 t + \Theta)] = \mathbb{E}[A] \cdot \mathbb{E}[\cos(2\pi \nu_0 t + \Theta)].KATEXPLACEHOLDER0END\mathbb{E}[\cos(2\pi \nu_0 t + \Theta)] = \frac{1}{2\pi}\int_0^{2\pi} \cos(2\pi \nu_0 t + \theta)\,d\theta = 0.$ $Therefore$ \mu_X(t) = 0 $for all$ t$.

Autocorrelation

$r_{XX}(t_1, t_2) = \mathbb{E}[A^2 \cos(2\pi \nu_0 t_1 + \Theta)\cos(2\pi \nu_0 t_2 + \Theta)].KATEXPLACEHOLDER0ENDr_{XX}(t_1, t_2) = \frac{\mathbb{E}[A^2]}{2}\left[\cos(2\pi \nu_0(t_1 - t_2)) + \mathbb{E}[\cos(2\pi \nu_0(t_1 + t_2) + 2\Theta)]\right].KATEXPLACEHOLDER1ENDr_{XX}(t_1, t_2) = \frac{\mathbb{E}[A^2]}{2}\cos(2\pi \nu_0 (t_1 - t_2)).$ $This depends only on$ \tau = t_1 - t_2 $, so$ r_{XX}(\tau) = \frac{\mathbb{E}[A^2]}{2}\cos(2\pi \nu_0 \tau)$.

Interpretation

The random sinusoid with uniform phase has zero mean and an autocorrelation that depends only on the time difference $\tau$ . As we will see in §13.2, this makes it a wide-sense stationary process. The average power is $r_{XX}(0) = \mathbb{E}[A^2]/2$ .

Example: i.i.d. Processes

Let $\{X_n : n \in \mathbb{Z}\}$ be an i.i.d. sequence with $\mathbb{E}[X_n] = \mu$ and $\mathbb{E}[X_n^2] = \sigma^2 + \mu^2$ . Find the autocorrelation $r_{xx}[n, m]$ .

Solution

Diagonal vs. off-diagonal

For $n = m$ : $r_{xx}[n, n] = \mathbb{E}[X_n^2] = \sigma^2 + \mu^2$ .

For $n \neq m$ : since $X_n$ and $X_m$ are independent, $r_{xx}[n, m] = \mathbb{E}[X_n]\mathbb{E}[X_m] = \mu^2$ .

Compact form

$r_{xx}[n, m] = \sigma^2 \delta[n - m] + \mu^2,$ $where$ \delta[k] $is the Kronecker delta. This depends only on$ n - m$, so the i.i.d. process is WSS (in fact, it is strictly stationary by construction).

Definition:
Second-Order Process

A process $\{X(t) : t \in \mathcal{T}\}$ is called a second-order process if $\mathbb{E}[|X(t)|^2] < \infty$ for all $t \in \mathcal{T}$ .

Second-order processes are the natural setting for correlation analysis. The autocorrelation $r_{XX}(t_1, t_2) = \mathbb{E}[X(t_1)X^*(t_2)]$ is well-defined whenever $\mathbb{E}[|X(t)|^2] < \infty$ , by the Cauchy-Schwarz inequality.

Quick Check

A Poisson counting process $N(t)$ , where $t \geq 0$ and $N(t) \in \{0, 1, 2, \ldots\}$ , is classified as:

Continuous-time, continuous-valued

Continuous-time, discrete-valued

Discrete-time, continuous-valued

Discrete-time, discrete-valued

Correction:

Continuous-time, discrete-valued

The index $t \geq 0$ is continuous, and $N(t)$ takes values in $\{0, 1, 2, \ldots\}$ .

Quick Check

For the random sinusoid $X(t) = A\cos(2\pi \nu_0 t + \Theta)$ with random $A$ and $\Theta$ , a sample path is obtained by:

Fixing $t$ and varying $\omega$

Fixing $\omega$ and varying $t$

Averaging over all $\omega$

Correction:

Fixing

\omega

and varying

t

A sample path is a deterministic function of $t$ for a fixed realization $(A(\\omega), \\Theta(\\omega))$ .

Historical Note: Kolmogorov and the Foundations of Stochastic Processes

1930s

Andrey Kolmogorov's 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of Probability Theory") placed probability on rigorous measure-theoretic ground and, as a byproduct, provided the extension theorem that guarantees the existence of stochastic processes from consistent finite-dimensional distributions. This resolved a fundamental question: does a "random function" actually exist as a mathematical object? Kolmogorov showed that it does, provided the fdds are consistent. His framework remains the standard foundation for all of modern probability and stochastic process theory.

Stochastic Process

A family of random variables $\{X(t) : t \in \mathcal{T}\}$ indexed by a parameter set $\mathcal{T}$ , typically representing time.

Sample Path

A single realization of a stochastic process, obtained by fixing the outcome $\omega$ and viewing the process as a deterministic function of time.

Related: Stochastic Process

Finite-Dimensional Distributions (fdds)

The collection of all joint distributions $F_{t_1, \ldots, t_N}$ of the process evaluated at finitely many time indices. The fdds completely characterize the process.

Related: Stochastic Process

Common Mistake: A Process Is Not Just a Sequence of Random Variables

Mistake:

Treating a stochastic process as merely a collection of independent random variables, ignoring the dependence structure across time.

Correction:

The joint behavior across time indices — captured by the fdds — is the essence of a stochastic process. A process with independent time samples (i.i.d.) is a very special case. In general, $X(t_1)$ and $X(t_2)$ are correlated, and this correlation is what makes the process useful for modeling physical phenomena like fading channels.

Why This Matters: The Wireless Channel as a Stochastic Process

A wireless channel gain $H(t)$ between a transmitter and receiver is a stochastic process: at each time $t$ , the gain is a complex random variable whose realization depends on the scattering environment and the relative motion of transmitter and receiver. A single "channel trace" measured with a channel sounder is one sample path. The statistical description of $H(t)$ — through its fdds and, more practically, its autocorrelation function — determines the coherence time and shapes the design of pilot spacing, coding, and feedback protocols.

See full treatment in Chapter 14

Key Takeaway

A stochastic process $\{X(t) : t \in \mathcal{T}\}$ is a random function of time, completely characterized by its finite-dimensional distributions. For engineering purposes, we almost always work with the first- and second-order statistics (mean and autocorrelation), which motivates the WSS framework developed in the following sections.

Definition and Classification

From Random Variables to Random Functions

Definition: Stochastic Process (Definition 48)

Definition: Classification by Index Set and State Space

The Four Classes of Stochastic Processes

Definition: Sample Path (Realization)

Sample Path Realizations

Parameters

Definition: Finite-Dimensional Distributions (fdds)

Theorem: Kolmogorov Consistency Conditions

Necessity

Sufficiency (Kolmogorov Extension Theorem)

Example: The Random Sinusoid

Mean function

Autocorrelation

Interpretation

Example: i.i.d. Processes

Diagonal vs. off-diagonal

Compact form

Definition: Second-Order Process

Quick Check

Quick Check

Historical Note: Kolmogorov and the Foundations of Stochastic Processes

Stochastic Process

Sample Path

Finite-Dimensional Distributions (fdds)

Common Mistake: A Process Is Not Just a Sequence of Random Variables

Why This Matters: The Wireless Channel as a Stochastic Process

Key Takeaway

Definition:
Stochastic Process (Definition 48)

Definition:
Classification by Index Set and State Space

Definition:
Sample Path (Realization)

Definition:
Finite-Dimensional Distributions (fdds)

Definition:
Second-Order Process