Ferkans — Interactive Telecom Tutor

The Three Pillars of a Probability Space

In Chapter 1 we introduced the probability space $(\Omega, \mathcal{F}, P)$ informally: $\Omega$ is the set of outcomes, $\mathcal{F}$ is the collection of events, and $P$ assigns probabilities. Now we make this precise. The key insight is that $\mathcal{F}$ cannot be "all subsets of $\Omega$ " when $\Omega$ is uncountable — doing so leads to contradictions (Vitali sets, Banach-Tarski). The sigma-algebra is the mathematical device that tells us which subsets are "measurable" and therefore eligible to receive a probability.

Definition:
Sigma-Algebra ( $\sigma$ -Algebra)

A collection $\mathcal{F}$ of subsets of $\Omega$ is a $\sigma$ -algebra (or $\sigma$ -field) if:

$\Omega \in \mathcal{F}$ .
If $A \in \mathcal{F}$ , then $A^c \in \mathcal{F}$ (closed under complementation).
If $A_1, A_2, \ldots \in \mathcal{F}$ , then $\bigcup_{n=1}^{\infty} A_n \in \mathcal{F}$ (closed under countable unions).

Properties 2 and 3 together imply closure under countable intersections (by De Morgan). Also, $\emptyset = \Omega^c \in \mathcal{F}$ . The pair $(\Omega, \mathcal{F})$ is called a measurable space.

Example: Examples of Sigma-Algebras

Identify the sigma-algebras in the following cases:

(a) $\Omega = \{H, T\}$ (coin flip). (b) $\Omega = \mathbb{R}$ , the Borel sigma-algebra. (c) The trivial and the discrete sigma-algebras on any $\Omega$ .

Solution

(a) Coin flip

$\mathcal{F} = \{\emptyset, \{H\}, \{T\}, \{H,T\}\} = 2^{\Omega}$ . When $\Omega$ is finite or countable, the power set is a valid sigma-algebra.

(b) Borel sigma-algebra

$\mathcal{B}(\mathbb{R})$ is the smallest sigma-algebra containing all open intervals $(a, b)$ . Equivalently, it is generated by all sets of the form $(-\infty, x]$ for $x \in \mathbb{R}$ . It contains all open sets, all closed sets, all countable unions and intersections thereof (the Borel hierarchy). It does not contain all subsets of $\mathbb{R}$ — Vitali's construction shows that non-measurable sets exist (assuming the axiom of choice).

(c) Trivial and discrete

The trivial sigma-algebra is $\mathcal{F} = \{\emptyset, \Omega\}$ — it contains no information about $\omega$ . The discrete sigma-algebra is $\mathcal{F} = 2^{\Omega}$ — it distinguishes every element. Every sigma-algebra on $\Omega$ sits between these two extremes.

,

Definition:
Generated Sigma-Algebra

Given any collection $\mathcal{C}$ of subsets of $\Omega$ , the sigma-algebra generated by $\mathcal{C}$ , written $\sigma(\mathcal{C})$ , is the smallest sigma-algebra containing $\mathcal{C}$ : $\sigma(\mathcal{C}) = \bigcap \{ \mathcal{F} : \mathcal{F} \text{ is a } \sigma\text{-algebra and } \mathcal{C} \subseteq \mathcal{F} \}.$ This intersection is well-defined because $2^{\Omega}$ is always a sigma-algebra containing $\mathcal{C}$ .

The Borel sigma-algebra is $\mathcal{B}(\mathbb{R}) = \sigma(\{(a,b) : a < b\})$ . Equivalently, $\mathcal{B}(\mathbb{R}) = \sigma(\{(-\infty, x] : x \in \mathbb{R}\})$ .

Definition:
Measure

A measure on a measurable space $(\Omega, \mathcal{F})$ is a function $\mu : \mathcal{F} \to [0, \infty]$ satisfying:

$\mu(\emptyset) = 0$ .
Countable additivity: If $A_1, A_2, \ldots \in \mathcal{F}$ are pairwise disjoint, then $\mu\!\left(\bigcup_{n=1}^{\infty} A_n\right) = \sum_{n=1}^{\infty} \mu(A_n)$ .

The triple $(\Omega, \mathcal{F}, \mu)$ is called a measure space. If $\mu(\Omega) = 1$ , then $\mu$ is a probability measure and we write $P$ instead of $\mu$ .

Definition:
Lebesgue Measure on $\mathbb{R}$

The Lebesgue measure $\lambda$ on $(\mathbb{R}, \mathcal{B}(\mathbb{R}))$ is the unique measure satisfying $\lambda((a, b]) = b - a$ for every interval $(a, b]$ . Its existence is guaranteed by the Carathodory extension theorem.

Key properties:

Translation invariance: $\lambda(A + x) = \lambda(A)$ for all $x \in \mathbb{R}$ .
Countable sets have measure zero: $\lambda(\mathbb{Q}) = 0$ .
The Cantor set has measure zero yet is uncountable.

Lebesgue measure is the "right" notion of length/area/volume for measurable subsets of $\mathbb{R}^n$ . In probability, a continuous random variable $X$ has a density $f_X$ if and only if its distribution $P_X$ is absolutely continuous with respect to Lebesgue measure — and the density is the Radon-Nikodym derivative $f_X = dP_X / d\lambda$ (Section 22.4).

,

Theorem: Carathodory Extension Theorem

Let $\mathcal{A}$ be an algebra of subsets of $\Omega$ and let $\mu_0 : \mathcal{A} \to [0, \infty]$ be a pre-measure (finitely additive and countably additive on $\mathcal{A}$ ). If $\mu_0$ is $\sigma$ -finite (i.e., $\Omega = \bigcup_{n} A_n$ with $\mu_0(A_n) < \infty$ ), then $\mu_0$ extends uniquely to a measure $\mu$ on $\sigma(\mathcal{A})$ .

The theorem says: if you know how to assign "lengths" to intervals in a consistent way (the pre-measure on the algebra of finite unions of intervals), then there is exactly one way to extend this assignment to all Borel sets. This is how Lebesgue measure is constructed — start from $\lambda((a,b]) = b - a$ and extend.

Proof

Outer measure construction

Define the outer measure $\mu^*(A) = \inf\left\{\sum_{n=1}^{\infty} \mu_0(A_n) : A \subseteq \bigcup_n A_n, \, A_n \in \mathcal{A}\right\}$ . This $\mu^*$ is defined on all subsets of $\Omega$ but is not necessarily additive.

Identify measurable sets

A set $E$ is $\mu^*$ -measurable (in the Carathodory sense) if for every $A \subseteq \Omega$ : $\mu^*(A) = \mu^*(A \cap E) + \mu^*(A \cap E^c)$ . The collection of all $\mu^*$ -measurable sets forms a sigma-algebra $\mathcal{M}$ , and $\mu^*$ restricted to $\mathcal{M}$ is a measure.

Uniqueness from $\sigma$-finiteness

One shows $\sigma(\mathcal{A}) \subseteq \mathcal{M}$ , so $\mu = \mu^*|_{\sigma(\mathcal{A})}$ extends $\mu_0$ . Uniqueness follows from $\sigma$ -finiteness and the $\pi$ - $\lambda$ theorem. $\blacksquare$

,

Definition:
Measurable Function (= Random Variable)

Let $(\Omega, \mathcal{F})$ and $(S, \mathcal{S})$ be measurable spaces. A function $X : \Omega \to S$ is $(\mathcal{F}, \mathcal{S})$ -measurable if $X^{-1}(B) = \{\omega \in \Omega : X(\omega) \in B\} \in \mathcal{F} \quad \text{for all } B \in \mathcal{S}.$

When $(\Omega, \mathcal{F}, P)$ is a probability space and $S = \mathbb{R}$ , $\mathcal{S} = \mathcal{B}(\mathbb{R})$ , a measurable function $X$ is called a random variable. The condition becomes: $\{\omega : X(\omega) \leq x\} \in \mathcal{F} \quad \text{for all } x \in \mathbb{R}.$

This is the formal version of what we used informally in Chapter 5: a random variable is a function from the sample space to $\mathbb{R}$ that is "compatible" with the sigma-algebra, so that we can compute probabilities of events like $\{X \leq x\}$ .

,

Definition:
Expectation as Lebesgue Integration

If $X$ is a random variable on $(\Omega, \mathcal{F}, P)$ , its expectation is: $\mathbb{E}[X] = \int_{\Omega} X(\omega)\, dP(\omega),$ where the right-hand side is the Lebesgue integral of $X$ with respect to the probability measure $P$ . This single definition unifies:

Discrete case: $\mathbb{E}[X] = \sum_x x \cdot P(X = x)$ (integral w.r.t. counting measure).
Continuous case: $\mathbb{E}[X] = \int_{-\infty}^{\infty} x f_X(x)\, dx$ (integral w.r.t. Lebesgue measure, via the change-of-variables formula).
Mixed/singular case: handled naturally.

Example: Expectation of a Mixed Random Variable

Let $X$ be a random variable with CDF: $F_X(x) = \begin{cases} 0 & x < 0 \\ x/2 & 0 \leq x < 1 \\ 1 & x \geq 1 \end{cases}$ Note that $F_X$ has a jump of size $1/2$ at $x = 1$ and a continuous part on $[0,1)$ . Compute $\mathbb{E}[X]$ .

Solution

Decompose the distribution

The distribution of $X$ is a mixture: with probability $1/2$ , $X$ is uniform on $[0,1)$ (continuous part with density $f(x) = 1/2$ on $[0,1)$ ), and with probability $1/2$ , $X = 1$ (point mass).

Compute via the Lebesgue integral

$\mathbb{E}[X] = \int_0^1 x \cdot \frac{1}{2}\, dx + 1 \cdot \frac{1}{2} = \frac{1}{2} \cdot \frac{1}{2} + \frac{1}{2} = \frac{3}{4}.$ $In the measure-theoretic framework,$ P_X = \frac{1}{2}\lambda|_{[0,1)} + \frac{1}{2}\delta_1 $and$ \mathbb{E}[X] = \int x, dP_X(x)$, which the Lebesgue integral handles seamlessly.

Riemann vs. Lebesgue Integration

Property	Riemann	Lebesgue
Partitions	Domain $[a,b]$	Range of $f$
Handles discontinuous functions	Only if discontinuities have measure zero	All measurable functions
Convergence theorems	Uniform convergence required	MCT, DCT (pointwise suffices)
Defines probability	Only for continuous and discrete RVs	All RVs (discrete, continuous, mixed, singular)
Completeness of $L^p$	No (Riesz-Fischer fails)	Yes ( $L^p$ is a Banach space)

Historical Note: Vitali's Non-Measurable Set and the Necessity of Sigma-Algebras

1905

In 1905, Giuseppe Vitali constructed a subset of $[0,1]$ that cannot be assigned a Lebesgue measure in any consistent way. The construction uses the axiom of choice: partition $[0,1]$ into equivalence classes where $x \sim y$ iff $x - y \in \mathbb{Q}$ , and select one representative from each class. The resulting set $V$ satisfies $[0,1] \subseteq \bigcup_{q \in \mathbb{Q} \cap [0,1]} (V + q)$ , but if $\lambda(V) = 0$ then $\lambda([0,1]) = 0$ (contradiction), and if $\lambda(V) > 0$ then $\lambda([0,1]) = \infty$ (contradiction).

This is why we need sigma-algebras: we cannot assign probabilities to all subsets of an uncountable set in a translation-invariant way. The Borel (or Lebesgue) sigma-algebra is the largest collection of "well-behaved" sets that admits a consistent measure.

Common Mistake: Not All Subsets of $\mathbb{R}$ Are Measurable

Mistake:

Assuming that any subset of $\mathbb{R}$ one can "describe" must be Lebesgue measurable, or that non-measurable sets are merely a logical curiosity with no practical consequence.

Correction:

Vitali's construction shows that non-measurable sets exist (under the axiom of choice). While these sets do not arise in engineering applications, their existence is the reason we need the formalism of sigma-algebras. Without it, the entire theory of probability measures on $\mathbb{R}$ is inconsistent.

Common Mistake: Borel Sets vs. Lebesgue-Measurable Sets

Mistake:

Conflating the Borel sigma-algebra $\mathcal{B}(\mathbb{R})$ with the Lebesgue sigma-algebra $\mathcal{L}(\mathbb{R})$ .

Correction:

The Lebesgue sigma-algebra is strictly larger: $\mathcal{B}(\mathbb{R}) \subsetneq \mathcal{L}(\mathbb{R})$ . There exist Lebesgue-measurable sets that are not Borel sets (subsets of measure-zero Cantor-type sets). For probability theory, the Borel sigma-algebra is almost always sufficient.

Quick Check

Which of the following is NOT a sigma-algebra on $\Omega = \{a, b, c\}$ ?

$\{\emptyset, \Omega\}$

$\{\emptyset, \{a\}, \{b,c\}, \Omega\}$

$\{\emptyset, \{a\}, \{b\}, \Omega\}$

$2^{\Omega}$ (the power set)

Correction:

\{\emptyset, \{a\}, \{b\}, \Omega\}

$\{a\} \cup \{b\} = \{a,b\}$ , which is not in the collection. So it fails closure under unions.

Quick Check

In the measure-theoretic definition, a random variable $X : \Omega \to \mathbb{R}$ is:

Any function from $\Omega$ to $\mathbb{R}$

A function such that $\{\\omega : X(\\omega) \\leq x\} \\in \\mathcal{F}$ for all $x \\in \\mathbb{R}$

A function with a PDF

Correction:

A function such that

\{\\omega : X(\\omega) \\leq x\} \\in \\mathcal{F}

for all

x \\in \\mathbb{R}

This is the measurability condition: the preimage of every Borel set (equivalently, every half-line) must be in the sigma-algebra.

Why This Matters: Mixed Distributions in Fading Channels

In wireless communications, mixed random variables arise naturally. Consider a Rayleigh fading channel where outage occurs: the effective rate is $R = \log_2(1 + \text{SNR} \cdot |h|^2)$ when the channel is not in outage, and $R = 0$ (with positive probability) during outage. The resulting $R$ has a point mass at zero plus a continuous part — exactly the type of distribution that the Lebesgue integral handles naturally but the Riemann integral cannot.

Sigma-Algebra

A collection $\mathcal{F}$ of subsets of $\Omega$ that is closed under complementation and countable unions, and contains $\Omega$ itself. It defines which events can be assigned a probability.

Borel Sigma-Algebra

The smallest sigma-algebra on $\mathbb{R}$ containing all open sets, denoted $\mathcal{B}(\mathbb{R})$ . Equivalently, generated by intervals $(-\infty, x]$ for $x \in \mathbb{R}$ .

Lebesgue Measure

The unique complete, translation-invariant measure on $\mathbb{R}$ that assigns $\lambda((a,b]) = b - a$ to every interval. The standard notion of "length" for measurable subsets of $\mathbb{R}$ .

Key Takeaway

A probability space $(\Omega, \mathcal{F}, P)$ is a measure space with total mass one. The sigma-algebra $\mathcal{F}$ tells us which questions about the outcome $\omega$ can be answered with a probability; a random variable is a measurable function that maps outcomes to numbers in a way compatible with $\mathcal{F}$ ; and expectation is the Lebesgue integral with respect to $P$ .

Sigma-Algebras, Measures, and Measurability

The Three Pillars of a Probability Space

Definition: Sigma-Algebra (σ\sigmaσ-Algebra)

Example: Examples of Sigma-Algebras

(a) Coin flip

(b) Borel sigma-algebra

(c) Trivial and discrete

Definition: Generated Sigma-Algebra

Definition: Measure

Definition: Lebesgue Measure on R\mathbb{R}R

Theorem: Carathodory Extension Theorem

Outer measure construction

Identify measurable sets

Uniqueness from $\sigma$-finiteness

Definition: Measurable Function (= Random Variable)

Definition: Expectation as Lebesgue Integration

Example: Expectation of a Mixed Random Variable

Decompose the distribution

Compute via the Lebesgue integral

Riemann vs. Lebesgue Integration

Historical Note: Vitali's Non-Measurable Set and the Necessity of Sigma-Algebras

Common Mistake: Not All Subsets of R\mathbb{R}R Are Measurable

Common Mistake: Borel Sets vs. Lebesgue-Measurable Sets

Quick Check

Quick Check

Why This Matters: Mixed Distributions in Fading Channels

Sigma-Algebra

Borel Sigma-Algebra

Lebesgue Measure

Key Takeaway

Definition:
Sigma-Algebra ( $\sigma$ -Algebra)

Definition:
Generated Sigma-Algebra

Definition:
Measure

Definition:
Lebesgue Measure on $\mathbb{R}$

Definition:
Measurable Function (= Random Variable)

Definition:
Expectation as Lebesgue Integration

Common Mistake: Not All Subsets of $\mathbb{R}$ Are Measurable