Ferkans — Interactive Telecom Tutor

Why Can't We Just Assign Probabilities to All Subsets?

When $\Omega$ is finite or countably infinite, we can and do assign probabilities to every subset. The power set $2^\Omega$ works fine.

When $\Omega$ is uncountable — say $\Omega = [0,1]$ — a disturbing fact emerges: it is mathematically impossible to define a function $\mathbb{P}: 2^{[0,1]} \to [0,1]$ that satisfies even the most basic requirements (translation invariance and countable additivity) simultaneously. The obstruction was made explicit by Giuseppe Vitali in 1905, who constructed a subset of $[0,1]$ that cannot be assigned a consistent length (the Vitali set — see the historical note below).

The resolution is not to assign probabilities to all subsets, but to restrict attention to a carefully chosen collection of subsets — the sigma-algebra — that is large enough to contain every event we will ever want to discuss, but small enough to exclude the pathological sets where probability breaks down.

For the practicing engineer, this matters the moment you work with continuous-valued random variables. Every CDF, every PDF, every integral of the form $\int_A f(x)\,dx$ implicitly uses the Borel sigma-algebra on $\mathbb{R}$ . Understanding sigma-algebras now will prevent confusion later when continuous random variables and stochastic processes enter the picture.

,

Definition:
Sigma-Algebra

A collection $\mathcal{F}$ of subsets of $\Omega$ is called a sigma-algebra (or $\sigma$ -field) if:

Contains the empty set: $\emptyset \in \mathcal{F}$ .
Closed under complementation: If $A \in \mathcal{F}$ , then $A^c \in \mathcal{F}$ .
Closed under countable unions: If $A_1, A_2, \ldots \in \mathcal{F}$ , then $\bigcup_{i=1}^{\infty} A_i \in \mathcal{F}$ .

The elements of $\mathcal{F}$ are called measurable sets or events.

From the three axioms, we immediately derive:

$\Omega = \emptyset^c \in \mathcal{F}$ .
Closed under countable intersections (by De Morgan: $\bigcap A_i = (\bigcup A_i^c)^c$ ).
Closed under set difference: $A \setminus B = A \cap B^c$ .

So a sigma-algebra is closed under all the operations we ever apply to events.

,

Example: Four Important Sigma-Algebras

For $\Omega = \{1,2,3,4,5,6\}$ (a single die roll) and for $\Omega = [0,1]$ , identify the following sigma-algebras: (a) The trivial sigma-algebra. (b) The sigma-algebra generated by the event $A = \{1,2,3\}$ . (c) The power set of $\{1,2,3,4,5,6\}$ . (d) The Borel sigma-algebra on $[0,1]$ .

Solution

(a) Trivial sigma-algebra

$\mathcal{F} = \{\emptyset, \Omega\}$ . This is the smallest sigma-algebra on any $\Omega$ . It contains only the impossible event and the certain event. Assigning a probability measure to $(\Omega, \mathcal{F})$ means only specifying $\mathbb{P}(\emptyset) = 0$ and $\mathbb{P}(\Omega) = 1$ — no information about individual outcomes can be extracted from measurements.

(b) Sigma-algebra generated by $A = \{1,2,3\}$

$\mathcal{F} = \{\emptyset, A, A^c, \Omega\} = \{\emptyset, \{1,2,3\}, \{4,5,6\}, \{1,2,3,4,5,6\}\}$ . This is the smallest sigma-algebra containing $A$ : we must include $A$ (given), $A^c$ (by closure under complementation), $\emptyset$ , and $\Omega$ . No further sets are needed to satisfy the axioms.

(c) Power set of $\{1,2,3,4,5,6\}$

$\mathcal{F} = 2^\Omega$ , with $|2^\Omega| = 2^6 = 64$ elements. For any finite $\Omega$ , the power set is a sigma-algebra — this is straightforward to verify. It is the largest possible sigma-algebra on $\Omega$ , and it is the natural choice whenever $\Omega$ is finite or countably infinite.

(d) Borel sigma-algebra on $[0,1]$

$\mathcal{B}([0,1])$ is the smallest sigma-algebra containing all open intervals $(a,b) \subseteq [0,1]$ . It contains all open sets, all closed sets, all countable unions and intersections thereof. Every set of the form $\{x : f(x) > c\}$ for a continuous function $f$ is a Borel set. The Borel sigma-algebra is the standard choice for $\Omega = \mathbb{R}$ or any interval, and it is what every CDF and PDF implicitly uses.

Historical Note: Vitali Sets and the Need for Sigma-Algebras

1905

In 1905, the Italian mathematician Giuseppe Vitali constructed the first example of a non-measurable set — a subset of $[0,1]$ that cannot be consistently assigned a length (measure). The construction uses the Axiom of Choice to select one representative from each equivalence class of real numbers under the relation $x \sim y \iff x - y \in \mathbb{Q}$ . The resulting set $V$ cannot be assigned a value $\mu(V) \in [0,\infty]$ that is consistent with the requirement that the measure of $[0,1]$ is 1 and that measure is translation invariant.

Vitali's example was a shock to early 20th-century mathematics: it showed that "set" and "measurable set" are not the same concept. The sigma-algebra formalizes exactly this distinction. For engineers, the practical import is that probability statements about continuous random variables should always be framed in terms of Borel sets (intervals and their countable combinations) — and one never needs to worry about Vitali sets in any physical application.

Definition:
Borel Sigma-Algebra

The Borel sigma-algebra on $\mathbb{R}$ , denoted $\mathcal{B}(\mathbb{R})$ , is the smallest sigma-algebra containing all open intervals $(a, b)$ for $a < b \in \mathbb{R}$ .

Equivalently, $\mathcal{B}(\mathbb{R})$ is generated by any of the following:

All open sets in $\mathbb{R}$ .
All half-open intervals $(-\infty, x]$ for $x \in \mathbb{R}$ .
All closed sets in $\mathbb{R}$ .

The restriction to $[0,1]$ gives $\mathcal{B}([0,1])$ ; the generalization to $\mathbb{R}^d$ gives $\mathcal{B}(\mathbb{R}^d)$ , generated by open rectangles (hyper-rectangles in $d$ dimensions).

The equivalence of these three generating families is a non-trivial theorem (proved in any measure theory course). For our purposes, the take-away is that the Borel sigma-algebra on $\mathbb{R}$ contains every set we will ever encounter in practice: intervals of every type, level sets of continuous functions, countable sets, and so on.

,

Key Takeaway

For finite and countably infinite $\Omega$ , use the power set. For uncountable $\Omega$ (the continuous case), use the Borel sigma-algebra. In both cases, the sigma-algebra specifies which events can be assigned a probability. From Chapter 5 onward, when random variables map outcomes into $\mathbb{R}$ , the Borel sigma-algebra is always in the background.

Sigma-Algebras for Different Sample Spaces

Sample Space $\Omega$	Natural $\mathcal{F}$	Size / Structure	Example
Finite: $\{\omega_1,\ldots,\omega_N\}$	Power set $2^\Omega$	$2^N$ elements	Coin flip, die roll, BPSK symbol
Countably infinite: $\mathbb{N}$	Power set $2^{\mathbb{N}}$	Uncountably many sets	Number of retransmissions
Uncountable: $\mathbb{R}$	Borel $\mathcal{B}(\mathbb{R})$	Uncountably many; excludes Vitali sets	Received signal amplitude
Uncountable: $\mathbb{R}^n$	Borel $\mathcal{B}(\mathbb{R}^n)$	Generated by open rectangles	MIMO received vector

Sigma-Algebra

A collection $\mathcal{F}$ of subsets of $\Omega$ that contains $\emptyset$ and is closed under complementation and countable unions. Specifies which events can be assigned a probability.

Borel Sigma-Algebra

The smallest sigma-algebra on $\mathbb{R}$ containing all open intervals. Standard choice for continuous probability spaces.

Related: Sigma-Algebra

Common Mistake: Not Every Subset Is an Event

Mistake:

When $\Omega$ is uncountable, it is tempting to assume that every subset can be assigned a probability. After all, what does it even mean for $\mathbb{P}(A)$ to be undefined for some set $A$ ?

Correction:

In the continuous setting, probability is defined only for measurable sets — elements of the sigma-algebra $\mathcal{F}$ . Non-measurable sets like the Vitali set genuinely exist (under the Axiom of Choice), but they never arise from any physical measurement or engineering calculation. The practical rule: if you specify an event using a finite combination of inequalities involving a continuous random variable ( $\{X \leq 3\}$ , $\{X^2 + Y^2 < 1\}$ , etc.), it is automatically a Borel set and is measurable.

Sigma-Algebras and Why They Matter