Prerequisites & Notation

Before You Begin

This chapter develops the measure-theoretic foundations that underpin modern probability theory. It is optional — the rest of the FSP book can be read without it — but essential for readers heading toward information theory research (Book ITA) or advanced stochastic processes.

  • Probability axioms, sample spaces, events(Review fsp/ch01)

    Self-check: Can you state Kolmogorov's axioms and explain what a probability space is?

  • Random variables, PMFs, PDFs, CDFs(Review fsp/ch05)

    Self-check: Can you define a random variable and compute its distribution?

  • Expectation, variance, conditional expectation(Review fsp/ch12)

    Self-check: Can you compute E[Y∣X=x]\mathbb{E}[Y \mid X = x] for jointly Gaussian (X,Y)(X,Y)?

  • Convergence of random variables (a.s., in probability, in distribution)(Review fsp/ch11)

    Self-check: Can you state the WLLN and CLT?

  • Basic real analysis: suprema, infima, limits of sequences of sets

    Self-check: Can you compute lim sup⁡n→∞An\limsup_{n \to \infty} A_n for a sequence of sets {An}\{A_n\}?

Notation for This Chapter

Symbols introduced in this chapter. See also the NGlobal Notation Table master table in the front matter.

SymbolMeaningIntroduced
(Ω,F,P)(\Omega, \mathcal{F}, P)Probability space: sample space, sigma-algebra, probability measures02
B(R)\mathcal{B}(\mathbb{R})Borel sigma-algebra on R\mathbb{R}s02
λ\lambdaLebesgue measure on R\mathbb{R}s02
E[X∣G]\mathbb{E}[X \mid \mathcal{G}]Conditional expectation of XX given sigma-algebra G\mathcal{G}s03
{Xn,Fn}\{X_n, \mathcal{F}_n\}Martingale: adapted sequence with E[Xn+1∣Fn]=Xn\mathbb{E}[X_{n+1} \mid \mathcal{F}_n] = X_ns03
dPdQ\frac{dP}{dQ}Radon-Nikodym derivative of PP w.r.t. QQs04
P≪QP \ll QPP is absolutely continuous with respect to QQs04
μ\muGeneric measure (not necessarily probability)s02
← Ch 21Why Measure Theory