Ferkans — Interactive Telecom Tutor

From Discrete to Continuous

In the discrete setting, every outcome carries a positive probability mass. For a continuous random variable, the probability of any single point is zero — and yet the random variable still has a well-defined distribution. The resolution of this apparent paradox is the probability density function: a non-negative function whose integral over any interval gives the probability of falling in that interval. The CDF, which we already defined for discrete RVs, remains valid and becomes the bridge between the discrete and continuous worlds.

Definition:
Continuous Random Variable

A random variable $X$ is continuous if its CDF $F(x) = \mathbb{P}(X \leq x)$ is a continuous function of $x$ . Equivalently, $\mathbb{P}(X = x) = 0$ for every $x \in \mathbb{R}$ .

Continuity of the CDF is a stronger condition than merely having $\mathbb{P}(X = x) = 0$ for all $x$ ; it requires that the CDF has no jumps anywhere.

Definition:
Probability Density Function (PDF)

Let $X$ be a continuous random variable with CDF $F$ . The probability density function (PDF) of $X$ is the function $f : \mathbb{R} \to [0, \infty)$ satisfying

$F(x) = \int_{-\infty}^{x} f(u)\,du \quad \text{for all } x \in \mathbb{R}.$

Equivalently, for any interval $(a, b]$ :

$\mathbb{P}(a < X \leq b) = \int_a^b f(x)\,dx.$

The PDF is not a probability — it can exceed 1. The quantity $f(x)\,dx$ represents the infinitesimal probability mass near $x$ : $\mathbb{P}(x < X \leq x + dx) \approx f(x)\,dx$ .

Theorem: Properties of the PDF

Let $f$ be the PDF of a continuous RV $X$ . Then:

$f(x) \geq 0$ for all $x \in \mathbb{R}$ .
$\int_{-\infty}^{\infty} f(x)\,dx = 1$ .
For any Borel set $A \subseteq \mathbb{R}$ : $\mathbb{P}(X \in A) = \int_A f(x)\,dx$ .

Conversely, any non-negative function satisfying property 2 is a valid PDF.

Proof

Non-negativity

Since $F(x)$ is non-decreasing and $F(x) = \int_{-\infty}^{x} f(u)\,du$ , we must have $f(x) \geq 0$ wherever the derivative exists.

Normalization

$\int_{-\infty}^{\infty} f(x)\,dx = \lim_{x \to \infty} F(x) = 1$ , since the CDF approaches 1 as $x \to \infty$ .

Probability of a Borel set

This follows from the countable additivity of the probability measure and the representation of Borel sets as countable unions and intersections of intervals. $\blacksquare$

Theorem: Fundamental Theorem: Differentiating the CDF

If $F$ is differentiable at $x$ , then

$f(x) = \frac{d}{dx}F(x) = F'(x).$

This is the continuous analog of the relation $p_X(x_i) = F(x_i) - F(x_i^-)$ for discrete RVs.

The CDF accumulates probability from $-\infty$ to $x$ . Its rate of accumulation at $x$ is exactly the density at $x$ .

Proof

Direct application of the Fundamental Theorem of Calculus

Since $F(x) = \int_{-\infty}^{x} f(u)\,du$ and $f$ is assumed continuous at $x$ (or at least locally integrable), the Fundamental Theorem of Calculus gives $F'(x) = f(x)$ . $\blacksquare$

Theorem: Properties of the CDF

The CDF $F(x) = \mathbb{P}(X \leq x)$ of any random variable $X$ satisfies:

$\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to +\infty} F(x) = 1$ .
$F$ is non-decreasing: $x \leq y \implies F(x) \leq F(y)$ .
$F$ is right-continuous: $\lim_{h \downarrow 0} F(x + h) = F(x)$ .

Moreover, these three properties characterize CDFs: any function satisfying all three is the CDF of some random variable.

Proof

Boundary limits

Define $A_n = \{X \leq -n\}$ . Then $A_n \downarrow \emptyset$ , so by continuity of probability, $F(-n) = \mathbb{P}(A_n) \to 0$ . Similarly for the limit at $+\infty$ .

Monotonicity

If $x \leq y$ , then $\{X \leq x\} \subseteq \{X \leq y\}$ , so $F(x) = \mathbb{P}(X \leq x) \leq \mathbb{P}(X \leq y) = F(y)$ .

Right-continuity

Define $B_n = \{X \leq x + 1/n\}$ . Then $B_n \downarrow \{X \leq x\}$ , so $F(x + 1/n) \to F(x)$ by continuity of probability from above.

Characterization (sketch)

Given a function $F$ satisfying the three properties, define a probability measure on $(\mathbb{R}, \mathcal{B})$ via $\mathbb{P}((a,b]) = F(b) - F(a)$ . The Carathéodory extension theorem guarantees a unique probability measure on the Borel $\sigma$ -algebra, and the identity random variable $X(\omega) = \omega$ has CDF $F$ . $\blacksquare$

Example: Computing Probabilities from the CDF

Let $X$ have CDF

$F(x) = \begin{cases} 0 & x < 0 \\ 1 - e^{-2x} & x \geq 0. \end{cases}$

Compute $\mathbb{P}(1 < X \leq 3)$ and $\mathbb{P}(X > 2)$ .

Solution

Interval probability

$\mathbb{P}(1 < X \leq 3) = F(3) - F(1) = (1 - e^{-6}) - (1 - e^{-2}) = e^{-2} - e^{-6} \approx 0.1329.$

Tail probability

$\mathbb{P}(X > 2) = 1 - F(2) = e^{-4} \approx 0.0183.$

Identify the distribution

The CDF $1 - e^{-2x}$ for $x \geq 0$ is that of an $\text{Exp}(2)$ random variable.

Example: Deriving the PDF from a Given CDF

A random variable $X$ has CDF

$F(x) = \begin{cases} 0 & x < 0 \\ x^2 & 0 \leq x < 1 \\ 1 & x \geq 1. \end{cases}$

Find the PDF $f(x)$ and compute $\mathbb{P}(1/4 < X \leq 3/4)$ .

Solution

Differentiate the CDF

$f(x) = F'(x) = 2x$ for $0 \leq x < 1$ , and $f(x) = 0$ otherwise.

Verify normalization

$\int_0^1 2x\,dx = [x^2]_0^1 = 1$ . $\checkmark$

Compute the probability

$\mathbb{P}(1/4 < X \leq 3/4) = F(3/4) - F(1/4) = (3/4)^2 - (1/4)^2 = 9/16 - 1/16 = 1/2.$

Example: The PDF Can Exceed 1

Let $X \sim \text{Uniform}[0, 1/3]$ . Show that $f(x)$ exceeds 1 for all $x$ in the support, and verify that $X$ is nonetheless a valid random variable.

Solution

Compute the PDF

$f(x) = \frac{1}{1/3 - 0} = 3$ for $x \in [0, 1/3]$ , and $0$ otherwise. So $f(x) = 3 > 1$ on the support.

Verify normalization

$\int_0^{1/3} 3\,dx = 1$ . The PDF integrates to 1, so this is a valid distribution. The density is a rate of probability per unit length, not a probability itself.

Common Mistake: Confusing PDF Values with Probabilities

Mistake:

Interpreting $f(x) = 3$ as meaning "the probability of $X = x$ is 3."

Correction:

For a continuous RV, $\mathbb{P}(X = x) = 0$ for every $x$ . The PDF is a density: $f(x)\,dx$ is the infinitesimal probability mass near $x$ . The density can be arbitrarily large; only its integral over any set must lie in $[0, 1]$ .

Quick Check

If $F(x) = 1 - e^{-3x}$ for $x \geq 0$ (and $0$ for $x < 0$ ), what is $f(2)$ ?

$1 - e^{-6}$

$3e^{-6}$

$e^{-6}$

$6e^{-3}$

Correction:

3e^{-6}

$f(x) = F'(x) = 3e^{-3x}$ , so $f(2) = 3e^{-6}$ .

Historical Note: The CDF: From Laplace to Kolmogorov

1812–1933

The idea of describing a random variable through its cumulative distribution dates back to Laplace's work on the "generating function of errors" in the early 19th century. However, the modern axiomatic treatment — where the CDF is the primary object and the PDF is derived from it — was established by Kolmogorov in his 1933 Grundbegriffe der Wahrscheinlichkeitsrechnung. Kolmogorov's key insight was that the CDF, being defined through the probability measure alone, works for discrete, continuous, and mixed distributions alike, without any need to distinguish cases at the foundational level.

Probability Density Function (PDF)

A non-negative function $f$ such that $\mathbb{P}(X \in A) = \int_A f(x)\,dx$ for every Borel set $A$ . The PDF is the derivative of the CDF wherever it exists.

Cumulative Distribution Function (CDF)

The function $F(x) = \mathbb{P}(X \leq x)$ . It is non-decreasing, right-continuous, with $\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to \infty} F(x) = 1$ .

Key Takeaway

The CDF is the universal descriptor of any random variable — discrete, continuous, or mixed. The PDF exists only for continuous RVs and is obtained by differentiating the CDF. For any interval, $\mathbb{P}(a < X \leq b) = F(b) - F(a) = \int_a^b f(x)\,dx$ .

PDF and CDF Explorer

Visualize the PDF and CDF of a Gaussian distribution. Adjust $\mu$ and $\sigma$ and observe how the area under the PDF curve corresponds to the CDF value.

Parameters

\mu

0

\sigma

1

x

1

Point at which to evaluate the CDF

Why This Matters: CDF and Outage Probability in Wireless

In wireless communications, the CDF of the received signal-to-noise ratio (SNR) $\gamma$ directly gives the outage probability: $P_{\text{out}}(\gamma_0) = F_\gamma(\gamma_0) = \mathbb{P}(\gamma \leq \gamma_0)$ . Designing a system to meet a target outage probability is equivalent to ensuring that the CDF of the SNR falls below the required threshold at the target rate. This is why the CDF — and the PDF from which it derives — is the fundamental tool in fading channel analysis.

PDF, CDF, and the Relationship Between Them

From Discrete to Continuous

Definition: Continuous Random Variable

Definition: Probability Density Function (PDF)

Theorem: Properties of the PDF

Non-negativity

Normalization

Probability of a Borel set

Theorem: Fundamental Theorem: Differentiating the CDF

Direct application of the Fundamental Theorem of Calculus

Theorem: Properties of the CDF

Boundary limits

Monotonicity

Right-continuity

Characterization (sketch)

Example: Computing Probabilities from the CDF

Interval probability

Tail probability

Identify the distribution

Example: Deriving the PDF from a Given CDF

Differentiate the CDF

Verify normalization

Compute the probability

Example: The PDF Can Exceed 1

Compute the PDF

Verify normalization

Common Mistake: Confusing PDF Values with Probabilities

Quick Check

Historical Note: The CDF: From Laplace to Kolmogorov

Probability Density Function (PDF)

Cumulative Distribution Function (CDF)

Key Takeaway

PDF and CDF Explorer

Parameters

Why This Matters: CDF and Outage Probability in Wireless

Definition:
Continuous Random Variable

Definition:
Probability Density Function (PDF)