Ferkans — Interactive Telecom Tutor

Why Joint Distributions?

In Chapters 5 and 6 we studied a single random variable at a time. But in virtually every engineering problem, multiple quantities interact: the signal and the noise, the channel gain and the interference, the transmit power and the received SNR. To reason about how two or more random variables relate to each other — whether they are dependent, how one conditions the other, what happens when we add or transform them — we need the joint distribution.

The marginal distributions $f_{X}$ and $f_{Y}$ alone do not determine the joint distribution $f_{X,Y}$ . The joint distribution is a strictly richer object: it encodes all marginals, all conditionals, and all dependence structure.

Definition:
Joint Cumulative Distribution Function

Let $X$ and $Y$ be random variables defined on a common probability space $(\Omega, \mathcal{F}, \mathbb{P})$ . The joint CDF of $(X, Y)$ is the function $F_{X,Y} : \mathbb{R}^2 \to [0,1]$ defined by

$F_{X,Y}(x, y) = \mathbb{P}(X \le x,\; Y \le y).$

The joint CDF is the fundamental object from which all other joint distributional quantities are derived. It always exists, regardless of whether the RVs are discrete, continuous, or mixed.

Theorem: Properties of the Joint CDF

Let $F_{X,Y}(x,y)$ be the joint CDF of $(X,Y)$ . Then:

Limits: $\lim_{x \to -\infty} F_{X,Y}(x,y) = 0$ for all $y$ , $\lim_{y \to -\infty} F_{X,Y}(x,y) = 0$ for all $x$ , and $\lim_{x,y \to +\infty} F_{X,Y}(x,y) = 1$ .
Monotonicity: $F_{X,Y}$ is non-decreasing in each argument.
Right-continuity: $F_{X,Y}$ is right-continuous in each argument.
Marginals: $F_{X}(x) = \lim_{y \to \infty} F_{X,Y}(x,y)$ and $F_{Y}(y) = \lim_{x \to \infty} F_{X,Y}(x,y)$ .
Rectangle probability: For $a < b$ and $c < d$ , $\mathbb{P}(a < X \le b,\; c < Y \le d) = F_{X,Y}(b,d) - F_{X,Y}(a,d) - F_{X,Y}(b,c) + F_{X,Y}(a,c) \ge 0.$

Proof

Limits and monotonicity

Property 1 follows from continuity of probability applied to increasing/decreasing sequences of events. For instance, $\{X \le x, Y \le y_n\} \downarrow \emptyset$ when $y_n \to -\infty$ , so $\mathbb{P}(X \le x, Y \le y_n) \to 0$ .

Monotonicity in $x$ : if $x_1 \le x_2$ then $\{X \le x_1, Y \le y\} \subseteq \{X \le x_2, Y \le y\}$ , so $F_{X,Y}(x_1, y) \le F_{X,Y}(x_2, y)$ .

Right-continuity

Fix $y$ and let $x_n \downarrow x$ . Then $\{X \le x_n, Y \le y\} \downarrow \{X \le x, Y \le y\}$ , so $F_{X,Y}(x_n, y) \to F_{X,Y}(x, y)$ by continuity of probability from above. The argument for $y$ is identical.

Marginals and rectangle formula

Letting $y \to \infty$ , the event $\{X \le x, Y \le y\} \uparrow \{X \le x\}$ , so $F_{X,Y}(x, y) \to F_{X}(x)$ .

The rectangle formula follows by inclusion-exclusion: $\{a < X \le b, c < Y \le d\}$ is the intersection of $\{a < X \le b\}$ and $\{c < Y \le d\}$ , and expanding via the CDF gives the alternating sum. Non-negativity follows because this is a probability. $\blacksquare$

Definition:
Joint Probability Mass Function

Let $X$ and $Y$ be discrete random variables with supports $\mathcal{X} = \{x_1, x_2, \ldots\}$ and $\mathcal{Y} = \{y_1, y_2, \ldots\}$ . The joint PMF is

$P_{X,Y}(x_i, y_j) = \mathbb{P}(X = x_i,\; Y = y_j),$

satisfying $P_{X,Y}(x_i, y_j) \ge 0$ for all $i, j$ and $\sum_i \sum_j P_{X,Y}(x_i, y_j) = 1$ .

The joint PMF can be displayed as a table (or matrix) indexed by the values of $X$ and $Y$ . Row sums give the marginal $P_{X}$ ; column sums give the marginal $P_{Y}$ .

Definition:
Marginal PMF from Joint PMF

Given the joint PMF $P_{X,Y}$ , the marginal PMFs are obtained by summing over the other variable:

$P_{X}(x_i) = \sum_j P_{X,Y}(x_i, y_j), \qquad P_{Y}(y_j) = \sum_i P_{X,Y}(x_i, y_j).$

The marginal PMFs are proper PMFs: each is non-negative and sums to 1.

Example: Joint PMF — Weather in Two Cities

Let $X$ and $Y$ denote the weather in Los Angeles and San Francisco, respectively, where $0$ = sunny and $1$ = cloudy. The joint PMF is given by the table:

$X \backslash Y$	0	1
0	0.2	0.5
1	0.1	0.2

Find the marginal PMFs and compute $\mathbb{P}(\text{at least one city is sunny})$ .

Solution

Marginal of $X$

$P_{X}(0) = 0.2 + 0.5 = 0.7$ , $P_{X}(1) = 0.1 + 0.2 = 0.3$ . So LA is sunny with probability 0.7.

Marginal of $Y$

$P_{Y}(0) = 0.2 + 0.1 = 0.3$ , $P_{Y}(1) = 0.5 + 0.2 = 0.7$ .

At least one sunny

The complement is "both cloudy": $\mathbb{P}(X = 1, Y = 1) = 0.2$ . Therefore $\mathbb{P}(\text{at least one sunny}) = 1 - 0.2 = 0.8$ .

Definition:
Joint Probability Density Function

Two random variables $X$ and $Y$ are jointly continuous if their joint CDF can be expressed as

$F_{X,Y}(x,y) = \int_{-\infty}^{x} \int_{-\infty}^{y} f_{X,Y}(u,v)\,dv\,du$

for some non-negative function $f_{X,Y}$ called the joint probability density function. Equivalently,

$f_{X,Y}(x,y) = \frac{\partial^2 F_{X,Y}(x,y)}{\partial x\,\partial y}$

wherever the mixed partial derivative exists.

The joint PDF satisfies $f_{X,Y}(x,y) \ge 0$ and $\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dx\,dy = 1$ .

The value $f_{X,Y}(x,y)$ is not a probability — it is a density. The probability of $(X,Y)$ falling in a region $A$ is $\mathbb{P}((X,Y) \in A) = \iint_A f_{X,Y}(x,y)\,dx\,dy$ .

Definition:
Marginal PDF from Joint PDF

Given the joint PDF $f_{X,Y}$ , the marginal PDFs are

$f_{X}(x) = \int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dy, \qquad f_{Y}(y) = \int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dx.$

Geometrically, the marginal $f_{X}(x)$ is obtained by integrating ("projecting") the joint density along the $y$ -axis.

Example: Bivariate Uniform on a Triangle

Let $(X, Y)$ be uniformly distributed on the triangle $\{(x,y) : 0 \le x \le 1,\; 0 \le y \le x\}$ , which has area $1/2$ . Find the joint PDF, the marginal PDFs, and $\mathbb{P}(Y > X/2)$ .

Solution

Joint PDF

The area of the triangle is $1/2$ , so the uniform density is $f_{X,Y}(x,y) = 2$ for $0 \le y \le x \le 1$ and zero otherwise.

Marginal of $X$

$f_{X}(x) = \int_0^x 2\,dy = 2x$ for $x \in [0,1]$ .

Marginal of $Y$

$f_{Y}(y) = \int_y^1 2\,dx = 2(1 - y)$ for $y \in [0,1]$ .

Compute $\mathbb{P}(Y > X/2)$

The region is $\{0 \le y \le x \le 1,\; y > x/2\}$ , i.e., $\{x/2 < y \le x,\; 0 \le x \le 1\}$ :

$\mathbb{P}(Y > X/2) = \int_0^1 \int_{x/2}^{x} 2\,dy\,dx = \int_0^1 2 \cdot \frac{x}{2}\,dx = \int_0^1 x\,dx = \frac{1}{2}.$

Joint probability density function

A non-negative function $f_{X,Y}(x,y)$ whose double integral over any region $A \subseteq \mathbb{R}^2$ gives $\mathbb{P}((X,Y) \in A)$ .

Related: Marginal distribution

Marginal distribution

The distribution of a single random variable obtained from a joint distribution by integrating (or summing) over all other variables.

Joint PDF and Marginal Projections

Animation showing how the marginal densities

f_{X}(x)

and

f_{Y}(y)

arise as projections (integrals) of the joint density

f_{X,Y}(x,y)

along each axis.

Joint PDF Contour Plot with Marginals

Explore the joint density of a bivariate Gaussian with adjustable means, variances, and correlation coefficient $\rho$ . The marginal densities are displayed on the side panels.

Parameters

\mu_X

0

\mu_Y

0

\sigma_X

1

\sigma_Y

1

\rho

0

Common Mistake: Marginals Do Not Determine the Joint Distribution

Mistake:

Assuming that knowing $f_{X}$ and $f_{Y}$ is enough to determine $f_{X,Y}$ .

Correction:

Infinitely many joint distributions share the same marginals. The joint distribution encodes the dependence structure between $X$ and $Y$ , which the marginals alone cannot capture. For instance, two standard Gaussian marginals can be paired with any correlation $\rho \in [-1, 1]$ to produce different bivariate Gaussian distributions.

Quick Check

If $f_{X,Y}(x,y) = 6(1-y)$ for $0 \le x \le y \le 1$ and zero otherwise, what is $f_{X}(x)$ for $x \in [0,1]$ ?

$3(1-x)^2$

$6(1-x)$

$3x(1-x)$

$6x$

Correction:

3(1-x)^2

$f_{X}(x) = \int_x^1 6(1-y)\,dy = 6\bigl[\frac{(1-y)^2}{-2}\bigr]_x^1 = 3(1-x)^2$ .

Historical Note: The Origins of Multivariate Distributions

1880s–1933

The study of joint distributions began in earnest with Francis Galton's work on regression and correlation in the 1880s. Galton noticed that the heights of fathers and sons formed an elliptical scatter pattern — the hallmark of a bivariate Gaussian. Karl Pearson formalized this observation into the multivariate normal distribution and introduced the correlation coefficient $\rho$ that we still use today. The generalization to arbitrary joint distributions, via the joint CDF, came later with Kolmogorov's axiomatization of probability in 1933.

Key Takeaway

The joint distribution $f_{X,Y}$ determines the marginals $f_{X}$ and $f_{Y}$ (by integration), but the converse is false. The joint distribution is a strictly richer object that encodes the full dependence structure between the random variables.

Joint PMFs and PDFs

Why Joint Distributions?

Definition: Joint Cumulative Distribution Function

Theorem: Properties of the Joint CDF

Limits and monotonicity

Right-continuity

Marginals and rectangle formula

Definition: Joint Probability Mass Function

Definition: Marginal PMF from Joint PMF

Example: Joint PMF — Weather in Two Cities

Marginal of $X$

Marginal of $Y$

At least one sunny

Definition: Joint Probability Density Function

Definition: Marginal PDF from Joint PDF

Example: Bivariate Uniform on a Triangle

Joint PDF

Marginal of $X$

Marginal of $Y$

Compute $\mathbb{P}(Y > X/2)$

Joint probability density function

Marginal distribution

Joint PDF and Marginal Projections

Joint PDF Contour Plot with Marginals

Parameters

Common Mistake: Marginals Do Not Determine the Joint Distribution

Quick Check

Historical Note: The Origins of Multivariate Distributions

Key Takeaway

Definition:
Joint Cumulative Distribution Function

Definition:
Joint Probability Mass Function

Definition:
Marginal PMF from Joint PMF

Definition:
Joint Probability Density Function

Definition:
Marginal PDF from Joint PDF