Ferkans — Interactive Telecom Tutor

Why Functions of Multiple RVs?

In Chapter 6 we found the distribution of $Y = g(X)$ for a single RV. Now we need the distribution of functions of two or more RVs: the sum of a signal and noise ( $Z = X + W$ ), the ratio of signal power to interference ( $\text{SINR} = S / I$ ), or more general transformations. Two techniques dominate: the Jacobian method for invertible transformations and the convolution formula for sums of independent RVs.

Theorem: Jacobian Transformation Formula

Let $(X, Y)$ be jointly continuous with PDF $f_{X,Y}$ and let $g : \mathbb{R}^2 \to \mathbb{R}^2$ be a one-to-one (invertible) transformation with inverse $g^{-1}(u, v) = (h_1(u,v), h_2(u,v))$ . Define $(U, V) = g(X, Y)$ . Then the joint PDF of $(U, V)$ is

$f_{U,V}(u, v) = f_{X,Y}\bigl(h_1(u,v),\, h_2(u,v)\bigr) \cdot |J_{g^{-1}}(u,v)|,$

where the Jacobian is

$J_{g^{-1}}(u,v) = \det\begin{pmatrix} \dfrac{\partial h_1}{\partial u} & \dfrac{\partial h_1}{\partial v} \\[6pt] \dfrac{\partial h_2}{\partial u} & \dfrac{\partial h_2}{\partial v} \end{pmatrix}.$

The Jacobian accounts for the stretching and compression of area elements under the transformation. The absolute value ensures the density remains non-negative regardless of the orientation of the mapping.

Proof

From the multivariable change of variables

For any measurable set $B \subseteq \mathbb{R}^2$ :

$\mathbb{P}((U,V) \in B) = \mathbb{P}((X,Y) \in g^{-1}(B)) = \iint_{g^{-1}(B)} f_{X,Y}(x,y)\,dx\,dy.$

Applying the substitution $(x,y) = g^{-1}(u,v)$ :

$= \iint_B f_{X,Y}(h_1(u,v), h_2(u,v))\,|J_{g^{-1}}(u,v)|\,du\,dv.$

Identify the density

Since this holds for all measurable $B$ , the integrand is the joint PDF of $(U,V)$ . $\blacksquare$

Jacobian Method — Step by Step

Input: Joint PDF f_{X,Y}, transformation (U,V) = g(X,Y)

1. If g maps R^2 to R (one output), introduce an auxiliary

variable V = Y (or V = X) to make the map 2-to-2.

2. Find the inverse: (X,Y) = g^{-1}(U,V) = (h_1(U,V), h_2(U,V)).

3. Compute the Jacobian matrix of g^{-1} and its determinant J.

4. Write f_{U,V}(u,v) = f_{X,Y}(h_1(u,v), h_2(u,v)) * |J|.

5. If an auxiliary variable was introduced in step 1,

marginalize: f_U(u) = integral of f_{U,V}(u,v) dv.

Output: f_U(u) (or the joint f_{U,V})

Example: Rayleigh Distribution from Two Gaussians

Let $X, Y$ be independent $\mathcal{N}(0, \sigma^2)$ . Define the polar coordinates $R = \sqrt{X^2 + Y^2}$ and $\Theta = \arctan(Y/X)$ . Find the joint PDF of $(R, \Theta)$ and the marginal PDF of $R$ .

Solution

Inverse transformation

$(X, Y) = (R\cos\Theta,\, R\sin\Theta)$ , so $h_1(r,\theta) = r\cos\theta$ and $h_2(r,\theta) = r\sin\theta$ .

Jacobian

$J = \det\begin{pmatrix} \cos\theta & -r\sin\theta \\ \sin\theta & r\cos\theta \end{pmatrix} = r.$ $

Joint PDF

$f_{X,Y}(r\cos\theta, r\sin\theta) = \frac{1}{2\pi\sigma^2}\exp\!\left(-\frac{r^2}{2\sigma^2}\right).KATEXPLACEHOLDER0ENDf_{R,\Theta}(r, \theta) = \frac{r}{2\pi\sigma^2}\exp\!\left(-\frac{r^2}{2\sigma^2}\right), \quad r \ge 0,\; \theta \in [0, 2\pi).$ $

Marginals

$\Theta$ is uniform on $[0, 2\pi)$ (so $f_\Theta(\theta) = 1/(2\pi)$ ), and

$f_{R}(r) = \frac{r}{\sigma^2}\exp\!\left(-\frac{r^2}{2\sigma^2}\right), \quad r \ge 0.$

This is the Rayleigh distribution with parameter $\sigma$ . Moreover, $R$ and $\Theta$ are independent — a special property of the isotropic Gaussian.

Theorem: Convolution Formula for Independent Sums

Let $X$ and $Y$ be independent continuous RVs with PDFs $f_{X}$ and $f_{Y}$ . Then $Z = X + Y$ has PDF

$f_{Z}(z) = (f_{X} * f_{Y})(z) = \int_{-\infty}^{\infty} f_{X}(u)\,f_{Y}(z - u)\,du,$

where $*$ denotes convolution.

To find the density of the sum at a point $z$ , we sweep over all ways of decomposing $z = u + (z - u)$ and weight each decomposition by the product of the individual densities. The convolution integral performs this sweep.

Proof

Auxiliary variable method

Let $U = X$ , $V = X + Y$ . The inverse is $X = U$ , $Y = V - U$ . The Jacobian is

$J = \det\begin{pmatrix} 1 & 0 \\ -1 & 1 \end{pmatrix} = 1.$

Joint PDF of $(U,V)$

$f_{U,V}(u, v) = f_{X,Y}(u, v - u) \cdot 1 = f_{X}(u)\,f_{Y}(v - u),$ $

using independence.

Marginalize

$f_{V}(v) = \int_{-\infty}^{\infty} f_{X}(u)\,f_{Y}(v - u)\,du = (f_{X} * f_{Y})(v). \quad\blacksquare$ $

Example: Sum of Two Independent Exponentials

Let $X_1 \sim \text{Exp}(\lambda)$ and $X_2 \sim \text{Exp}(\lambda)$ be independent. Find the PDF of $Z = X_1 + X_2$ .

Solution

Apply convolution

For $z \ge 0$ :

$f_{Z}(z) = \int_0^z \lambda e^{-\lambda u} \cdot \lambda e^{-\lambda(z-u)}\,du = \lambda^2 e^{-\lambda z} \int_0^z du = \lambda^2 z\, e^{-\lambda z}.$

Identify the distribution

This is the $\text{Gamma}(2, \lambda)$ density. More generally, the sum of $n$ independent $\text{Exp}(\lambda)$ RVs has a $\text{Gamma}(n, \lambda)$ distribution — a fact we use extensively in queueing theory and order statistics.

Example: Sum of Independent Gaussians

Let $X \sim \mathcal{N}(\mu_1, \sigma_1^2)$ and $Y \sim \mathcal{N}(\mu_2, \sigma_2^2)$ be independent. Find the distribution of $Z = X + Y$ .

Solution

Convolution of Gaussians

The convolution of two Gaussian densities is Gaussian. By completing the square in the convolution integral (or by using moment generating functions, which factor under independence):

$Z \sim \mathcal{N}(\mu_1 + \mu_2,\; \sigma_1^2 + \sigma_2^2).$

Significance

This closure under convolution is one of the most important properties of the Gaussian family. It implies that the sum of any number of independent Gaussian RVs is Gaussian — the foundation for analyzing linear systems driven by Gaussian noise.

Convolution of Two Densities

Choose two distribution families and watch the convolution integral sweep out the density of their sum. The sliding density $f_{Y}(z - u)$ is shown in red, and the area under the product gives $f_{Z}(z)$ .

Parameters

Distribution of

X

Distribution of

Y

z

1

The Jacobian Method for Transformations

Step-by-step visualization of the Jacobian transformation applied to the polar coordinate transformation

(X,Y) \to (R, \Theta)

, showing how area elements are stretched by the factor

|J| = r

.

Convolution

The operation $(f_{X} * f_{Y})(z) = \int f_{X}(u)\,f_{Y}(z-u)\,du$ that gives the PDF of the sum of two independent random variables.

Jacobian

The determinant of the matrix of partial derivatives of a transformation's inverse. It measures the local scaling of area (or volume) elements under the transformation.

Related: Convolution

Common Mistake: Forgetting the Jacobian

Mistake:

Writing $f_{U,V}(u,v) = f_{X,Y}(h_1(u,v), h_2(u,v))$ without the Jacobian factor.

Correction:

The Jacobian $|J_{g^{-1}}|$ is mandatory. Without it, the density does not integrate to 1. A useful sanity check: always verify that $\int\!\!\int f_{U,V}(u,v)\,du\,dv = 1$ .

🔧Engineering Note

Computing Convolutions via FFT

In numerical simulations, evaluating the convolution integral directly is $O(N^2)$ for $N$ sample points. A much faster approach exploits the convolution theorem: $f_{Z} = f_{X} * f_{Y}$ corresponds to pointwise multiplication in the frequency domain. Using the FFT, one computes $\mathcal{F}\{f_{X}\} \cdot \mathcal{F}\{f_{Y}\}$ and then inverse-transforms, achieving $O(N \log N)$ complexity. This is the standard technique for computing the distribution of sums in signal processing and communications simulation.

Key Takeaway

The Jacobian transformation formula gives the joint density of any invertible function of two RVs. For sums of independent RVs, the density is the convolution of the individual densities: $f_{X+Y} = f_{X} * f_{Y}$ . The Gaussian family is closed under convolution — sums of independent Gaussians are Gaussian.

Functions of Two Random Variables

Why Functions of Multiple RVs?

Theorem: Jacobian Transformation Formula

From the multivariable change of variables

Identify the density

Jacobian Method — Step by Step

Example: Rayleigh Distribution from Two Gaussians

Inverse transformation

Jacobian

Joint PDF

Marginals

Theorem: Convolution Formula for Independent Sums

Auxiliary variable method

Joint PDF of $(U,V)$

Marginalize

Example: Sum of Two Independent Exponentials

Apply convolution

Identify the distribution

Example: Sum of Independent Gaussians

Convolution of Gaussians

Significance

Convolution of Two Densities

Parameters

The Jacobian Method for Transformations

Convolution

Jacobian

Common Mistake: Forgetting the Jacobian

Computing Convolutions via FFT

Key Takeaway