Ferkans — Interactive Telecom Tutor

What Happens When You Square and Multiply Gaussians

Many quantities in statistics and engineering are quadratic functions of Gaussian vectors: the sample variance, the SNR, the test statistic in hypothesis testing, the eigenvalues of sample covariance matrices. This section develops the distributions that arise from such quadratic operations: the chi-squared for sums of squared Gaussians, the non-central chi-squared when the Gaussians have non-zero means, and the Wishart distribution for the sample covariance matrix.

Definition:
Chi-Squared Distribution

Let $Z_1, \ldots, Z_n$ be i.i.d. $\mathcal{N}(0, 1)$ . The random variable

$Q = \sum_{i=1}^n Z_i^2$

has the chi-squared distribution with $n$ degrees of freedom, written $Q \sim \chi^2_n$ . Its PDF is

$f_Q(q) = \frac{q^{n/2 - 1} e^{-q/2}}{2^{n/2}\,\Gamma(n/2)}, \quad q > 0.$

The mean is $\mathbb{E}[Q] = n$ and the variance is $\text{Var}(Q) = 2n$ .

The chi-squared with $n$ degrees of freedom is a Gamma distribution with shape $n/2$ and rate $1/2$ : $\chi^2_n = \text{Gamma}(n/2, 1/2)$ .

Definition:
Non-Central Chi-Squared Distribution

Let $Z_i \sim \mathcal{N}(\mu_i, 1)$ be independent. Then

$Q = \sum_{i=1}^n Z_i^2 \sim \chi^2_n(\delta),$

where $\delta = \sum_{i=1}^n \mu_i^2$ is the non-centrality parameter. When $\delta = 0$ , this reduces to the central chi-squared.

Theorem: Distribution of Quadratic Forms

Let $\mathbf{X} \sim \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})$ with $\boldsymbol{\Sigma} \succ 0$ . The quadratic form

$Q = (\mathbf{X} - \boldsymbol{\mu})^T \boldsymbol{\Sigma}^{-1} (\mathbf{X} - \boldsymbol{\mu}) \sim \chi^2_n.$

More generally, if $\mathbf{A}$ is a symmetric $n \times n$ matrix, then $\mathbf{X}^T \mathbf{A} \mathbf{X}$ is a weighted sum of independent (possibly non-central) chi-squared random variables, with weights given by the eigenvalues of $\boldsymbol{\Sigma}^{1/2}\mathbf{A}\boldsymbol{\Sigma}^{1/2}$ .

Proof

Whiten

Let $\mathbf{W} = \boldsymbol{\Sigma}^{-1/2}(\mathbf{X} - \boldsymbol{\mu}) \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$ . Then $Q = \mathbf{W}^T\mathbf{W} = \sum_{i=1}^n W_i^2$ , a sum of $n$ independent squared standard Gaussians.

Identify as chi-squared

By definition, $\sum_{i=1}^n W_i^2 \sim \chi^2_n$ .

Chi-Squared Distribution Explorer

Explore how the chi-squared PDF changes with the degrees of freedom $n$ . Observe that for large $n$ , the distribution becomes approximately Gaussian (by the CLT).

Parameters

Degrees of freedom

n

5

Definition:
Wishart Distribution

Let $\mathbf{X}_1, \ldots, \mathbf{X}_m$ be i.i.d. $\mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$ random vectors in $\mathbb{R}^n$ . The random matrix

$\mathbf{W} = \sum_{i=1}^m \mathbf{X}_i \mathbf{X}_i^T$

has the Wishart distribution $\mathcal{W}_n(m, \boldsymbol{\Sigma})$ . When $m \geq n$ , $\mathbf{W}$ is positive definite almost surely. The Wishart is the matrix analogue of the chi-squared.

The sample covariance matrix $\hat{\boldsymbol{\Sigma}} = \frac{1}{m-1}\sum_{i=1}^m (\mathbf{X}_i - \bar{\mathbf{X}})(\mathbf{X}_i - \bar{\mathbf{X}})^T$ is a scaled Wishart: $(m-1)\hat{\boldsymbol{\Sigma}} \sim \mathcal{W}_n(m-1, \boldsymbol{\Sigma})$ .

Wishart distribution

The distribution of the sample scatter matrix $\mathbf{W} = \sum_{i=1}^m \mathbf{X}_i\mathbf{X}_i^T$ when the $\mathbf{X}_i$ are i.i.d. Gaussian. The matrix analogue of the chi-squared distribution.

Related: Covariance matrix

Example: Rayleigh Distribution from Bivariate Gaussian

Let $(X, Y) \sim \mathcal{N}(\mathbf{0}, \sigma^2 \mathbf{I}_2)$ . Find the distribution of $R = \sqrt{X^2 + Y^2}$ .

Solution

Distribution of $R^2$

$X/\sigma$ and $Y/\sigma$ are i.i.d. $\mathcal{N}(0,1)$ , so $(X^2 + Y^2)/\sigma^2 \sim \chi^2_2$ . The $\chi^2_2$ distribution is Exponential with rate $1/2$ : $R^2/\sigma^2 \sim \text{Exp}(1/2)$ , i.e., $R^2 \sim \text{Exp}(1/(2\sigma^2))$ .

Transform to $R$

By the change-of-variable formula $f_R(r) = 2r\,f_{R^2}(r^2)$ :

$f_R(r) = \frac{r}{\sigma^2}\exp\!\left(-\frac{r^2}{2\sigma^2}\right), \quad r \geq 0.$

This is the Rayleigh distribution with parameter $\sigma$ .

Engineering significance

The Rayleigh distribution models the envelope of a narrowband signal passing through a rich scattering channel (the "Rayleigh fading" model in wireless communications).

Why This Matters: Rayleigh Fading from the Gaussian Model

In a wireless channel with many scatterers and no line-of-sight component, the received complex baseband coefficient is $h = X + jY$ where $X, Y \sim \mathcal{N}(0, \sigma^2)$ are independent (by the CLT applied to the sum of many scattered paths). The envelope $|h| = \sqrt{X^2 + Y^2}$ is Rayleigh-distributed, and the power $|h|^2$ is exponentially distributed. This is the i.i.d. Rayleigh fading model — the default model for MIMO channels (Book MIMO, Chapter 2) and the starting point for all diversity analysis (Book Telecom, Chapter 10).

⚠️Engineering Note

Sample Covariance Matrix and the Wishart in Practice

In massive MIMO, the base station estimates the channel covariance from $m$ pilot observations: $\hat{\boldsymbol{\Sigma}} = \frac{1}{m}\sum_{i=1}^m \mathbf{h}_i\mathbf{h}_i^H$ . When $m$ is comparable to $n$ (the number of antennas), the sample covariance is a poor estimate of the true covariance — the eigenvalues spread out (the Marchenko-Pastur effect). Understanding the Wishart distribution is essential for analyzing when and how covariance estimation can be trusted. This topic is developed further in the random matrix theory chapter (Chapter 21).

Quick Check

If $\mathbf{X} \sim \mathcal{N}(\boldsymbol{\mu}, \sigma^2\mathbf{I}_4)$ , what is the distribution of $\|\mathbf{X} - \boldsymbol{\mu}\|^2/\sigma^2$ ?

$\chi^2_4$

$\chi^2_3$

$\mathcal{N}(0, 4)$

Exponential with rate $1/2$

Correction:

\chi^2_4

$\|\mathbf{X} - \boldsymbol{\mu}\|^2/\sigma^2 = \sum_{i=1}^4 ((X_i - \mu_i)/\sigma)^2$ , a sum of 4 independent squared standard Gaussians.

Chi-Squared, Wishart, and Related Distributions