Ferkans — Interactive Telecom Tutor

Transforms as the Right Language

The characteristic function (CF) provides the most elegant and general route to the multivariate Gaussian. It exists for every distribution (unlike the MGF), it characterizes the distribution uniquely, and it makes affine transformation results almost trivial to prove. Moreover, the CF of the Gaussian is itself a Gaussian in the frequency domain — a beautiful parallel to the fact that the Fourier transform of a Gaussian is a Gaussian.

Theorem: MGF and CF of the Multivariate Gaussian

Let $\mathbf{X} \sim \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})$ . The moment generating function (where it exists) is

$M_{\mathbf{X}}(\mathbf{t}) = \mathbb{E}\!\left[e^{\mathbf{t}^T \mathbf{X}}\right] = \exp\!\left(\mathbf{t}^T \boldsymbol{\mu} + \frac{1}{2}\mathbf{t}^T \boldsymbol{\Sigma}\, \mathbf{t}\right), \quad \mathbf{t} \in \mathbb{R}^n.$

The characteristic function is

$\phi_{\mathbf{X}}(\boldsymbol{\omega}) = \mathbb{E}\!\left[e^{j\boldsymbol{\omega}^T \mathbf{X}}\right] = \exp\!\left(j\boldsymbol{\omega}^T \boldsymbol{\mu} - \frac{1}{2}\boldsymbol{\omega}^T \boldsymbol{\Sigma}\, \boldsymbol{\omega}\right), \quad \boldsymbol{\omega} \in \mathbb{R}^n.$

The CF is obtained from the MGF by replacing $\mathbf{t}$ with $j\boldsymbol{\omega}$ . The quadratic form in the exponent is the Fourier-domain representation of the covariance structure.

Proof

Reduce to the scalar case

For any $\mathbf{t}$ , define $Z = \mathbf{t}^T \mathbf{X}$ . By the affine transformation theorem, $Z \sim \mathcal{N}(\mathbf{t}^T\boldsymbol{\mu}, \mathbf{t}^T\boldsymbol{\Sigma}\mathbf{t})$ .

Apply the scalar MGF

The MGF of a scalar Gaussian $Z \sim \mathcal{N}(m, v)$ is $M_Z(s) = \exp(ms + vs^2/2)$ . Evaluating at $s = 1$ :

$M_{\mathbf{X}}(\mathbf{t}) = \mathbb{E}[e^Z] = \exp\!\left(\mathbf{t}^T\boldsymbol{\mu} + \tfrac{1}{2}\mathbf{t}^T\boldsymbol{\Sigma}\,\mathbf{t}\right).$

CF follows by substitution

Replace $\mathbf{t} \to j\boldsymbol{\omega}$ : $\phi_{\mathbf{X}}(\boldsymbol{\omega}) = M_{\mathbf{X}}(j\boldsymbol{\omega}) = \exp(j\boldsymbol{\omega}^T\boldsymbol{\mu} - \tfrac{1}{2}\boldsymbol{\omega}^T\boldsymbol{\Sigma}\,\boldsymbol{\omega})$ .

The CF Defines the Gaussian Even When the PDF Does Not Exist

When $\boldsymbol{\Sigma}$ is singular ( $\det(\boldsymbol{\Sigma}) = 0$ ), the multivariate Gaussian PDF does not exist in $\mathbb{R}^n$ . However, the characteristic function $\phi(\boldsymbol{\omega}) = \exp(j\boldsymbol{\omega}^T\boldsymbol{\mu} - \tfrac{1}{2}\boldsymbol{\omega}^T\boldsymbol{\Sigma}\,\boldsymbol{\omega})$ is always well-defined for any PSD $\boldsymbol{\Sigma}$ , and it uniquely determines the distribution. This is the most general definition of the multivariate Gaussian.

Example: Sum of Independent Gaussians via MGF

Let $X_1, \ldots, X_n$ be independent with $X_i \sim \mathcal{N}(\mu_i, \sigma_i^2)$ . Use the MGF to find the distribution of $S = \sum_{i=1}^n X_i$ .

Solution

Product of MGFs

Since $X_1, \ldots, X_n$ are independent,

$M_S(t) = \prod_{i=1}^n M_{X_i}(t) = \prod_{i=1}^n \exp\!\left(\mu_i t + \frac{\sigma_i^2 t^2}{2}\right) = \exp\!\left(t\sum_i \mu_i + \frac{t^2}{2}\sum_i \sigma_i^2\right).$

Identify the distribution

This is the MGF of $\mathcal{N}\!\left(\sum_i \mu_i, \sum_i \sigma_i^2\right)$ . The sum of independent Gaussians is Gaussian.

Definition:
Joint Gaussianity via Characteristic Function

A random vector $\mathbf{X} \in \mathbb{R}^n$ is jointly Gaussian if its characteristic function has the form

$\phi_{\mathbf{X}}(\boldsymbol{\omega}) = \exp\!\left(j\boldsymbol{\omega}^T\boldsymbol{\mu} - \frac{1}{2}\boldsymbol{\omega}^T \boldsymbol{\Sigma}\,\boldsymbol{\omega}\right)$

for some $\boldsymbol{\mu} \in \mathbb{R}^n$ and PSD matrix $\boldsymbol{\Sigma} \in \mathbb{R}^{n \times n}$ .

Equivalently, $\mathbf{X}$ is jointly Gaussian iff every linear combination $\mathbf{a}^T\mathbf{X}$ is a (possibly degenerate) scalar Gaussian.

Characteristic function

The function $\phi_{\mathbf{X}}(\boldsymbol{\omega}) = \mathbb{E}[e^{j\boldsymbol{\omega}^T\mathbf{X}}]$ . It always exists, uniquely determines the distribution, and is the Fourier transform of the PDF (when the PDF exists).

Moment Generating and Characteristic Functions