Ferkans — Interactive Telecom Tutor

Why the Characteristic Function?

The MGF is a powerful tool, but it has a fatal weakness: it does not always exist. The Cauchy distribution, the log-normal, and many heavy-tailed distributions have infinite MGF for all $t \neq 0$ .

The characteristic function resolves this by replacing $e^{tX}$ with $e^{juX}$ . Since $|e^{juX}| = 1$ for all real $u$ and $X$ , the expectation is always finite. The characteristic function exists for every random variable, without exception.

Definition:
Characteristic Function

The characteristic function (CF) of a random variable $X$ is

$\phi_X(u) = \mathbb{E}[e^{juX}] = \int_{-\infty}^{\infty} e^{jux}\,dF(x), \qquad u \in \mathbb{R},$

where $j = \sqrt{-1}$ . For a continuous random variable with PDF $f(x)$ , this is the Fourier transform of the density with sign convention:

$\phi_X(u) = \mathcal{F}[f](-u).$

The characteristic function always exists and is bounded: $|\phi_X(u)| \leq 1$ for all $u$ , with $\phi_X(0) = 1$ . This is the decisive advantage over the MGF.

Characteristic Function

The function $\phi_X(u) = \mathbb{E}[e^{juX}]$ . It is the Fourier transform of the distribution, always exists, and uniquely determines the distribution of $X$ .

Theorem: Basic Properties of the Characteristic Function

Let $\phi_X(u) = \mathbb{E}[e^{juX}]$ be the characteristic function of $X$ . Then:

$\phi_X(0) = 1$ and $|\phi_X(u)| \leq 1$ for all $u$ .
$\phi_X$ is uniformly continuous on $\mathbb{R}$ .
$\phi_X$ is Hermitian: $\phi_X(-u) = \overline{\phi_X(u)}$ .
$\phi_X$ is non-negative definite: for any reals $u_1, \ldots, u_n$ and complex $a_1, \ldots, a_n$ , $\sum_{i,k} \phi_X(u_i - u_k)\,a_i\,\overline{a_k} \geq 0.$

Proof

Boundedness

$|\phi_X(u)| = |\mathbb{E}[e^{juX}]| \leq \mathbb{E}[|e^{juX}|] = \mathbb{E}[1] = 1$ .

Uniform continuity

$|\phi_X(u+h) - \phi_X(u)| = |\mathbb{E}[e^{juX}(e^{jhX} - 1)]| \leq \mathbb{E}[|e^{jhX} - 1|]$ . By the bounded convergence theorem, the right side tends to $0$ as $h \to 0$ , uniformly in $u$ .

Hermitian symmetry

$\phi_X(-u) = \mathbb{E}[e^{-juX}] = \mathbb{E}[\overline{e^{juX}}] = \overline{\mathbb{E}[e^{juX}]} = \overline{\phi_X(u)}$ .

Non-negative definiteness

$\sum_{i,k} \phi_X(u_i - u_k) a_i \overline{a_k} = \mathbb{E}\!\left[\left|\sum_i a_i e^{ju_iX}\right|^2\right] \geq 0$ .

Theorem: Moments from the Characteristic Function

If $\mathbb{E}[|X|^k] < \infty$ , then $\phi_X$ is $k$ -times differentiable and

$\mathbb{E}[X^k] = j^{-k}\,\phi_X^{(k)}(0).$

Moreover, $\phi_X(u) = \sum_{i=0}^{k} \frac{\mathbb{E}[X^i]}{i!}(ju)^i + o(u^k)$ as $u \to 0$ .

Proof

Taylor expansion of the integrand

$e^{juX} = \sum_{i=0}^{k} \frac{(juX)^i}{i!} + R_k(u, X)$ where $|R_k| \leq \frac{|uX|^{k+1}}{(k+1)!}$ (but we only need $|R_k| \leq 2\frac{|uX|^k}{k!}$ ).

Take expectations

Since $\mathbb{E}[|X|^k] < \infty$ , the dominated convergence theorem justifies taking expectations term-by-term, giving the Taylor approximation. The $k$ -th derivative formula follows by differentiating under the integral and evaluating at $u = 0$ .

Theorem: Analytic Extension from MGF to CF

If $M_X(t) < \infty$ for $|t| < a$ with $a > 0$ , then

$\phi_X(u) = M_X(ju).$

That is, the characteristic function is obtained by evaluating the MGF along the imaginary axis.

The MGF $M_X(t)$ is defined for real $t$ , but when it converges in a strip around the origin, it extends to a complex analytic function. Evaluating at $t = ju$ gives the CF. This means that whenever the MGF exists, you can compute the CF by substitution — no separate calculation is needed.

Proof

Analytic continuation

The function $M(s) = \int e^{sx}\,dF(x)$ converges absolutely for $|\text{Re}(s)| < a$ , which is a vertical strip in the complex plane containing the imaginary axis. By Morera's theorem, $M(s)$ is analytic in this strip. Setting $s = ju$ : $M(ju) = \mathbb{E}[e^{juX}] = \phi_X(u)$ .

Definition:
Convolution-to-Product Property of the CF

If $X$ and $Y$ are independent, then

$\phi_{X+Y}(u) = \phi_X(u) \cdot \phi_Y(u).$

If $Y = aX + b$ , then $\phi_Y(u) = e^{jbu}\,\phi_X(au)$ .

Example: CF of the Gaussian Distribution

Find the characteristic function of $X \sim \mathcal{N}(\mu, \sigma^2)$ .

Solution

Use the analytic extension

From Example EMGF of the Gaussian Distribution, $M_X(t) = \exp(\mu t + \sigma^2 t^2/2)$ . Since this is finite for all $t \in \mathbb{R}$ , we apply Theorem TAnalytic Extension from MGF to CF:

$\phi_X(u) = M_X(ju) = \exp\!\left(j\mu u - \frac{\sigma^2 u^2}{2}\right).$

Interpret

The CF has magnitude $|\phi_X(u)| = e^{-\sigma^2 u^2/2}$ (a Gaussian bell in the frequency domain) and phase $\arg\phi_X(u) = \mu u$ (a linear phase shift encoding the mean). A wider distribution ( $\sigma^2$ large) gives a narrower CF, and vice versa — the uncertainty principle in action.

Definition:
Joint Characteristic Function

For a random vector $\mathbf{X} \in \mathbb{R}^n$ with joint CDF $F_{\mathbf{X}}$ , the joint characteristic function is

$\phi_{\mathbf{X}}(\mathbf{u}) = \mathbb{E}\!\left[e^{j\mathbf{u}^T\mathbf{X}}\right] = \int_{\mathbb{R}^n} e^{j\sum_{i=1}^n u_i x_i}\,dF_{\mathbf{X}}(\mathbf{x}), \qquad \mathbf{u} \in \mathbb{R}^n.$

For a pair $(X, Y)$ : $\phi_{X,Y}(u, v) = \mathbb{E}[e^{j(uX + vY)}]$ . If $X \perp Y$ , then $\phi_{X,Y}(u, v) = \phi_X(u)\,\phi_Y(v)$ .

Characteristic Function: Magnitude and Phase

Explore the characteristic function $\phi_X(u)$ for different distributions. The magnitude $|\phi_X(u)|$ and phase $\arg\phi_X(u)$ are shown separately. Notice how the mean shifts the phase and the variance controls how fast the magnitude decays.

Parameters

Distribution

Mean (or location)0

Scale parameter1

Common Mistake: Fourier Transform Sign Convention

Mistake:

Confusing $\phi_X(u) = \mathbb{E}[e^{juX}]$ with the engineering Fourier transform $\hat{f}(\omega) = \int f(x) e^{-j\omega x}\,dx$ . The sign in the exponent is opposite.

Correction:

The CF uses the $+j$ convention: $\phi_X(u) = \mathcal{F}[f](-u)$ . When using Fourier inversion tables, replace $\omega$ with $-u$ , or equivalently, conjugate the result. In Caire's notation, $j$ (not $i$ ) denotes the imaginary unit.

Quick Check

The Cauchy distribution has CF $\phi_X(u) = e^{-|u|}$ . Does its MGF exist for any $t \neq 0$ ?

No — the MGF is infinite for all $t \neq 0$

Yes — the MGF equals $e^{-|t|}$

Yes — for $|t| < 1$

It depends on the location parameter

Correction:

No — the MGF is infinite for all

t \neq 0

The Cauchy distribution has such heavy tails that $\mathbb{E}[e^{tX}] = \infty$ for all $t \neq 0$ . Even $\mathbb{E}[|X|]$ is infinite. The CF exists because $|e^{juX}| = 1$ regardless of the tail behavior.

Quick Check

If $X$ is a real random variable, which of the following is always true about $\phi_X(u)$ ?

$\phi_X(-u) = \overline{\phi_X(u)}$ (Hermitian symmetry)

$\phi_X(u)$ is always real-valued

$|\phi_X(u)| = 1$ for all $u$

$\phi_X(u) > 0$ for all $u$

Correction:

\phi_X(-u) = \overline{\phi_X(u)}

(Hermitian symmetry)

$\phi_X(-u) = \mathbb{E}[e^{-juX}] = \mathbb{E}[\overline{e^{juX}}] = \overline{\phi_X(u)}$ . For real-valued $X$ , the CF has conjugate symmetry, so $|\phi_X(-u)| = |\phi_X(u)|$ and the phase is odd.

The Characteristic Function

Why the Characteristic Function?

Definition: Characteristic Function

Characteristic Function

Theorem: Basic Properties of the Characteristic Function

Boundedness

Uniform continuity

Hermitian symmetry

Non-negative definiteness

Theorem: Moments from the Characteristic Function

Taylor expansion of the integrand

Take expectations

Theorem: Analytic Extension from MGF to CF

Analytic continuation

Definition: Convolution-to-Product Property of the CF

Example: CF of the Gaussian Distribution

Use the analytic extension

Interpret

Definition: Joint Characteristic Function

Characteristic Function: Magnitude and Phase

Parameters

Common Mistake: Fourier Transform Sign Convention

Quick Check

Quick Check

Definition:
Characteristic Function

Definition:
Convolution-to-Product Property of the CF

Definition:
Joint Characteristic Function