Ferkans — Interactive Telecom Tutor

Why the Poisson Distribution?

The binomial distribution $\text{Bin}(n, p)$ is the natural model for counting successes in $n$ independent trials. But when $n$ is large and $p$ is small — so that the expected count $\lambda = np$ is moderate — the binomial PMF is numerically unwieldy (large factorials). The Poisson distribution provides an elegant and accurate approximation that depends on the single parameter $\lambda$ . More than a mere computational convenience, the Poisson distribution is the natural model for rare events: the number of packet arrivals in a time slot, the number of bit errors in a long codeword, the number of active users in a massive access system.

Definition:
Poisson Distribution

A discrete random variable $X$ has the Poisson distribution with parameter $\lambda > 0$ , written $X \sim \text{Poisson}(\lambda)$ , if: $P(k) = \mathbb{P}(X = k) = e^{-\lambda} \frac{\lambda^k}{k!}, \qquad k = 0, 1, 2, \ldots$

The PMF sums to 1 because $\sum_{k=0}^{\infty} \lambda^k / k! = e^{\lambda}$ . The mean and variance are both equal to $\lambda$ : $\mathbb{E}[X] = \text{Var}(X) = \lambda$ .

,

Theorem: Mean and Variance of the Poisson Distribution

If $X \sim \text{Poisson}(\lambda)$ , then $\mathbb{E}[X] = \lambda$ and $\text{Var}(X) = \lambda$ .

The equality of mean and variance is a distinctive fingerprint of the Poisson distribution. In data analysis, if the sample variance of count data is close to the sample mean, a Poisson model is a natural first choice.

Proof

Compute $\mathbb{E}[X]$

$\mathbb{E}[X] = \sum_{k=0}^{\infty} k \cdot e^{-\lambda}\frac{\lambda^k}{k!} = \lambda e^{-\lambda} \sum_{k=1}^{\infty} \frac{\lambda^{k-1}}{(k-1)!} = \lambda e^{-\lambda} \cdot e^{\lambda} = \lambda$ $

Compute $\mathbb{E}[X(X-1)]$

$\mathbb{E}[X(X-1)] = \sum_{k=2}^{\infty} k(k-1) e^{-\lambda}\frac{\lambda^k}{k!} = \lambda^2 e^{-\lambda}\sum_{k=2}^{\infty}\frac{\lambda^{k-2}}{(k-2)!} = \lambda^2$ $

Derive the variance

$\text{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2 = \mathbb{E}[X(X-1)] + \mathbb{E}[X] - (\mathbb{E}[X])^2 = \lambda^2 + \lambda - \lambda^2 = \lambda \qquad \square$ $

Theorem: Poisson Limit Theorem

Let $X_n \sim \text{Bin}(n, p_n)$ where $n p_n \to \lambda > 0$ as $n \to \infty$ . Then for every fixed $k \geq 0$ : $\lim_{n \to \infty} \binom{n}{k} p_n^k (1 - p_n)^{n-k} = e^{-\lambda} \frac{\lambda^k}{k!}$ That is, $X_n$ converges in distribution to $\text{Poisson}(\lambda)$ .

When $n$ is large and $p$ is small, each trial is an almost-never event but there are many of them. The total count depends only on the product $\lambda = np$ . The Poisson distribution captures this regime: it is the distribution of the total count of many independent rare events.

Proof

Write the binomial PMF

Set $p_n = \lambda/n$ . Then: $\binom{n}{k}p_n^k(1 - p_n)^{n-k} = \frac{n!}{k!(n-k)!} \cdot \frac{\lambda^k}{n^k} \cdot \left(1 - \frac{\lambda}{n}\right)^{n-k}$

Separate the factors

$= \frac{\lambda^k}{k!} \cdot \underbrace{\frac{n(n-1)\cdots(n-k+1)}{n^k}}_{\to 1} \cdot \underbrace{\left(1 - \frac{\lambda}{n}\right)^n}_{\to e^{-\lambda}} \cdot \underbrace{\left(1 - \frac{\lambda}{n}\right)^{-k}}_{\to 1}$ $

Take the limit

As $n \to \infty$ with $k$ fixed:

$\frac{n(n-1)\cdots(n-k+1)}{n^k} = \prod_{j=0}^{k-1}(1 - j/n) \to 1$ .
$(1 - \lambda/n)^n \to e^{-\lambda}$ (definition of $e$ ).
$(1 - \lambda/n)^{-k} \to 1$ .

Therefore: $\lim_{n\to\infty}\binom{n}{k}p_n^k(1-p_n)^{n-k} = e^{-\lambda}\frac{\lambda^k}{k!} \qquad \square$

, ,

Example: Typographical Errors

A 500-page book has 200 typographical errors scattered uniformly at random. What is the probability that a given page has no errors? Exactly 1 error?

Solution

Model

Each error independently lands on any of the 500 pages with probability $1/500$ . The number of errors on a given page is $X \sim \text{Bin}(200, 1/500)$ , with $\lambda = np = 200/500 = 0.4$ .

Poisson approximation

$\mathbb{P}(X = 0) \approx e^{-0.4} \approx 0.6703$ . $\mathbb{P}(X = 1) \approx 0.4 \cdot e^{-0.4} \approx 0.2681$ .

Exact binomial (for comparison)

$\mathbb{P}(X = 0) = (499/500)^{200} \approx 0.6703$ . The Poisson approximation agrees to four decimal places — the approximation is excellent when $n = 200$ and $p = 0.002$ .

Theorem: Le Cam's Inequality

Let $X_1, \ldots, X_n$ be independent Bernoulli random variables with $\mathbb{P}(X_i = 1) = p_i$ . Let $W = \sum_{i=1}^{n} X_i$ and $\lambda = \sum_{i=1}^{n} p_i$ . Then: $d_{\mathrm{TV}}(\mathcal{L}(W), \text{Poisson}(\lambda)) \leq \sum_{i=1}^{n} p_i^2$ where $d_{\mathrm{TV}}$ denotes total variation distance: $d_{\mathrm{TV}}(P, Q) = \frac{1}{2}\sum_{k=0}^{\infty} |P(k) - Q(k)|$ .

The bound says the Poisson approximation is accurate when each individual $p_i$ is small, even if the $p_i$ are not all equal. The total variation distance is at most $\sum p_i^2$ , which is small when each $p_i \ll 1$ . For the homogeneous case $p_i = p$ , the bound becomes $np^2 = \lambda p$ , which vanishes as $p \to 0$ with $\lambda$ fixed.

Proof

Coupling argument (sketch)

The proof constructs a coupling between $W$ and a Poisson random variable $Z \sim \text{Poisson}(\lambda)$ by replacing each Bernoulli $X_i$ with an independent Poisson $Y_i \sim \text{Poisson}(p_i)$ . Since $\text{Poisson}(p_i)$ assigns probability $e^{-p_i} \approx 1 - p_i$ to 0 and $p_i e^{-p_i} \approx p_i$ to 1, the total variation distance between $\text{Bernoulli}(p_i)$ and $\text{Poisson}(p_i)$ is at most $p_i^2$ .

Sum the contributions

By the triangle inequality for total variation and the additivity of independent sums under Poisson, the total variation distance between $\mathcal{L}(W)$ and $\text{Poisson}(\lambda)$ is at most $\sum_{i=1}^{n} p_i^2$ . $\qquad \square$

,

Le Cam for the Homogeneous Case

When $p_i = p$ for all $i$ , Le Cam's inequality gives: $d_{\mathrm{TV}}(\text{Bin}(n, p), \text{Poisson}(np)) \leq np^2 = \lambda p$ So if $\lambda = np$ is moderate and $p$ is small (equivalently, $n$ is large), the Poisson approximation is accurate. For instance, with $n = 1000$ and $p = 0.005$ ( $\lambda = 5$ ), the bound is $5 \times 0.005 = 0.025$ : the total variation distance is at most 2.5%.

Poisson Approximation to the Binomial

Compare the binomial PMF $\text{Bin}(n, \lambda/n)$ to the Poisson PMF $\text{Poisson}(\lambda)$ and observe convergence as $n$ grows. The total variation distance is displayed.

Parameters

\lambda

(rate)5

n

(number of trials)20

Birthday Problem: Collision Probability

The birthday problem asks: in a group of $k$ people, what is the probability that at least two share a birthday? This is a Poisson-approximable rare event. Compare the exact formula with the Poisson approximation.

Parameters

n

(number of possible birthdays)365

k_{\\max}

(max group size)60

Example: Prussian Horse Kicks (the Classic Example)

Ladislaus Bortkiewicz (1898) collected data on deaths from horse kicks in the Prussian army: 14 corps observed over 20 years (280 corps-years), with a total of 196 deaths. Test whether the Poisson model fits.

Solution

Estimate the rate

$\hat{\lambda} = 196/280 = 0.7$ deaths per corps per year.

Expected frequencies

Under $\text{Poisson}(0.7)$ :

$\mathbb{P}(X = 0) = e^{-0.7} \approx 0.4966$ $\Rightarrow$ 139 expected.
$\mathbb{P}(X = 1) \approx 0.3476$ $\Rightarrow$ 97.3 expected.
$\mathbb{P}(X = 2) \approx 0.1217$ $\Rightarrow$ 34.1 expected.
$\mathbb{P}(X \geq 3) \approx 0.0341$ $\Rightarrow$ 9.6 expected.

Compare with observed data

Observed: 144, 91, 32, 13. The Poisson model fits remarkably well — this was one of the first empirical validations of the Poisson distribution for rare events.

Example: Packet Arrivals in a Time Slot

A router receives packets from $n = 10{,}000$ independent sources, each transmitting in a given time slot with probability $p = 5 \times 10^{-4}$ . What is the probability that at most 3 packets arrive?

Solution

Compute $\lambda$

$\lambda = np = 10{,}000 \times 5 \times 10^{-4} = 5$ .

Poisson approximation

$\mathbb{P}(X \leq 3) = \sum_{k=0}^{3} e^{-5}\frac{5^k}{k!} = e^{-5}\left(1 + 5 + \frac{25}{2} + \frac{125}{6}\right) \approx 0.2650$ $

Le Cam bound

$d_{\mathrm{TV}} \leq np^2 = 10{,}000 \times (5 \times 10^{-4})^2 = 0.0025$ . The approximation error is at most 0.25% in total variation — excellent for engineering purposes.

Why This Matters: Poisson Model for Massive Access

In massive machine-type communication (mMTC), a large number $N$ of devices (say $N = 10^6$ ) are registered, but in any given time slot each device transmits with a small probability $p$ (say $p = 10^{-4}$ ). The number of active devices $K$ is then well-modeled by $\text{Poisson}(\lambda)$ with $\lambda = Np = 100$ . This Poisson model is the foundation of the unsourced random access framework developed by Polyanskiy (2017), which is a major research direction in the CommIT group.

🔧Engineering Note

When to Use Poisson vs. Exact Binomial

In modern computing, evaluating the binomial PMF is trivial for moderate $n$ (say $n \leq 10^6$ ). The Poisson approximation is valuable when: (1) $n$ is unknown or variable but $\lambda$ can be estimated; (2) the model is inherently Poisson (e.g., arrivals from a Poisson process); (3) you need closed-form expressions for further analysis (e.g., deriving capacity formulas that involve sums over Poisson-distributed counts).

Historical Note: Siméon Denis Poisson and the Law of Small Numbers

19th–20th century

Siméon Denis Poisson published his treatise Recherches sur la probabilité des jugements in 1837, where the distribution that bears his name appears as a limit of the binomial. The catchy name "law of small numbers" (Gesetz der kleinen Zahlen) was coined by Ladislaus Bortkiewicz in 1898, who demonstrated the empirical fit on data ranging from Prussian cavalry deaths to children's suicides. The Poisson distribution later became the cornerstone of queueing theory through the work of A. K. Erlang on telephone traffic (1909).

Historical Note: Lucien Le Cam and Approximation Theory

20th century

Lucien Le Cam proved his celebrated inequality in 1960 while working at UC Berkeley. The result was a byproduct of his broader program on the approximation of statistical experiments. Le Cam's inequality remains the standard tool for bounding the error of the Poisson approximation and has been refined by Stein, Chen, and others into the powerful "Stein-Chen method" for Poisson approximation of dependent rare events.

Poisson Limit: $\\text{Bin}(n, \\lambda/n) \\to \\text{Poisson}(\\lambda)$

Watch the binomial PMF bars morph into the Poisson PMF as

n

increases from 5 to 500 with

\lambda = 5

fixed.

As

n \\to \\infty

with

np = \\lambda

fixed, the binomial bars converge to the Poisson dots.

Theorem: Sum of Independent Poisson Random Variables

If $X \sim \text{Poisson}(\lambda_1)$ and $Y \sim \text{Poisson}(\lambda_2)$ are independent, then $X + Y \sim \text{Poisson}(\lambda_1 + \lambda_2)$ .

Poisson counts are closed under addition of independent components. If packets arrive from two independent sources at rates $\lambda_1$ and $\lambda_2$ , the total arrival count is Poisson with rate $\lambda_1 + \lambda_2$ . This superposition property is fundamental to queueing theory.

Proof

Compute the PMF of $X + Y$

$\mathbb{P}(X + Y = k) = \sum_{j=0}^{k}\mathbb{P}(X = j)\mathbb{P}(Y = k-j) = \sum_{j=0}^{k} e^{-\lambda_1}\frac{\lambda_1^j}{j!} \cdot e^{-\lambda_2}\frac{\lambda_2^{k-j}}{(k-j)!}$ $

Simplify using the binomial theorem

$= \frac{e^{-(\lambda_1+\lambda_2)}}{k!} \sum_{j=0}^{k}\binom{k}{j}\lambda_1^j \lambda_2^{k-j} = \frac{e^{-(\lambda_1+\lambda_2)}(\lambda_1+\lambda_2)^k}{k!}$ $This is the PMF of$ \text{Poisson}(\lambda_1 + \lambda_2) $.$ \qquad \square$

Common Mistake: Mean Equals Variance Does Not Imply Poisson

Mistake:

Concluding that data is Poisson just because the sample mean and sample variance are approximately equal.

Correction:

The condition $\mathbb{E}[X] = \text{Var}(X)$ is necessary but not sufficient for a Poisson distribution. Many other distributions (e.g., certain negative binomial or compound Poisson distributions) can also have equal mean and variance. A proper goodness-of-fit test (e.g., chi-squared) is needed.

Common Mistake: Applying Poisson Approximation When $p$ Is Not Small

Mistake:

Using the Poisson approximation $\text{Bin}(n, p) \approx \text{Poisson}(np)$ when $p$ is, say, 0.3 and $n = 50$ .

Correction:

Le Cam's bound gives $d_{\mathrm{TV}} \leq np^2 = 50 \times 0.09 = 4.5$ , which is useless (total variation is at most 1). The Poisson approximation requires $p \ll 1$ . For moderate $p$ and large $n$ , use the normal approximation (CLT) instead.

Quick Check

If $X \sim \\text{Bin}(1000, 0.003)$ , which distribution best approximates $X$ ?

$\text{Poisson}(3)$

$\text{Poisson}(0.003)$

$\mathcal{N}(3, 3)$

$\\text{Poisson}(1000)$

Correction:

\text{Poisson}(3)

$\lambda = np = 1000 \times 0.003 = 3$ , and $p = 0.003$ is small, so the Poisson approximation is excellent.

Quick Check

Le Cam's inequality bounds the total variation distance between the distribution of a sum of independent Bernoullis and a Poisson by:

$\sum_i p_i$

$\\max_i p_i$

$\sum_i p_i^2$

$(\sum_i p_i)^2$

Correction:

\sum_i p_i^2

Correct. The total variation distance is at most $\sum_i p_i^2$ .

🎓CommIT Contribution(2017)

Unsourced Random Access and the Poisson User Model

Y. Polyanskiy, G. Caire — IEEE International Symposium on Information Theory (ISIT)

The unsourced random access paradigm introduced by Polyanskiy (2017) models the number of active users in a massive IoT system as a Poisson random variable. This elegant abstraction removes the need to identify individual users and focuses on the fundamental limits of communicating a list of messages. The CommIT group at TU Berlin has been a leading contributor to practical coding schemes for this setting, including slotted transmission and tree-based decoding algorithms.

massive-accesspoisson-modelrandom-access

Poisson distribution

A discrete distribution on $\{0, 1, 2, \ldots\}$ with parameter $\lambda > 0$ and PMF $\mathbb{P}(X = k) = e^{-\lambda}\lambda^k/k!$ . Both the mean and variance equal $\lambda$ .

Related: binomial distribution

total variation distance

For discrete distributions $P$ and $Q$ on the same space: $d_{\mathrm{TV}}(P, Q) = \frac{1}{2}\sum_x |P(x) - Q(x)|$ . It equals the maximum difference $\max_A |P(A) - Q(A)|$ over all events $A$ .

Related: Poisson distribution

Key Takeaway

The Poisson limit theorem shows that $\text{Bin}(n, \lambda/n) \to \text{Poisson}(\lambda)$ as $n \to \infty$ : the Poisson distribution is the natural model for the count of many independent rare events. Le Cam's inequality $d_{\mathrm{TV}} \leq \sum p_i^2$ quantifies the approximation error. The Poisson distribution's additivity and single-parameter simplicity make it indispensable for modeling packet arrivals, interference events, and user activity in telecommunications.

The Poisson Approximation

Why the Poisson Distribution?

Definition: Poisson Distribution

Theorem: Mean and Variance of the Poisson Distribution

Compute $\mathbb{E}[X]$

Compute $\mathbb{E}[X(X-1)]$

Derive the variance

Theorem: Poisson Limit Theorem

Write the binomial PMF

Separate the factors

Take the limit

Example: Typographical Errors

Model

Poisson approximation

Exact binomial (for comparison)

Theorem: Le Cam's Inequality

Coupling argument (sketch)

Sum the contributions

Le Cam for the Homogeneous Case

Poisson Approximation to the Binomial

Parameters

Birthday Problem: Collision Probability

Parameters

Example: Prussian Horse Kicks (the Classic Example)

Estimate the rate

Expected frequencies

Compare with observed data

Example: Packet Arrivals in a Time Slot

Compute $\lambda$

Poisson approximation

Le Cam bound

Why This Matters: Poisson Model for Massive Access

When to Use Poisson vs. Exact Binomial

Historical Note: Siméon Denis Poisson and the Law of Small Numbers

Historical Note: Lucien Le Cam and Approximation Theory

Poisson Limit: textBin(n,lambda/n)totextPoisson(lambda)\\text{Bin}(n, \\lambda/n) \\to \\text{Poisson}(\\lambda)textBin(n,lambda/n)totextPoisson(lambda)

Theorem: Sum of Independent Poisson Random Variables

Compute the PMF of $X + Y$

Simplify using the binomial theorem

Common Mistake: Mean Equals Variance Does Not Imply Poisson

Common Mistake: Applying Poisson Approximation When ppp Is Not Small

Quick Check

Quick Check

Unsourced Random Access and the Poisson User Model

Poisson distribution

total variation distance

Key Takeaway

Definition:
Poisson Distribution

Poisson Limit: $\\text{Bin}(n, \\lambda/n) \\to \\text{Poisson}(\\lambda)$

Common Mistake: Applying Poisson Approximation When $p$ Is Not Small