Ferkans — Interactive Telecom Tutor

A Toolkit of Named Distributions

A handful of discrete distributions appear so frequently in applications that they have earned their own names. Each captures a different random mechanism: coin flips (Bernoulli/binomial), waiting times (geometric/negative binomial), rare events (Poisson), equal likelihood (discrete uniform), and sampling without replacement (hypergeometric). Knowing these distributions — and recognizing when a problem maps to one of them — saves enormous effort in both analysis and computation.

Definition:
Bernoulli Distribution

$X \sim \text{Bernoulli}(p)$ with $p \in [0, 1]$ :

$P(k) = p^k (1-p)^{1-k}, \quad k \in \{0, 1\}.$

Mean: $\mathbb{E}[X] = p$ .
Variance: $\text{Var}(X) = p(1-p)$ .
MGF: $M_X(t) = (1-p) + pe^t$ .

The Bernoulli distribution models a single trial with two outcomes. It is the building block for the binomial distribution (sum of i.i.d. Bernoulli trials).

Definition:
Binomial Distribution

$X \sim \text{Binomial}(n, p)$ : the number of successes in $n$ independent Bernoulli( $p$ ) trials.

$P(k) = \binom{n}{k} p^k (1-p)^{n-k}, \quad k = 0, 1, \ldots, n.$

Mean: $\mathbb{E}[X] = np$ .
Variance: $\text{Var}(X) = np(1-p)$ .
MGF: $M_X(t) = \left[(1-p) + pe^t\right]^n$ .

,

Theorem: Mean and Variance of the Binomial via Indicators

If $X \sim \text{Binomial}(n, p)$ , then $\mathbb{E}[X] = np$ and $\text{Var}(X) = np(1-p)$ .

Proof

Decompose into indicators

Write $X = \sum_{i=1}^n X_i$ where $X_i \sim \text{Bernoulli}(p)$ are independent. By linearity, $\mathbb{E}[X] = \sum_{i=1}^n \mathbb{E}[X_i] = np$ .

Variance by independence

Since the $X_i$ are independent, $\text{Var}(X) = \sum_{i=1}^n \text{Var}(X_i) = np(1-p)$ . $\blacksquare$

Example: Bit Errors in a Transmitted Block

A transmitter sends a block of $n = 1000$ bits over a binary symmetric channel with bit error probability $p = 0.01$ . Let $X$ be the number of errors. Find $\mathbb{E}[X]$ and $\text{Var}(X)$ , and estimate $\mathbb{P}(X > 15)$ .

Solution

Model

$X \sim \text{Binomial}(1000, 0.01)$ , so $\mathbb{E}[X] = 10$ and $\text{Var}(X) = 9.9$ .

Markov bound

$\mathbb{P}(X > 15) \leq \mathbb{E}[X]/15 = 10/15 \approx 0.667$ (crude).

Poisson approximation

Since $n$ is large and $p$ is small, $X \approx \text{Poisson}(10)$ . Using the Poisson CDF: $\mathbb{P}(X > 15) \approx 1 - \sum_{k=0}^{15} e^{-10} 10^k / k! \approx 0.049$ .

Binomial PMF Explorer

Explore how the binomial PMF changes shape as $n$ and $p$ vary. For large $n$ and small $p$ , the binomial approaches the Poisson. For moderate $p$ and large $n$ , it approaches the Gaussian.

Parameters

n

20

p

0.5

Show Normal approximation

Show Poisson approximation

Definition:
Geometric Distribution

$X \sim \text{Geometric}(p)$ : the number of trials until the first success.

$P(k) = (1-p)^{k-1} p, \quad k = 1, 2, 3, \ldots$

Mean: $\mathbb{E}[X] = 1/p$ .
Variance: $\text{Var}(X) = (1-p)/p^2$ .
MGF: $M_X(t) = \frac{pe^t}{1 - (1-p)e^t}$ for $t < -\ln(1-p)$ .

Some authors define the geometric distribution as the number of failures before the first success, giving $P(k) = (1-p)^k p$ for $k = 0, 1, 2, \ldots$ and mean $(1-p)/p$ . We follow the convention of Caire's FSP course, counting the trial on which the first success occurs.

,

Theorem: Memoryless Property of the Geometric Distribution

If $X \sim \text{Geometric}(p)$ , then for all $m, n \geq 1$ :

$\mathbb{P}(X > m + n \mid X > m) = \mathbb{P}(X > n).$

Moreover, the geometric distribution is the only discrete distribution with this memoryless property.

Given that you have already waited $m$ trials without success, the remaining wait has the same distribution as if you were starting fresh. This is the discrete analogue of the memoryless property of the exponential distribution.

Proof

Compute the tail probability

$\mathbb{P}(X > k) = \sum_{j=k+1}^{\infty} (1-p)^{j-1}p = (1-p)^k$ .

Apply Bayes

$\mathbb{P}(X > m+n \mid X > m) = \frac{\mathbb{P}(X > m+n)}{\mathbb{P}(X > m)} = \frac{(1-p)^{m+n}}{(1-p)^m} = (1-p)^n = \mathbb{P}(X > n). \quad \blacksquare$ $

,

Geometric Memoryless Property

Visualize the memoryless property: the conditional PMF of $X - m$ given $X > m$ is identical to the original PMF, regardless of $m$ .

Parameters

p

0.3

m

(trials already elapsed)5

Definition:
Negative Binomial Distribution

$X \sim \text{NegBin}(r, p)$ : the number of trials until the $r$ -th success.

$P(k) = \binom{k-1}{r-1} p^r (1-p)^{k-r}, \quad k = r, r+1, r+2, \ldots$

Mean: $\mathbb{E}[X] = r/p$ .
Variance: $\text{Var}(X) = r(1-p)/p^2$ .
MGF: $M_X(t) = \left(\frac{pe^t}{1 - (1-p)e^t}\right)^r$ for $t < -\ln(1-p)$ .

The negative binomial is a sum of $r$ independent Geometric( $p$ ) random variables. For $r = 1$ , it reduces to the geometric distribution.

Definition:
Poisson Distribution

$X \sim \text{Poisson}(\lambda)$ with $\lambda > 0$ :

$P(k) = \frac{e^{-\lambda} \lambda^k}{k!}, \quad k = 0, 1, 2, \ldots$

Mean: $\mathbb{E}[X] = \lambda$ .
Variance: $\text{Var}(X) = \lambda$ .
MGF: $M_X(t) = \exp\!\left(\lambda(e^t - 1)\right)$ .

The Poisson distribution is remarkable in that its mean and variance are equal. This "equi-dispersion" property is a quick diagnostic: if the sample mean and variance of a count data set are roughly equal, the Poisson may be a good model.

,

Theorem: Poisson Limit Theorem (Law of Rare Events)

If $X_n \sim \text{Binomial}(n, p_n)$ with $np_n \to \lambda$ as $n \to \infty$ , then for every fixed $k \geq 0$ :

$\lim_{n \to \infty} \mathbb{P}(X_n = k) = \frac{e^{-\lambda} \lambda^k}{k!}.$

When the number of trials is large but each trial has a small success probability, the binomial distribution is well approximated by the Poisson. This is the "law of rare events" — the Poisson distribution naturally governs counts of rare occurrences.

Proof

Write out the binomial PMF

With $p_n = \lambda/n$ :

$\binom{n}{k} p_n^k (1-p_n)^{n-k} = \frac{n!}{k!(n-k)!} \cdot \frac{\lambda^k}{n^k} \cdot \left(1 - \frac{\lambda}{n}\right)^{n-k}.$

Take the limit

As $n \to \infty$ :

$\frac{n!}{(n-k)! \cdot n^k} = \frac{n(n-1)\cdots(n-k+1)}{n^k} \to 1$ .
$(1 - \lambda/n)^n \to e^{-\lambda}$ .
$(1 - \lambda/n)^{-k} \to 1$ .

So the limit is $\frac{\lambda^k}{k!} \cdot e^{-\lambda}$ . $\blacksquare$

,

Poisson PMF vs $\lambda$

Explore how the Poisson PMF changes as $\lambda$ increases. For large $\lambda$ , the distribution becomes approximately Gaussian by the CLT.

Parameters

\lambda

5

Show Normal approximation

Definition:
Discrete Uniform Distribution

$X \sim \text{Uniform}\{a, a+1, \ldots, b\}$ for integers $a \leq b$ :

$P(k) = \frac{1}{b - a + 1}, \quad k = a, a+1, \ldots, b.$

Mean: $\mathbb{E}[X] = (a + b)/2$ .
Variance: $\text{Var}(X) = \frac{(b - a)(b - a + 2)}{12}$ .

The fair die ( $a = 1$ , $b = 6$ ) is the prototypical example. The discrete uniform distribution maximizes entropy over its support — we will formalize this in Section 5.5.

Definition:
Hypergeometric Distribution

Draw $r$ items without replacement from a population of $n$ items, of which $n_1$ are "good." Let $X$ = number of good items drawn.

$P(k) = \frac{\binom{n_1}{k}\binom{n - n_1}{r - k}}{\binom{n}{r}}, \quad k = \max(0, r - n + n_1), \ldots, \min(r, n_1).$

Mean: $\mathbb{E}[X] = r \cdot n_1/n$ .
Variance: $\text{Var}(X) = r \cdot \frac{n_1}{n} \cdot \frac{n - n_1}{n} \cdot \frac{n - r}{n - 1}$ .

When $n$ is much larger than $r$ , sampling without replacement is approximately the same as sampling with replacement, and the hypergeometric approaches the Binomial $(r, n_1/n)$ . The correction factor $(n-r)/(n-1)$ is the finite-population correction.

,

Summary of Common Discrete Distributions

Distribution	PMF $P(k)$	Mean	Variance
Bernoulli $(p)$	$p^k(1-p)^{1-k}$ , $k \in \{0,1\}$	$p$	$p(1-p)$
Binomial $(n,p)$	$\binom{n}{k}p^k(1-p)^{n-k}$	$np$	$np(1-p)$
Geometric $(p)$	$(1-p)^{k-1}p$ , $k \geq 1$	$1/p$	$(1-p)/p^2$
NegBin $(r,p)$	$\binom{k-1}{r-1}p^r(1-p)^{k-r}$	$r/p$	$r(1-p)/p^2$
Poisson $(\lambda)$	$e^{-\lambda}\lambda^k/k!$	$\lambda$	$\lambda$
Uniform $\{a,\ldots,b\}$	$1/(b-a+1)$	$(a+b)/2$	$(b-a)(b-a+2)/12$
Hypergeometric	$\binom{n_1}{k}\binom{n-n_1}{r-k}/\binom{n}{r}$	$rn_1/n$	See definition

Why This Matters: Poisson Models for Network Traffic

The Poisson distribution is the workhorse model for packet arrivals in telecommunication networks. If users initiate sessions independently at a low individual rate, the total number of arrivals in a time interval is approximately Poisson (by the law of rare events). This is the starting point for queueing theory — the Poisson arrival process feeds into the M/M/1 queue, the simplest model for a network switch.

⚠️Engineering Note

When Poisson Fails: Overdispersion in Real Networks

Real network traffic often exhibits overdispersion — the variance exceeds the mean, violating the Poisson assumption. This arises from burstiness and long-range dependence. The negative binomial distribution, which allows $\text{Var}(X) > \mathbb{E}[X]$ , is a common alternative. Always check equi-dispersion before blindly applying Poisson models.

Quick Check

A random variable $X$ has PMF $P(k) = e^{-3} \cdot 3^k / k!$ for $k = 0, 1, 2, \ldots$ What are $\mathbb{E}[X]$ and $\text{Var}(X)$ ?

$\mathbb{E}[X] = 3$ , $\text{Var}(X) = 9$

$\mathbb{E}[X] = 3$ , $\text{Var}(X) = 3$

$\mathbb{E}[X] = 9$ , $\text{Var}(X) = 3$

$\mathbb{E}[X] = 3$ , $\text{Var}(X) = \sqrt{3}$

Correction:

\mathbb{E}[X] = 3

,

\text{Var}(X) = 3

This is Poisson( $\lambda = 3$ ), and for Poisson, mean = variance = $\lambda$ .

Historical Note: Poisson and the Law of Rare Events

1837

Siméon Denis Poisson published his eponymous distribution in 1837 in his work Recherches sur la probabilité des jugements. But the distribution did not become famous until Ladislaus Bortkiewicz's 1898 monograph, which used it to model the number of Prussian cavalry soldiers killed by horse kicks per year — the classic "law of small numbers" example. Bortkiewicz showed that across 14 Prussian army corps over 20 years, the number of deaths per corps per year followed a Poisson distribution with $\lambda \approx 0.7$ remarkably well.

Poisson Distribution

A discrete distribution with parameter $\lambda > 0$ , PMF $e^{-\lambda}\lambda^k/k!$ , and the property that mean equals variance equals $\lambda$ .

Moment Generating Function (MGF)

$M_X(t) = \mathbb{E}[e^{tX}]$ . When it exists in a neighborhood of $t = 0$ , it uniquely determines the distribution and generates moments via $\mathbb{E}[X^k] = M_X^{(k)}(0)$ .

Related: Expectation, Variance

Key Takeaway

The Poisson distribution arises as the limit of the binomial when $n$ is large and $p$ is small with $np \to \lambda$ . Whenever you count rare events among many independent trials, the Poisson is the natural model. Its defining signature is that mean equals variance.

Common Discrete Distributions

A Toolkit of Named Distributions

Definition: Bernoulli Distribution

Definition: Binomial Distribution

Theorem: Mean and Variance of the Binomial via Indicators

Decompose into indicators

Variance by independence

Example: Bit Errors in a Transmitted Block

Model

Markov bound

Poisson approximation

Binomial PMF Explorer

Parameters

Definition: Geometric Distribution

Theorem: Memoryless Property of the Geometric Distribution

Compute the tail probability

Apply Bayes

Geometric Memoryless Property

Parameters

Definition: Negative Binomial Distribution

Definition: Poisson Distribution

Theorem: Poisson Limit Theorem (Law of Rare Events)

Write out the binomial PMF

Take the limit

Poisson PMF vs λ\lambdaλ

Parameters

Definition: Discrete Uniform Distribution

Definition: Hypergeometric Distribution

Summary of Common Discrete Distributions

Why This Matters: Poisson Models for Network Traffic

When Poisson Fails: Overdispersion in Real Networks

Quick Check

Historical Note: Poisson and the Law of Rare Events

Poisson Distribution

Moment Generating Function (MGF)

Key Takeaway

Definition:
Bernoulli Distribution

Definition:
Binomial Distribution

Definition:
Geometric Distribution

Definition:
Negative Binomial Distribution

Definition:
Poisson Distribution

Poisson PMF vs $\lambda$

Definition:
Discrete Uniform Distribution

Definition:
Hypergeometric Distribution