Exercises

Let $\epsilon > 0$. Enumerate $\mathbb{Q} = \{q_1, q_2, \ldots\}$ and cover $q_n$ by the interval $(q_n - \epsilon/2^{n+1}, q_n + \epsilon/2^{n+1})$. The total length is $\sum_{n=1}^{\infty} \epsilon/2^n = \epsilon$. Since $\lambda(\mathbb{Q}) \leq \epsilon$ for every $\epsilon > 0$, $\lambda(\mathbb{Q}) = 0$. $\blacksquare$

1. $\Omega \in \mathcal{F}$ by construction.
2. $\emptyset^c = \Omega \in \mathcal{F}$, $A^c \in \mathcal{F}$, $(A^c)^c = A \in \mathcal{F}$, $\Omega^c = \emptyset \in \mathcal{F}$.
3. Any countable union of elements from $\{\emptyset, A, A^c, \Omega\}$ yields one of $\emptyset, A, A^c, A \cup A^c = \Omega$, all of which are in $\mathcal{F}$. $\blacksquare$

$\int_0^1 n x^n\, dx = n \cdot \frac{x^{n+1}}{n+1}\Big|_0^1 = \frac{n}{n+1} \to 1$.

For $x \in [0, 1)$: $f_n(x) = n x^n \to 0$ (exponential decay beats linear growth). At $x = 1$: $f_n(1) = n \to \infty$. So $f_n \to 0$ a.e. on $[0,1]$ (the point $x=1$ has measure zero), but $\int f_n \to 1 \neq 0 = \int \lim f_n$.

DCT: For $x$ near 1, $\sup_n f_n(x)$ is unbounded, and there is no integrable dominating function $g$ with $f_n \leq g$ for all $n$. MCT: $f_n$ is not monotone (for fixed $x \in (0,1)$, $f_n(x) = n x^n$ first increases then decreases as $n$ grows). This example shows that convergence theorems have genuine hypotheses.

At step $k$, we have $2^k$ intervals each of length $3^{-k}$, so the total remaining length is $(2/3)^k \to 0$. The Cantor set is the intersection, so $\lambda(C) \leq (2/3)^k$ for all $k$, giving $\lambda(C) = 0$.

Each $x \in C$ has a ternary expansion using only digits 0 and 2: $x = \sum_{k=1}^{\infty} a_k 3^{-k}$ with $a_k \in \{0, 2\}$. The map $x \mapsto \sum_{k} (a_k/2) 2^{-k}$ is a bijection from $C$ onto $[0,1]$ (via binary expansions). Since $[0,1]$ is uncountable, so is $C$.

$X^{-1}(\mathbb{R}) = \Omega \in \sigma(X)$.

If $X^{-1}(B) \in \sigma(X)$, then $(X^{-1}(B))^c = X^{-1}(B^c) \in \sigma(X)$ since $B^c \in \mathcal{B}(\mathbb{R})$.

If $X^{-1}(B_n) \in \sigma(X)$ for $n = 1, 2, \ldots$, then $\bigcup_n X^{-1}(B_n) = X^{-1}(\bigcup_n B_n) \in \sigma(X)$ since $\bigcup_n B_n \in \mathcal{B}(\mathbb{R})$. $\blacksquare$

$(a, b) = \bigcup_{n=1}^{\infty} (a, b - 1/n]$ (for large enough $n$). And $(a, x] = (-\infty, x] \setminus (-\infty, a]$, which is in $\sigma(\{(-\infty, x] : x \in \mathbb{R}\})$.

Every open set in $\mathbb{R}$ is a countable union of open intervals. Since $\mathcal{B}(\mathbb{R})$ is generated by open sets, and we can generate all open sets from half-open intervals, the two $\sigma$-algebras are equal. $\blacksquare$

$B = A \cup (B \setminus A)$ is a disjoint union, so $\mu(B) = \mu(A) + \mu(B \setminus A) \geq \mu(A)$ since $\mu(B \setminus A) \geq 0$. $\blacksquare$

Let $Z = \mathbb{E}[X \mid \mathcal{G}]$. Then: $$\mathbb{E}[(X - Y)^2] = \mathbb{E}[(X - Z)^2] + 2\mathbb{E}[(X - Z)(Z - Y)] + \mathbb{E}[(Z - Y)^2].$$

$(Z - Y)$ is $\mathcal{G}$-measurable, so by "taking out what is known": $$\mathbb{E}[(X - Z)(Z - Y)] = \mathbb{E}[\mathbb{E}[(X - Z)(Z - Y) \mid \mathcal{G}]] = \mathbb{E}[(Z - Y)\mathbb{E}[X - Z \mid \mathcal{G}]] = 0,$$ since $\mathbb{E}[X - Z \mid \mathcal{G}] = Z - Z = 0$.

$\mathbb{E}[(X-Y)^2] = \mathbb{E}[(X-Z)^2] + \mathbb{E}[(Z-Y)^2] \geq \mathbb{E}[(X-Z)^2]$, with equality iff $Y = Z$ a.s. $\blacksquare$

$\mathbb{E}[X \mid \mathcal{H}]$ is $\mathcal{H}$-measurable by definition.

For any $H \in \mathcal{H} \subseteq \mathcal{G}$: $$\int_H \mathbb{E}[X \mid \mathcal{G}]\, dP = \int_H X\, dP$$ (by the defining property of $\mathbb{E}[X \mid \mathcal{G}]$, since $H \in \mathcal{G}$). Also $\int_H \mathbb{E}[X \mid \mathcal{H}]\, dP = \int_H X\, dP$. So both sides agree on all $H \in \mathcal{H}$, proving by uniqueness that they are equal a.s. $\blacksquare$

$S_{n+1}^2 = (S_n + X_{n+1})^2 = S_n^2 + 2S_n X_{n+1} + X_{n+1}^2$.

$\mathbb{E}[S_{n+1}^2 \mid \mathcal{F}_n] = S_n^2 + 2S_n \cdot 0 + \sigma^2 = S_n^2 + \sigma^2$. Hence $\mathbb{E}[M_{n+1} \mid \mathcal{F}_n] = S_n^2 + \sigma^2 - (n+1)\sigma^2 = S_n^2 - n\sigma^2 = M_n$. $\blacksquare$

Let $k$ be the number of red balls after $n$ draws ($n+2$ total). Then:

• With probability $k/(n+2)$, we draw red: $R_{n+1} = (k+1)/(n+3)$.
• With probability $(n+2-k)/(n+2)$, we draw blue: $R_{n+1} = k/(n+3)$.

$\mathbb{E}[R_{n+1} \mid \mathcal{F}_n] = \frac{k}{n+2} \cdot \frac{k+1}{n+3} + \frac{n+2-k}{n+2} \cdot \frac{k}{n+3} = \frac{k(n+3)}{(n+2)(n+3)} = \frac{k}{n+2} = R_n$.

$R_n$ is a martingale bounded in $[0,1]$, so by the martingale convergence theorem, $R_n \to R_{\infty}$ a.s. One can show $R_{\infty} \sim \text{Beta}(1,1) = \text{Uniform}[0,1]$ using exchangeability (de Finetti's theorem) or by computing moments.

$\frac{dP}{dQ}(x) = \frac{f_P(x)}{f_Q(x)} = \frac{\exp(-(x-\mu)^2/(2\sigma^2))}{\exp(-x^2/(2\sigma^2))} = \exp\left(\frac{\mu x}{\sigma^2} - \frac{\mu^2}{2\sigma^2}\right).$$

$\mathbb{E}_{P_0}[dP_1/dP_0] = \int \frac{dP_1}{dP_0}\, dP_0 = \int dP_1 = P_1(\Omega) = 1$. $\blacksquare$

$\frac{dP_1^{(n)}}{dP_0^{(n)}}(\mathbf{x}) = \prod_{i=1}^n \exp(\mu x_i - \mu^2/2) = \exp\left(\mu \sum_{i=1}^n x_i - n\mu^2/2\right).$$

$\log \frac{dP_1^{(n)}}{dP_0^{(n)}} = \mu \sum_{i=1}^n x_i - \frac{n\mu^2}{2}.$$The sufficient statistic is$T = \sum_{i=1}^n x_i = n\bar{x}$. The Neyman-Pearson test rejects$H_0$when$T > \eta$.

$\mathbb{E}_0[L_{n+1} \mid \mathcal{F}_n] = L_n \cdot \mathbb{E}_0\left[\frac{f_1(X_{n+1})}{f_0(X_{n+1})}\right] = L_n \cdot \int \frac{f_1(x)}{f_0(x)} f_0(x)\, dx = L_n \cdot 1 = L_n.$$

The SPRT stops at $T = \min\{n : L_n \leq B \text{ or } L_n \geq A\}$. By optional stopping (with appropriate regularity conditions): $\mathbb{E}_0[L_T] = L_0 = 1$. Since $L_T \approx A$ with probability $\alpha$ (false alarm) and $L_T \approx B$ with probability $1 - \alpha$: $A\alpha + B(1-\alpha) \approx 1$. Combined with a similar equation under $P_1$, one gets $\alpha \approx (1-B)/(A-B)$ and the miss probability $\beta \approx (A - 1)/(A - B)$ (Wald's approximation).

Define $g_n = \inf_{k \geq n} f_k$. Then $g_n \leq f_n$ (so $\int g_n \leq \int f_n$) and $g_1 \leq g_2 \leq \cdots$ (the infimum over a shrinking set is non-decreasing). Also $\lim_n g_n = \sup_n \inf_{k \geq n} f_k = \liminf_n f_n$.

By the monotone convergence theorem: $\int \liminf f_n\, d\mu = \int \lim_n g_n\, d\mu = \lim_n \int g_n\, d\mu$. Since $\int g_n \leq \int f_n$ for all $n$: $\lim_n \int g_n = \liminf_n \int g_n \leq \liminf_n \int f_n$. $\blacksquare$

Since $P \perp Q$, there exists $A$ with $P(A) = 1$ and $Q(A) = 0$. If $dP/dQ = f$ existed, then $P(A) = \int_A f\, dQ = 0$ (since $Q(A) = 0$). But $P(A) = 1 \neq 0$, contradiction. $\blacksquare$

Since $\varphi$ is convex, for every $y \in \mathbb{R}$ there exists $\lambda(y) \in \partial\varphi(y)$ (subgradient) with $\varphi(x) \geq \varphi(y) + \lambda(y)(x - y)$ for all $x$.

Set $y = \mathbb{E}[X \mid \mathcal{G}]$ (which is $\mathcal{G}$-measurable) and $x = X$: $\varphi(X) \geq \varphi(\mathbb{E}[X \mid \mathcal{G}]) + \lambda(\mathbb{E}[X \mid \mathcal{G}])(X - \mathbb{E}[X \mid \mathcal{G}])$. Take $\mathbb{E}[\cdot \mid \mathcal{G}]$ of both sides. The left side gives $\mathbb{E}[\varphi(X) \mid \mathcal{G}]$. On the right, use "taking out what is known" and $\mathbb{E}[X - \mathbb{E}[X \mid \mathcal{G}] \mid \mathcal{G}] = 0$. Result: $\mathbb{E}[\varphi(X) \mid \mathcal{G}] \geq \varphi(\mathbb{E}[X \mid \mathcal{G}])$. $\blacksquare$

$M_n = \mathbb{E}[X \mid \mathcal{F}_n]$ is $\mathcal{F}_n$-measurable by definition. $\mathbb{E}[|M_n|] \leq \mathbb{E}[\mathbb{E}[|X| \mid \mathcal{F}_n]] = \mathbb{E}[|X|] < \infty$.

$\mathbb{E}[M_{n+1} \mid \mathcal{F}_n] = \mathbb{E}[\mathbb{E}[X \mid \mathcal{F}_{n+1}] \mid \mathcal{F}_n] = \mathbb{E}[X \mid \mathcal{F}_n] = M_n$ by the tower property (since $\mathcal{F}_n \subseteq \mathcal{F}_{n+1}$). $\blacksquare$

For $g = \mathbf{1}_A$: $\mathbb{E}_1[\mathbf{1}_A] = P_1(A) = \int_A L\, dP_0 = \mathbb{E}_0[\mathbf{1}_A \cdot L]$.

By linearity, the formula holds for simple functions. For general $g \geq 0$, approximate by simple $g_n \uparrow g$ and apply MCT on both sides. For general bounded $g$, use $g = g^+ - g^-$. $\blacksquare$