Exercises

For every $\omega \in \Omega$, $X_n(\omega) = Y(\omega)/n \to 0$ since $Y(\omega)$ is a fixed finite number. This is actually sure convergence (even stronger than a.s.).

$\mathbb{E}[|X_n|^r] = \mathbb{E}[Y^r]/n^r = \Gamma(r+1)/n^r \to 0$ for all $r \geq 1$.

Since a.s. convergence implies convergence in probability, the result follows. Alternatively, $L^r$ convergence implies convergence in probability. $\blacksquare$

$\mu = 1/2$, $\sigma^2 = 1/12$.

The $X_i$ are i.i.d. with finite variance, so by the WLLN: $\bar{X}_n \xrightarrow{P} \mu = 1/2$. Explicitly: $\mathbb{P}(|\bar{X}_n - 1/2| \geq \epsilon) \leq \frac{1}{12 n \epsilon^2} \to 0$. $\blacksquare$

$|\mathbb{E}[X_n] - \mathbb{E}[X]| \leq \mathbb{E}[|X_n - X|] \leq \sqrt{\mathbb{E}[(X_n - X)^2]} \to 0$ by Jensen (or Cauchy-Schwarz with the constant 1).

$|\mathbb{E}[X_n^2] - \mathbb{E}[X^2]| = |\mathbb{E}[(X_n-X)(X_n+X)]| \leq \sqrt{\mathbb{E}[(X_n-X)^2]} \sqrt{\mathbb{E}[(X_n+X)^2]}$. The first factor $\to 0$. The second factor is bounded (since $L^2$ convergence implies $\mathbb{E}[X_n^2]$ is bounded). Hence the product $\to 0$. $\blacksquare$

$\mathbb{P}(S_n \leq k) = \mathbb{P}\!\left(Z_n \leq \frac{k - np}{\sqrt{np(1-p)}}\right) \approx \Phi\!\left(\frac{k - np}{\sqrt{np(1-p)}}\right).$$

We need $\mathbb{P}(|\bar{X}_n - 0.5| \leq 0.01) \geq 0.99$. By CLT: $\mathbb{P}(|Z_n| \leq 0.01\sqrt{n}/0.5) \approx 2\Phi(0.02\sqrt{n}) - 1 \geq 0.99$. So $\Phi(0.02\sqrt{n}) \geq 0.995$, giving $0.02\sqrt{n} \geq 2.576$, hence $\sqrt{n} \geq 128.8$ and $n \geq 16{,}590$.

Compare with the Chebyshev bound: $n \geq 0.25/(0.01^2 \times 0.01) = 250{,}000$. The CLT gives a factor-15 improvement.

Since $\mathbb{P}(|X_n - X| \geq 1/k) \to 0$ for each $k$, we can find $n_k$ (with $n_1 < n_2 < \cdots$) such that $\mathbb{P}(|X_{n_k} - X| \geq 1/k) \leq 2^{-k}$.

Let $A_k = \{|X_{n_k} - X| \geq 1/k\}$. Then $\sum_k \mathbb{P}(A_k) \leq \sum_k 2^{-k} = 1 < \infty$. By Borel-Cantelli: $\mathbb{P}(A_k \text{ i.o.}) = 0$. This means a.s., for all sufficiently large $k$, $|X_{n_k} - X| < 1/k \to 0$. Hence $X_{n_k} \xrightarrow{\text{a.s.}} X$. $\blacksquare$

$S_{100}$ has mean $100$ and standard deviation $10$. So: $$\mathbb{P}(S_{100} > 110) = \mathbb{P}(Z > (110-100)/10) = \mathbb{P}(Z > 1) \approx Q(1) \approx 0.1587.$$

$S_{100} \sim \text{Gamma}(100, 1)$. Using numerical evaluation: $\mathbb{P}(S_{100} > 110) = 1 - F_{\text{Gamma}}(110; 100, 1) \approx 0.1517$. The CLT approximation error is about $0.007$ — quite good for $n = 100$. $\blacksquare$

$\mathbb{P}(\hat{\theta}_n \leq x) = \mathbb{P}(\text{all } X_i \leq x) = (x/\theta)^n$ for $x \in [0, \theta]$.

$\mathbb{P}(n(\theta - \hat{\theta}_n) > t) = \mathbb{P}(\hat{\theta}_n < \theta - t/n) = (1 - t/(n\theta))^n \to e^{-t/\theta}.$

This is the survival function of $\text{Exp}(1/\theta)$.

The MLE $\hat{\theta}_n$ converges at rate $1/n$ (not $1/\sqrt{n}$), and its limiting distribution is exponential, not Gaussian. The CLT does not apply because $\hat{\theta}_n$ is not a smooth function of the sample mean — it is the maximum, which depends on the extreme order statistic. $\blacksquare$

$\sqrt{n}(\bar{X}_n - \mu) \xrightarrow{d} \mathcal{N}(0, \sigma^2)$.

$g(x) = \log(x)$, $g'(\mu) = 1/\mu$. By the delta method: $$\sqrt{n}(\log(\bar{X}_n) - \log(\mu)) \xrightarrow{d} \mathcal{N}\!\left(0, \frac{\sigma^2}{\mu^2}\right).$$ Equivalently, $\log(\bar{X}_n) \approx \mathcal{N}(\log\mu, \sigma^2/(n\mu^2))$. $\blacksquare$

For every $\epsilon > 0$: $$L_n(\epsilon) = \frac{1}{s_n^2}\sum_{i=1}^n \mathbb{E}\!\left[X_i^2 \mathbf{1}_{\{|X_i| > \epsilon s_n\}}\right] \to 0.$$ This says no single summand dominates the sum.

The Ch.F of $S_n/s_n$ is $\prod_{i=1}^n \phi_{X_i}(u/s_n)$. Taylor-expand each factor: $\phi_{X_i}(u/s_n) = 1 - \sigma_i^2 u^2/(2s_n^2) + r_i$ where the remainder $r_i$ is controlled by the Lindeberg condition. The product converges to $\exp(-u^2/2)$ by showing $\sum_i \log(1 - \sigma_i^2 u^2/(2s_n^2) + r_i) \to -u^2/2$. The full argument is in Billingsley (1995), §27. $\blacksquare$

$g(x) = x^2$, $g'(\mu) = 2\mu$. By CLT + delta method: $$\sqrt{n}((\bar{X}_n)^2 - \mu^2) \xrightarrow{d} \mathcal{N}(0, 4\mu^2\sigma^2).$$

The asymptotic variance of $\bar{X}_n^2$ as an estimator of $\mu^2$ is $4\mu^2\sigma^2/n$. Note that $\bar{X}_n^2$ is biased: $\mathbb{E}[\bar{X}_n^2] = \mu^2 + \sigma^2/n$. The bias vanishes at rate $1/n$, which is faster than the standard deviation ($1/\sqrt{n}$), so the bias is asymptotically negligible. $\blacksquare$

$\hat{\sigma}_n^2 = \frac{1}{n}\sum X_i^2 - \bar{X}_n^2$. By WLLN: $\frac{1}{n}\sum X_i^2 \xrightarrow{P} \mathbb{E}[X^2]$ and $\bar{X}_n^2 \xrightarrow{P} \mu^2$. So $\hat{\sigma}_n^2 \xrightarrow{P} \mathbb{E}[X^2] - \mu^2 = \sigma^2$.

Since $\hat{\sigma}_n^2 \xrightarrow{P} \sigma^2$ and $\sigma^2 > 0$, by the continuous mapping theorem $\hat{\sigma}_n \xrightarrow{P} \sigma$. Write $T_n = \frac{\sqrt{n}(\bar{X}_n - \mu)/\sigma}{\hat{\sigma}_n/\sigma}$. By CLT, the numerator $\xrightarrow{d} \mathcal{N}(0,1)$. By above, the denominator $\xrightarrow{P} 1$. By Slutsky: $T_n \xrightarrow{d} \mathcal{N}(0,1)$. $\blacksquare$

$\rho = \mathbb{E}[|X - p|^3] = p(1-p)^3 + (1-p)p^3 = p(1-p)(p^2 + (1-p)^2)$. For $p = 0.5$: $\rho = 0.25 \times 0.5 = 0.125$, $\sigma^3 = (0.5)^3 = 0.125$.

Error $\leq C\rho/(\sigma^3\sqrt{n}) = 0.4748 \times 0.125/(0.125 \times 10) = 0.04748$.

So the CDF error is at most about $4.7\%$ for $n = 100$ with $p = 0.5$, uniformly in $z$. This confirms that the CLT is a reasonable approximation for this case. $\blacksquare$

$\sqrt{n}(\bar{\mathbf{X}}_n - \boldsymbol{\mu}) \xrightarrow{d} \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$.

$g(x_1, x_2) = x_1/x_2$. The gradient at $\boldsymbol{\mu}$ is $\nabla g = (1/\mu_2, -\mu_1/\mu_2^2)^\mathsf{T}$.

$$\sqrt{n}(R_n - \mu_1/\mu_2) \xrightarrow{d} \mathcal{N}(0, \nabla g^\mathsf{T} \boldsymbol{\Sigma} \nabla g).$$

The asymptotic variance is: $$\frac{\Sigma_{11}}{\mu_2^2} - \frac{2\mu_1 \Sigma_{12}}{\mu_2^3} + \frac{\mu_1^2 \Sigma_{22}}{\mu_2^4}. \quad \blacksquare$$

For $\epsilon > 0$: $\mathbb{P}(|X_n - c| \geq \epsilon) = \mathbb{P}(X_n \leq c - \epsilon) + \mathbb{P}(X_n > c + \epsilon)$. Since $c - \epsilon$ and $c + \epsilon$ are continuity points of $F_c$: $\mathbb{P}(X_n \leq c - \epsilon) \to F_c(c - \epsilon) = 0$ and $\mathbb{P}(X_n > c + \epsilon) \to 1 - F_c(c + \epsilon) = 0$. $\blacksquare$

$X_i = e^{U_i^2}$ are i.i.d. with $\mathbb{E}[X_i] = I$ and finite variance (since $e^{x^2} \leq e$ on $[0,1]$). By the SLLN: $\hat{I}_n \xrightarrow{\text{a.s.}} I$.

$\sigma^2 = \mathbb{E}[e^{2U^2}] - I^2 = \int_0^1 e^{2x^2} dx - I^2$. Numerically: $I \approx 1.4627$, $\int_0^1 e^{2x^2} dx \approx 2.3613$, so $\sigma^2 \approx 2.3613 - 1.4627^2 \approx 0.2214$, $\sigma \approx 0.4705$.

The 95% CI is $\hat{I}_n \pm 1.96 \times 0.4705/\sqrt{10000} = \hat{I}_n \pm 0.0092$. $\blacksquare$

$\mathbb{E}[Z_n^2] = n \cdot \text{Var}(\bar{X}_n)/\sigma^2 = n \cdot (\sigma^2/n)/\sigma^2 = 1$. This is exact for all $n$, not just asymptotic.

$L^2$ convergence to $Z \sim \mathcal{N}(0,1)$ requires the variables to be defined on the same probability space. Even then, convergence in distribution plus $\mathbb{E}[Z_n^2] \to \mathbb{E}[Z^2]$ is not sufficient for $L^2$ convergence in general (it suffices if uniform integrability holds, which the finite $(2+\delta)$ moment does provide). So yes, under the stated conditions, $Z_n \xrightarrow{L^2} Z$ if they share a probability space (which can always be arranged by Skorokhod's theorem). $\blacksquare$