Exercises

(a) CT, discrete-valued (counting process, values in $\{0, 1, 2, \ldots\}$). (b) CT, continuous-valued. (c) DT, continuous-valued (prices are real-valued; the day index is discrete). (d) DT, discrete-valued.

$\mu_X(t) = 3\mathbb{E}[\cos(10t + \Theta)] = 0$.

$r_{XX}(t, t-\tau) = 9\mathbb{E}[\cos(10t + \Theta)\cos(10(t-\tau) + \Theta)]$ $= \frac{9}{2}[\cos(10\tau) + \mathbb{E}[\cos(20t - 10\tau + 2\Theta)]]$ $= \frac{9}{2}\cos(10\tau)$.

$\text{Var}(X(t)) = \mathbb{E}[|X(t)|^2] - |\mu_X(t)|^2 = r_{XX}(t, t) - |\mu|^2 = r_{XX}(0) - |\mu|^2$. Both $r_{XX}(0)$ and $\mu$ are constants (by WSS), so the variance is constant.

For $k \geq 1$: $r_{xx}[k] = \mathbb{E}[X_n X_{n-k}^*] = a\,\mathbb{E}[X_{n-1} X_{n-k}^*] + \mathbb{E}[W_n X_{n-k}^*]$. Since $W_n$ is independent of $X_{n-k}$ for $k \geq 1$: $r_{xx}[k] = a\,r_{xx}[k-1]$.

$r_{xx}[k] = a^k r_{xx}[0]$ for $k \geq 0$. By Hermitian symmetry: $r_{xx}[k] = a^{|k|} r_{xx}[0]$.

For $k = 0$: $r_{xx}[0] = a^2 r_{xx}[0] + \sigma^2$, so $r_{xx}[0] = \sigma^2 / (1 - a^2)$. Therefore $r_{xx}[k] = \frac{\sigma^2}{1 - a^2} a^{|k|}$.

$r_{XX}(-\tau) = \mathbb{E}[X(t)X(t+\tau)]$. Setting $s = t + \tau$: $r_{XX}(-\tau) = \mathbb{E}[X(s-\tau)X(s)] = r_{XX}(\tau)$ (by the WSS definition). For complex: $r_{XX}(\tau) = r_{XX}^*(-\tau)$, which reduces to $r_{XX}(\tau) = r_{XX}(-\tau)$ when real.

$\mu_Z = 0$ (constant). $r_{ZZ}(\tau) = \mathbb{E}[(A + Y(t))(A + Y(t-\tau))] = \sigma^2 + r_{YY}(\tau)$, which depends only on $\tau$. So $Z(t)$ is WSS.

$c_{ZZ}(\tau) = r_{ZZ}(\tau) - 0 = \sigma^2 + c_{YY}(\tau)$. As $|\tau| \to \infty$: $c_{ZZ}(\tau) \to \sigma^2 \neq 0$. The sufficient condition fails, and indeed $\langle Z \rangle_T \to A + 0 = A \neq 0 = \mu_Z$. Not mean-ergodic.

$r_{XY}(\tau) = \mathbb{E}[X(t)Y^*(t-\tau)]$. $r_{YX}(-\tau) = \mathbb{E}[Y(t)X^*(t+\tau)] = \mathbb{E}[Y(s-\tau)X^*(s)]$ (set $s = t + \tau$). Taking conjugate: $r_{YX}^*(-\tau) = \mathbb{E}[X(s)Y^*(s-\tau)] = r_{XY}(\tau)$.

$r_{xx}[0] = 5(0.8)^0 = 5$.

$\text{Var}(X_n) = c_{xx}[0] = r_{xx}[0] - |\mu|^2 = 5 - 0 = 5$.

$c_{xx}[k] = r_{xx}[k] = 5(0.8)^{|k|} \to 0$ as $|k| \to \infty$. The sufficient condition holds, so the process is mean-ergodic.

$\{X_n\}$ is Gaussian and WSS. By Lemma 43, a WSS Gaussian process is SSS. Therefore $\{X_n\}$ is strict-sense stationary.

$\mu_Y = \sum_k h_k \mathbb{E}[X_{n-k}] = 0$ (constant).

$r_{yy}[m] = \mathbb{E}[Y_n Y_{n-m}^*] = \sum_{k=0}^{L-1}\sum_{\ell=0}^{L-1} h_k h_\ell^* \mathbb{E}[X_{n-k}X_{n-m-\ell}^*]$ $= \sum_{k=0}^{L-1}\sum_{\ell=0}^{L-1} h_k h_\ell^* \delta[m + \ell - k]$ $= \sum_{k} h_k h_{k-m}^*$ where indices are restricted to $[0, L-1]$.

$r_{yy}[m] = \sum_{k=\max(0,m)}^{\min(L-1,L-1+m)} h_k h_{k-m}^*$ for $|m| < L$, and $r_{yy}[m] = 0$ for $|m| \geq L$. This depends only on $m$, confirming WSS. This is the deterministic autocorrelation of the filter coefficients.

Let $W = \sum_{i=1}^{N} a_i X(t_i)$. Then: $$0 \leq \mathbb{E}[|W|^2] = \mathbb{E}\left[\left(\sum_i a_i X(t_i)\right)\left(\sum_j a_j X(t_j)\right)^*\right]$$ $$= \sum_{i=1}^{N}\sum_{j=1}^{N} a_i a_j^* \mathbb{E}[X(t_i)X^*(t_j)] = \sum_{i=1}^{N}\sum_{j=1}^{N} a_i a_j^* r_{XX}(t_i - t_j).$$

$\text{Var}(\langle X \rangle_N) = \frac{1}{(2N+1)^2}\sum_{n=-N}^{N}\sum_{m=-N}^{N} c_{xx}[n-m].KATEXPLACEHOLDER0END\text{Var}(\langle X \rangle_N) = \frac{1}{2N+1}\sum_{k=-2N}^{2N}\left(1 - \frac{|k|}{2N+1}\right)c_{xx}[k].$$

If $c_{xx}[k] \to 0$ as $|k| \to \infty$, then for large $N$, the terms with large $|k|$ are small, and the $1/(2N+1)$ factor drives the entire sum to zero. Hence $\text{Var}(\langle X \rangle_N) \to 0$.

$r_{XY}(\tau) = 3e^{-2|\tau|} \neq 0$ for all finite $\tau$. Not orthogonal.

For real processes: $r_{YX}(\tau) = r_{XY}(-\tau) = 3e^{-2|-\tau|} = 3e^{-2|\tau|}$. So $r_{YX}(\tau) = r_{XY}(\tau)$ in this case (because $r_{XY}$ is even).

$|r_{XY}(5)| = 3e^{-10} \approx 1.36 \times 10^{-4}$. $\sqrt{r_{XX}(0)r_{YY}(0)} = \sqrt{40} \approx 6.32$. Yes, $1.36 \times 10^{-4} \leq 6.32$. The bound is easily satisfied.

$\mu_X(t) = \mathbb{E}[A]\cos(2\pi f_0 t) + \mathbb{E}[B]\sin(2\pi f_0 t) = 0$.

$r_{XX}(t_1, t_2) = \mathbb{E}[A^2]\cos(2\pi f_0 t_1)\cos(2\pi f_0 t_2) + \mathbb{E}[B^2]\sin(2\pi f_0 t_1)\sin(2\pi f_0 t_2)$ $+ \mathbb{E}[AB][\cos(2\pi f_0 t_1)\sin(2\pi f_0 t_2) + \sin(2\pi f_0 t_1)\cos(2\pi f_0 t_2)]$.

Since $A, B$ uncorrelated with zero mean: $\mathbb{E}[AB] = 0$. Using $\cos\alpha\cos\beta + \sin\alpha\sin\beta = \cos(\alpha - \beta)$: $$r_{XX}(t_1, t_2) = \sigma^2 \cos(2\pi f_0(t_1 - t_2)) = \sigma^2 \cos(2\pi f_0 \tau).$$

$r_{XX}(\tau) = \sigma^2 \cos(2\pi f_0 \tau)$ depends only on $\tau$, and $\mu = 0$ is constant. Hence $X(t)$ is WSS.

By induction: $X_0 \sim \mathcal{N}(0,1)$. If $X_{n-1} \sim \mathcal{N}(0,1)$, then $X_n = \rho X_{n-1} + W_n$ is Gaussian with mean 0 and variance $\rho^2 \cdot 1 + (1 - \rho^2) = 1$. So $X_n \sim \mathcal{N}(0,1)$ for all $n$.

Each $X_n$ is a linear function of $(X_0, W_1, \ldots, W_n)$, which are jointly Gaussian. Therefore any finite collection $(X_{n_1}, \ldots, X_{n_k})$ is jointly Gaussian.

The process is Gaussian (part b). $\mu[n] = 0$ (constant). The autocorrelation: $r_{xx}[n,m] = \mathbb{E}[X_n X_m] = \rho^{|n-m|}$ (can be shown by iterating the recursion), which depends only on $|n - m|$. So the process is WSS. By Lemma 43, WSS + Gaussian $\Rightarrow$ SSS.

True, provided the second moment exists. For i.i.d. $\{X_n\}$: $\mu[n] = \mu$ (constant) and $r_{xx}[n,m] = \sigma^2\delta[n-m] + \mu^2$ depends only on $n - m$. In fact, every i.i.d. sequence with finite second moment is strict-sense stationary.

$r_{XX}(0) = 4\,\text{sinc}(0) = 4$.

$r_{XX}(0.5) = 4\,\text{sinc}(1) = 4 \cdot \frac{\sin(\pi)}{\pi} = 0$.

$c_{XX}(\tau) = r_{XX}(\tau) = 4\,\text{sinc}(2\tau) \to 0$ as $|\tau| \to \infty$. The process is mean-ergodic.

$\mathbb{E}[\hat{r}(\tau)] = \frac{1}{2T}\int_{-T}^{T}\mathbb{E}[X(t+\tau)X(t)]\,dt = \frac{1}{2T}\int_{-T}^{T} r_{XX}(\tau)\,dt = r_{XX}(\tau).$$

The process $Z(t) = X(t+\tau)X(t)$ is WSS (can be verified). The estimator $\hat{r}(\tau) = \langle Z \rangle_T$ is a time average. By the mean-ergodic theorem, $\hat{r}(\tau) \to r_{XX}(\tau)$ in m.s. if $c_{ZZ}(\alpha) \to 0$ as $|\alpha| \to \infty$.

For a Gaussian process, $c_{ZZ}$ can be computed explicitly using the fourth-moment theorem, and the condition reduces to $r_{XX}(\alpha) \to |\mu|^2$ as $|\alpha| \to \infty$, i.e., $c_{XX}(\alpha) \to 0$. This is the standard ergodicity condition.