Ferkans — Interactive Telecom Tutor

Relating Two Processes

In many engineering systems, we observe the output of a system and want to relate it to the input. The input signal $X(t)$ and the output signal $Y(t)$ are two different processes with a known relationship (e.g., $Y(t) = h(t) * X(t) + N(t)$ ). The cross-correlation function captures the statistical dependence between $X$ and $Y$ at different time instants. It is the key to deriving the input-output relations for LTI systems (Chapter 15) and to designing optimal estimators like the Wiener filter.

Definition:
Cross-Correlation Function

For two second-order processes $\{X(t)\}$ and $\{Y(t)\}$ , the cross-correlation is $r_{XY}(t_1, t_2) = \mathbb{E}[X(t_1)Y^*(t_2)].$

For jointly WSS processes, this depends only on $\tau = t_1 - t_2$ : $r_{XY}(\tau) = \mathbb{E}[X(t)Y^*(t - \tau)].$

For discrete-time jointly WSS processes: $r_{xy}[k] = \mathbb{E}[X_n Y_{n-k}^*]$ .

Definition:
Cross-Covariance Function

The cross-covariance of two processes is $c_{XY}(t_1, t_2) = \text{Cov}(X(t_1), Y(t_2)) = r_{XY}(t_1, t_2) - \mu_X(t_1)\mu_Y^*(t_2).$

For jointly WSS processes with means $\mu_X$ and $\mu_Y$ : $c_{XY}(\tau) = r_{XY}(\tau) - \mu_X \mu_Y^*.$

Definition:
Jointly Wide-Sense Stationary

Two processes $\{X(t)\}$ and $\{Y(t)\}$ are jointly WSS if:

Each is individually WSS, and
The cross-correlation $r_{XY}(t_1, t_2) = r_{XY}(t_1 - t_2)$ depends only on the time difference.

Being individually WSS is necessary but not sufficient for jointly WSS. The cross-correlation must also be shift-invariant.

Theorem: Properties of Cross-Correlation (Jointly WSS)

Let $\{X(t)\}$ and $\{Y(t)\}$ be jointly WSS. Then:

Conjugate symmetry: $r_{XY}(\tau) = r_{YX}^*(-\tau)$ . (For real processes: $r_{XY}(\tau) = r_{YX}(-\tau)$ .)
Cauchy-Schwarz bound: $|r_{XY}(\tau)| \leq \sqrt{r_{XX}(0) \cdot r_{YY}(0)}$ .
Generalized bound: $|r_{XY}(\tau)|^2 \leq r_{XX}(0) \cdot r_{YY}(0)$ for all $\tau$ .

Note: $r_{XY}(\tau)$ is not necessarily even, and $|r_{XY}(\tau)|$ need not be maximized at $\tau = 0$ .

Proof

Conjugate symmetry

$r_{XY}(\tau) = \mathbb{E}[X(t)Y^*(t-\tau)]$ . Set $s = t - \tau$ : $r_{YX}(-\tau) = \mathbb{E}[Y(s)X^*(s+\tau)] = \mathbb{E}[Y(s)X^*(t)]$ . Taking the conjugate: $r_{YX}^*(-\tau) = \mathbb{E}[X(t)Y^*(s)] = \mathbb{E}[X(t)Y^*(t-\tau)] = r_{XY}(\tau)$ .

Cauchy-Schwarz bound

By the Cauchy-Schwarz inequality for random variables: $|r_{XY}(\tau)| = |\mathbb{E}[X(t)Y^*(t-\tau)]| \leq \sqrt{\mathbb{E}[|X(t)|^2]\,\mathbb{E}[|Y(t-\tau)|^2]} = \sqrt{r_{XX}(0)\,r_{YY}(0)}.$

Definition:
Orthogonal and Uncorrelated Processes

Two WSS processes $\{X(t)\}$ and $\{Y(t)\}$ are:

Orthogonal if $r_{XY}(\tau) = 0$ for all $\tau$ .
Uncorrelated if $c_{XY}(\tau) = 0$ for all $\tau$ , i.e., $r_{XY}(\tau) = \mu_X \mu_Y^*$ for all $\tau$ .

If either process has zero mean, orthogonality and uncorrelatedness are equivalent.

Example: Cross-Correlation of LTI Input and Output

Let $Y_n = \sum_{k=0}^{L} h[k] X_{n-k}$ be the output of a causal FIR filter with impulse response $h[0], h[1], \ldots, h[L]$ driven by a zero-mean WSS input $\{X_n\}$ . Find $r_{YX}[m] = \mathbb{E}[Y_n X_{n-m}^*]$ .

Solution

Substitute the filter expression

$r_{YX}[m] = \mathbb{E}\left[\sum_{k=0}^{L} h[k] X_{n-k} \cdot X_{n-m}^*\right] = \sum_{k=0}^{L} h[k]\,\mathbb{E}[X_{n-k} X_{n-m}^*].$ $

Use WSS

$\mathbb{E}[X_{n-k} X_{n-m}^*] = r_{xx}[m - k]$ (depends only on the lag $(n-k) - (n-m) = m - k$ ). Therefore: $r_{YX}[m] = \sum_{k=0}^{L} h[k]\,r_{xx}[m - k] = (h * r_{xx})[m],$ the convolution of the impulse response with the input autocorrelation.

Interpretation

The input-output cross-correlation is the convolution of $h$ with $r_{xx}$ . This relationship is central to system identification: if the input is white noise ( $r_{xx}[m] = \sigma^2 \delta[m]$ ), then $r_{YX}[m] = \sigma^2 h[m]$ — the cross-correlation directly reveals the impulse response.

Example: Proper (Circularly Symmetric) Complex Processes

Let $Z_n = X_n + jY_n$ be a complex-valued WSS process with $\mathbb{E}[Z_n] = 0$ . Define the pseudo-covariance $\tilde{c}_{ZZ}[n,m] = \mathbb{E}[Z_n Z_m]$ (without conjugation). Show that if $Z_n$ is proper ( $\tilde{c}_{ZZ}[n,m] = 0$ for all $n,m$ ), then $c_{XX}[k] = c_{YY}[k]$ and $c_{XY}[k] = -c_{YX}[k]$ .

Solution

Expand the pseudo-covariance

$\tilde{c}_{ZZ}[k] = \mathbb{E}[Z_n Z_{n-k}] = \mathbb{E}[(X_n + jY_n)(X_{n-k} + jY_{n-k})]KATEXPLACEHOLDER0END= c_{XX}[k] - c_{YY}[k] + j(c_{YX}[k] + c_{XY}[k]).$ $

Set to zero

If $\tilde{c}_{ZZ}[k] = 0$ for all $k$ , then both real and imaginary parts vanish: $c_{XX}[k] = c_{YY}[k] \quad \text{(equal auto-covariances)},$ $c_{XY}[k] = -c_{YX}[k] \quad \text{(anti-symmetric cross-covariances)}.$

Significance

Proper complex processes arise naturally in communications when the in-phase and quadrature components of a signal have identical statistics and are related by a Hilbert transform. The $\ntn{gauss_c}(0, \sigma^2)$ distribution is proper. From the course (slide 382 onward), we adopt the convention that complex-valued processes are proper unless stated otherwise.

Quick Check

Two zero-mean WSS processes $X(t)$ and $Y(t)$ satisfy $r_{XY}(\tau) = 0$ for all $\tau$ . Which of the following is true?

$X(t)$ and $Y(t)$ are independent

$X(t)$ and $Y(t)$ are both orthogonal and uncorrelated

$r_{XX}( au) = r_{YY}( au)$

Correction:

X(t)

and

Y(t)

are both orthogonal and uncorrelated

For zero-mean processes, $c_{XY}(\tau) = r_{XY}(\tau) - \mu_X\mu_Y^* = 0$ . Orthogonality and uncorrelatedness coincide.

Common Mistake: Assuming Cross-Correlation Is Even

Mistake:

Writing $r_{XY}(\tau) = r_{XY}(-\tau)$ , by analogy with the autocorrelation.

Correction:

The cross-correlation satisfies $r_{XY}(\tau) = r_{YX}^*(-\tau)$ , not $r_{XY}(\tau) = r_{XY}(-\tau)$ . In general, $r_{XY}$ is neither even nor odd. Only the auto-correlation has the even symmetry property.

Orthogonal vs. Uncorrelated vs. Independent Processes

Concept	Condition	Zero-mean simplification
Orthogonal	$r_{XY}(\tau) = 0$ for all $\tau$	Same as uncorrelated
Uncorrelated	$c_{XY}(\tau) = 0$ for all $\tau$	Same as orthogonal
Independent	Joint fdds factor for all orders	Implies uncorrelated (not converse)

🔧Engineering Note

System Identification via Cross-Correlation

If a BIBO-stable LTI system with impulse response $h[n]$ is excited by zero-mean white noise with $r_{xx}[k] = \sigma^2\delta[k]$ , then the input-output cross-correlation is $r_{yx}[k] = \sigma^2 h[k]$ . This gives a direct method for estimating $h[k]$ : inject white noise and cross-correlate the output with the input. In wireless systems, this principle underlies pilot-based channel estimation, where known pilot symbols play the role of the white-noise input.

Cross-Correlation

$r_{XY}(\tau) = \mathbb{E}[X(t)Y^*(t-\tau)]$ for jointly WSS processes. Measures the linear dependence between two processes at different time instants.

Related: Autocorrelation Function

Orthogonal Processes

Two processes with $r_{XY}(\tau) = 0$ for all $\tau$ . For zero-mean processes, equivalent to being uncorrelated.

Related: Cross-Correlation

Key Takeaway

The cross-correlation $r_{XY}(\tau)$ measures the linear relationship between two processes at different time instants. Unlike the autocorrelation, $r_{XY}(\tau)$ is generally neither even nor maximized at the origin. The key relationship $r_{YX}[k] = (h * r_{XX})[k]$ for an LTI system is the foundation of the Wiener filter and pilot-based channel estimation.

Cross-Correlation and Cross-Covariance