Autocorrelation - Stochastic vs deterministic processes

Question

On one hand, there is the statistical definition:

$R_x(\tau) = \frac{\operatorname{E}[(X_t - \mu)(X_{t+\tau} - \mu)]}{\sigma^2}$

and on the other hand, there is this other definition (e.g. see Dilip's comment here):

$R_x (\tau) = \int_{-\infty}^{\infty} x(t) x(t+\tau) {\rm d}t$

Why are these two definitions different? Is the first one defined for stochastic processes, while the second one defined for deterministic signals?

If so, does the Wiener-Khinichin theorem hold for the deterministic variant? (the Wikipedia article only discusses it in the context of a stochastic process)

Answer 1

You are correct that these two definitions are for different processes.

$R_x\left(\tau\right) = \frac{1}{\sigma^2} \mathbb{E}\left[\left(X_t - \mu\right)\left(X_{t+\tau} - \mu\right)\right]$

is defined for a stochastic process $X$ distributed according to a law $\nu$.

Whilst for a deterministic, finite-energy signal, $x\left(t\right)$ the autocorrelation is:

$R_x\left(\tau\right) = \int_{-\infty}^\infty x\left(\tau+t\right)x\left(\tau\right)dt$

The Wiener-Khinchin theorem will hold for any square-integrable deterministic $x\left(t\right)$ that is:

$\int_{-\infty}^\infty |x\left(\omega\right)|^2 d\omega < \infty$

so that the Fouier integral exists and converges in the $L^2$ sense (i.e. signals with finite energy).

The two definitions are different because the integrals used in the definitions are different. The deterministic case implicitly assumes a Riemann integral - that the process is nice in some way - it is continuous, and if differentiable the derivatives are bounded etc. However for random processes, $dt$ is difficult to define - the process could jump from $-\infty$ to $\infty$ over any inteval I could possibly define, so the Riemann limit isn't appropriate.

Instead we use the Lebesgue integral, where we can define the expectation of a random process: all that's required is that the range is smooth (for example $\mathbb{R}$), and we can still take the expectation with respect to some measure, modulo some restrictions on the random process (so that the integral converges in a monotone or dominated sense).

Integration of Random Process is a tricky business, and the definitions are written differently to keep people mindful of what they are working with.

One other note - the deterministic case isn't so interesting, as we can take the FT of a deterministic signal: the whole point of the WK theorem is for random processes - where the Fourier integral may not exist. I find it a particularly satisfying theorem, and the work to understand it definitely worthwhile.