1

Suppose $X$ is a real-valued N-dimensional Gaussian vector, $X \sim \mathcal{N}(\mathbf{0}, C_X)$. The discrete Fourier transform can be obtained by left-multiplying with the unitary DFT matrix, i.e. $\hat{X} = W X$ (where $W$ is defined as in this page). What is the distribution of $\hat{X}$?

My attempt to answer this is as follows: Since the DFT is linear, $\hat{X}$ will also be Gaussian, and thus $\hat{X} \sim \mathcal{N}(\mathbf{0}, WC_XW^\top)$... BUT that doesn't make sense to me; $\hat{X}$ is inevitably complex valued!

On the other hand, saying $\hat{X} \sim \mathcal{CN}(\mathbf{0}, WC_XW^H)$ doesn't make sense either; even if it is correct, what does it mean to have complex valued variances? Do we not have to account for the fact that $X$ is purely real?

DangerousTim
  • 25
  • 5
  • The closest I could find to an answer only mentions the element-wise variances, not the entire covariance matrix, and only addresses the specific case of zero-mean i.i.d noise. – DangerousTim Nov 11 '23 at 17:30

1 Answers1

1

As the DFT of real $X$ is conjugate symmetric, $\hat{X}$ is not N-dimensional jointly Gaussian and neither your two distributions is correct.

Representing the N-dimensional DFT by a $2N$ dimensional linear transformation $$Y =\begin{bmatrix}\hat{X}_{re}\\\hat{X}_{im}\end{bmatrix}=\begin{bmatrix}W_{re}&-W_{im}\\W_{im}&W_{re}\end{bmatrix}\begin{bmatrix}X\\0\end{bmatrix}=\begin{bmatrix}W_{re}\\W_{im}\end{bmatrix}X=\tilde{W}X$$

The covariance matrix $\tilde{W}C_X\tilde{W}^T$ is singular and the distribution is degenerate. You can find the subspace to which $\hat{X}$ is mapped by exploiting the conjugate symmetric property as follows.

Exclude the last $N/2$ lines of both $W_{re}$ and $W_{im}$, and the line corresponding to the imaginary part of DC from the matrix $\tilde{W}$ to get $\tilde{W}_{sub}$ (by $\tilde{W}_{sub}=S\tilde{W}$ with $S$ is a diagonal matrix with corrresponding 1 and 0 in its diagonal). Hence the dimensions of $\tilde{W}_{sub}$ is $(N-1)\times N$.

$\tilde{W}_{sub}C_X\tilde{W}_{sub}^T$ is non-singular and the $SY$ is a $(N-1)$-dimensional zero mean multivariate Normal with covariance $\tilde{W}_{sub}C_X\tilde{W}_{sub}^T$.

We know that the distribution of $SY$ fully characterizes $Y$, and $\hat{X}$, because the conjugate symmetric property implies that the dimension of $\hat{X}$ is at most $N-1$.

The $2N$-dimensional real $Y$, and the $N$-dimensional complex $\hat{X}$, has zero probability outside of the space of $SY$ and is impulsive from the view of such higher-than-$(N-1)$-real-dimensional density.

AlexTP
  • 6,585
  • 2
  • 21
  • 38
  • Thanks!! Could you please suggest a resource for understanding why $\hat{X}$ won't be Gaussian, and how to deal with degenerate Gaussian distributions? The wiki article goes into measure theory stuff, surely there must be a simpler way to understand it? – DangerousTim Nov 15 '23 at 06:34
  • My understanding is still shaky, but what if we instead excluded the last N/2 lines from the original complex matrix $W$ and set the imaginary DC part to zero - is this equivalent to your method? Can we then say $\hat{X} \sim \mathcal{CN}(0, \bar{W}C_X\bar{W}^H$? Intuitively I'm trying to get at a DFT matrix that corresponds to something like what numpy.fft.rfft does. – DangerousTim Nov 15 '23 at 06:59
  • @DangerousTim To understand why the elements of $\hat{X}$ are not jointly Gaussian, you can read Chapter 3 of "STOCHASTIC PROCESSES: THEORY FOR APPLICATIONS" of Robert Gallager. Specifically, 3.4.3 for real vectors and 3.7 for the complex model. – AlexTP Nov 15 '23 at 10:36
  • For the degenerate distributions, you might have a simpler way if you can exploit the structure of the non-singular covariance matrix induced by the DFT. I don't know how. – AlexTP Nov 15 '23 at 10:39
  • @DangerousTim For the question about manipulating directly the complex DFT matrix $W$, you can but this does not mean that the remaining part of $\hat{X}$ is circularly symmetric complex Gaussian like your notation because the resulting vector cannot be written as a linear transform of iid circularly symmetric gaussian random variables (note that the definition of complex gaussian vector is not standardized and I am using the definition of the aforementionned Gallager book and the Wikipedia article). – AlexTP Nov 15 '23 at 10:51
  • @DangerousTim you can argue that you don't need your complex $\hat{X}$ being circularly symmetric, but then (a) you need to check the assumption numpy lib, and (b) the distribution must be characterized by two matrix covariance and pseudo covariance if (c) you can find a way to write the distribution of my real $SY$ in the complex notations. I don't say (c) is impossible but I don't see how straightfoward. – AlexTP Nov 15 '23 at 10:55