Variance of periodogram estimate of the power spectrum

Question

I have been reading chapter 13.4. ("Power Spectrum Estimation Using the FFT") of the Numerical Recipies Book.

Some things related to the expectation value of the "periodogram estimate of the power spectrum" became not clear to me though.

Background

Suppose we have an equally-spaced $N$-point sample $ c_0 ... c_{N-1} $ of the discrete signal $ c(t) $ and its Fourier transform $$ C_k = \sum_{j=0}^{N−1} = c_j \exp{ 2πijk/N} $$ $$k = 0, . . . , N − 1 $$ Then we can define the "periodogram estimate of the power spectrum" $P(f_k)$ as $$ P(f_k) = \frac{1}{N^2} [{|C_k|^2 + |C_{N−k}|^2]} $$

with $k = 1, 2, . . . , \frac{N}{2}− 1$ and $f_k = \frac{k}{N\Delta}$ for $k = 0, 1, . . . ,\frac{N}{2}$ ($\Delta$ is the sampling interval).

So far everything is clear.

Now the book says

"... the variance of the periodogram estimate at a frequency $f_k$ is always equal to the square of its expectation value at that frequency. In other words, the standard deviation is always 100 percent of the value, independent of N!"

My questions

1. What is the expectation value of the periodogram estimate at a frequency $f_k$? Why (can you show me how to derive it)?
2. The expectation value doesn't change with $N$, so changing the length of the input doesn't improve the power spectrum estimation. But if I have a non-integral number of periods in my sample, more samples (longer sampling time) means less weight on the end of the signal (where there is a disconnect) so less smearing of frequency components measured due to the disconnect between the beginning and the end of the signal. So in this case a higher $N$ means a better estimation? Where am I wrong?
3. If the SD is always 100 percent of the expectation value, doesn't that make this simple periodogram estimation (using just a single DFT of $N$ samples) pretty much useless? Should one always opt for Welch's or Bartlett's method e.g.?

Thank you very much in advance!

Answer 1

There are two sources of error in estimating a quantity: variance and bias. In your question you have only considered variance.

The improvement in the spectrum estimate using the periodogram comes because there is less smearing of frequency components as you include more samples in your estimate of the power spectrum. Using a longer data window causes less power from one frequency to be leaked into other frequencies. However, this only reduces the bias of the estimate.

So, as you use more samples the bias is decreased, which means you do have a better estimate. However, the variance remains unchanged, as stated in Numerical Recipes. A discussion of this is given on this web page:

http://matlab.izmiran.ru/help/toolbox/signal/spectra7.html

Answer 2

As an example variance of a periodogram of a Gaussian process is given as:

$\ var\{{I_{xx}}\} = S^2_{xx}e^{(j\Omega)}.[1 + (sinN\Omega / Nsin\Omega)^2]$

As $\ N \rightarrow \infty $ denominator of the second part in the parenthesis goes to infinity and the division goes to zero because of the N but summing with 1 prevents all equation to become zero. So variance converges to the square of the PSD, not the zero.