Let's calculate: Suppose we have a $\underline{x} \sim \mathcal{N}\left ( \underline{0}, \sigma_x^2 I_N \right )$, where $I_N$ is the $N \times N$ identity matrix.
Denote by $\bf{W}$ the unitary DFT matrix, and by $\bf{W}^H$ its Hermitian transpose.
Note that $\bf{W}\bf{W}^H=\bf{W}^H\bf{W}=I_N$, thus, $\underline{X} = DFT\left ( \underline{x}\right ) = \bf{W} \underline{x}$ and
\begin{equation}
E\left \{ \underline{X} \right \} = E\left \{ \bf{W} \underline{x} \right \} = \bf{W} E\left \{ \underline{x} \right \} = \underline{0},
\end{equation}
\begin{equation}
COV\left \{ \underline{X} \right \} = COV\left \{ \bf{W} \underline{x} \right \} = \bf{W} COV\left \{ \underline{x} \right \} \bf{W}^H = \sigma^2I_N.
\end{equation}
so in the frequency domain, $\underline{X} \sim \mathcal{N}\left ( \underline{0}, \sigma_x^2 I_N \right )$, thus, you can take the mean in the frequency domain as well as in the time domain and the estimation will be minimum variance unbiased.
@PeterK. The white noise is uniformly distributed in the frequency domain, therefore, average is not even an unbiased estimator for a constant in uniformly distributed noise
– Amro Apr 18 '16 at 03:06