What happens when we oversample?

Question

I had an interview for a wireless communication position and one of the interviewers asked me this question in regard to signal processing.

If I have signal and I sample at the Nyquist frequency and then it goes through a quantizer and if we have the same signal and goes through oversampling and same quantizer, which of the two cases would result in higher signal quality at output and why?

I was thinking that since both are not under-sampled then both should be OK. But maybe not.

My thinking is the second, but I am not sure.

Answer 1

First, the Nyquist rate is an absolute lower bound to what will possibly work if you're sampling a strictly bandlimited signal. The closer you get to the Nyquist rate when you're sampling, the more expensive your system gets in various ways.

In particular, the closer you sample to the Nyquist rate, the sharper the cutoff you need in your reconstruction filter. The sharper the cutoff of a filter is, the longer it takes to settle. A perfect lowpass filter with a boxcar response has an infinite settling time. Just for reasons of the reconstruction filter alone, you want to sample some amount above Nyquit -- 2x or 3x is comfortable, 10x or more is not out of bounds (although things get expensive if you oversample too, which is what keeps engineers gainfully employed).

Second, quantization noise can usually be treated as being uniformly distributed random noise that's white after sampling. For a given amount of quantization, the higher your sampling rate, the lower the noise spectral density will be compared to your desired signal. So you can sample fast, then lowpass filter to recover your signal, and you lose some of that quantization noise.

Answer 2

Usually, these kinds of questions are somehow intentionally ambiguous, just for you to take well-justified assumptions.

Ideally, both solutions could give the same results. However, signals are usually not band-limited. What anti-alias filter is being used before the acquisition? The transition band of this filter, no matter how high its order is, will never be zero width. Thus, sampling at Nyquist rate is almost never a good idea. Taking a higher sampling rate allows your spectrum repetitions to be more separated, thus letting the transition band do its work on band-limiting the signal.

On the other hand, oversampling has a price: you need a faster clock, need to do more processing, and in general terms, need a "more expensive" solution. I believe the question was meant to make you discuss some of theses topics.

Answer 3

Actually, the oversampled signal will have less quantization noise in the original baseband (those are the frequencies below the original Nyquist frequency, which is half of the original samplerate which, in your hypothetical is the Nyquist rate).

The power of the quantization noise is the variance of the quantization error of a uniform quantizer, which is $\frac{\Delta^2}{12}$ (where $\Delta$ is the step size of the uniform quantizer). If the quantization is good and random (which it normally is when the signal swing is much larger than the quantization step size) then that noise power is uniformly distributed over all frequencies between DC and Nyquist. The higher Nyquist is, then the wider that rectangle (the integral of the power spectrum) is, but the area of the rectangle is still $\frac{\Delta^2}{12}$. That means the noise density is lower everywhere including the original baseband.

If noise shaping is not used in the quantizer, then you have to oversample by a factor of 4 just to get one additional meaningful bit (6 dB more S/N ratio) in the signal in the baseband.

Answer 4

If you sample a pure non-zero sinusoid at exactly its Nyquist rate, the output could well be zero, depending on the phase relationships. Obviously a garbage result.

If you sample a signal that is uncorrelated with the sample clock, or contains noise than is uncorrelated with the sample clock, then the higher the sample rate, the more any quantization noise will be spread out over a wider spectrum, away from the signal(s) of interest.

Answer 5

If we go mad and inject an element of real world signal processing, an ordinary FM bandwidth of 20,000 Hz will carry speech and/or music for a standard broadcast signal, on a VHF/FM transmission.

Where you have a continuous audio signal for sampling, the Nyquist rate is double the highest frequency (i.e. double the bandwidth). Hence a signal ranging from 0 to 20,000 Hz implies a Nyquist rate of 40kHz. If the sampling rate is at least equal to this, the resulting digitised signal will be free of aliasing distortion. Hence, standard sampling rates of 44,100 Hz and 48,000 Hz are commonly used in commercial radio broadcasting.

We might describe the use of 48,000 Hz as over-sampling. In practice, because the difference from 44,100 Hz is not very great, it is not generally regarded as 'true' oversampling.

Historically, there have been higher sampling rates. For instance, once upon a time DAT audio (Digital Audio Tape) used a sampling rate of 96,000 Hz -- double the more usual 48,000 Hz.

Obviously, you can potentially get greater resolution -- better signal quality -- if you double the sampling rate, since you are reducing the 'step' size, in fact halving it. But you are doubling the effective data rate too. From an economic standpoint this implies a greater cost, on transmission (and on storage space), even though in engineering terms it represents a 'purer' signal.

The lowpass filter will pass through all signal not exceeding 20,000 Hz. But by sampling at not less than double this rate, you avoid the economic expense of requiring a too sharp cut-off, and a too long settling duration, in this filter.

Sampling at the higher rate of 96kHz -- so called "oversampling" (sampling at greater than Nyquist rate) -- has engineering benefits, i.e. sound quality benefits.

Firstly, for any given level of quantization (16-bit quantization is mostly used), the higher the sampling rate the lower the quantization noise (inherent to 16-bit sampling) as there are fewer rounding errors in the maths. By reducing the quantization "noise", it thereby improves the SNR (signal-to-noise ratio).

This was a factor which made DAT attractive, due to it having a sampling rate which, at 96kHz, was double the standard 48kHz sampling rate.

Secondly, as quantization noise is random (i.e. evenly distributed), the quantization noise spreads randomly across (half of) the sampling frequency; and since this spread broadens as the sampling rate increases, the higher rate of 96kHz allows the lowpass filter to remove more of it.

Accordingly, the conclusion is that using the Nyquist rate of 40kHz (or any higher rate) for sampling the audio will avoid any aliasing distortion, and will avoid any economic drawbacks in the cost of the lowpass filter. Also, sampling at a higher rate than the Nyquist rate will provides audio quality improvements, at some increased economic cost.

Answer 6

Without getting into much details on sampling and quantization, one generic answer can be made. Let us first take a distant point of view. You have a common quantization operator $Q$, and two sampling options: one at Nyquist, sampled at times $kT_N$ by some method $S_N$, and one oversampled by a factor $\alpha\ge 1$, therefore at times $kT_N/\alpha$ by some (potentially different) method $S_o$.

By design, the Nyquist version is a special case of one oversampled version, by taking $\alpha=1$ and $S_o=S_N$. Therefore, the oversampled case may be at least as good as the Nyquist case. So if properly designed to cope with the signal and the noise properties, one can hope to be better with oversampling for $\alpha\ge 1$. However, a poor choice for $S_o$ may also lead to underperformance.

Then in a second time, one can talk about some expected qualitative properties, like a better spread quantization noise, an overall better fidelity to the analog signal.

Finally, if you possess a finer knowledge on a specific oversampling techniques, you can provide more technical and quantitative details.

Going from the general to the particular is something I may expect from interviews when I ask this kind of questions.