Theoretical considerations for Using Finite Arrays to Represent Samples of Signals of Infinite Length

Question

Discrete signals can come from sampling continuous-time signals. A continuous-time signal can be of infinite support. For instance, a sin wave. In terms of mathematical description of signals and systems, it is assumed that a discrete signal is a function defined on $\mathbb{Z}$ and mathematical derivation proceed happily with that. However, in real applications of the theory on digital devices, we cannot directly store all samples of a sequence $x: \mathbb{Z} \to \mathbb{R}$, due to limited memory. There seems to be a gap between the mathematical description of the theory and the true signal played with on computers.

I am recently trying to use a numerical example to verify equations from a resample system. For instance, to generate a new sequence from an old one, so that the new sequence can be viewed as sampling the original continuous-time signal at a different rate. I would like to conduct the verification using a $\sin$ wave. The procedure goes as follows: first sample the $\sin$ wave with an interval $T_{1}$ to generate a sequence $x_{1}: \mathbb{Z} \to \mathbb{R}$. Then sample the $\sin$ wave with a new interval $T_{2}$ to generate a new sequence $x_{2}: \mathbb{Z} \to \mathbb{R}$, which is a resampled version of $x_{1}$. Then generate a sequence $x_{3}: \mathbb{Z} \to \mathbb{R}$ from $x_{1}$ directly using equations from the analysis of resample system to ensure that $x_{3}$ is also of a sampling rate $T_{2}$. For instance, use the concatenation of an interpolator, a low pass filter and a decimator. Then a verification can be made by comparing $x_{2}$ and $x_{3}$. They should be the same sequence.

When I executed the plan, I found an issue: $\sin$ is of infinite support, and equations from resample analysis assumes discrete signals of infinite support. Generating $x_{2}$ from direct math formula for sampling is not difficult. The problem is, $x_{1}$ in practice is a finite sequence which only represent truncated $\sin$ wave, and thus deviates from mathematical analysis of resampling. I am afraid that this deviation may cause the verification to fail, even if the equations are correct. Can someone provide ideas for my thoughts?

Answer 1

You are 100% correct.

The sampling theorem requires a signal to be bandlimited. This in turn, requires the signal to be of infinite support which doesn't exist in reality. That means, you can NOT sample a real world signal without some amount of error. By choosing sampling parameters carefully you can reduce the error to whatever is "good enough" for your specific application, but some error will always be there.

Resampling is a good example of this problem. You can do it on paper using sine waves and sinc functions, but as soon as you store numbers in a computer all the nice theory goes out the window and it get's rather messy.

Sample rate conversion is a fairly complicated tradeoff between passband amplitude distortion, latency, phase/group-delay distortion, preservation of causality, residual aliasing, Mips and memory, etc.

I am afraid that this deviation may cause the verification to fail

It will indeed. You cannot resample a real world signal without some amount of error. You cannot even just sample a real world signal without error.

Answer 2

When you sample a signal with a computer, the computer has finite memory (plus storage), so that implies a finite sample size, which implies that you must be applying a window to any signal longer than the amount you have sampled, such as an infinite length one.

Thus you are not sampling your original infinite signal (a sinusoid), but an infinite signal that has had a window (usually rectangular) applied (a windowed sinusoid), and as a result is zero for an infinite span outside that window.

Windowing artifacts are to be expected. So are aliasing artifacts due any finite length window in the time domain having infinite support in the frequency domain.

There is no error or deviation if you don't falsely assume that you are sampling the longer infinite signal, but instead assume your have sampled a windowed signal, the window being the length (or less) of your sample set. Thus, there will be no verification error if you include the effects of all the windowing artifacts in your comparison theoretical analysis.

Answer 3

The accepted answer is wrong.

There's always discretization error in the sense that any number is representable to finite number of decimal points, but no one cares about that (in context of continuous <-> discrete).

It's perfectly possible to work with signals of infinite time and/or frequency support with zero error. That is, the end result of a discrete operation will be indistinguishable from directly sampling the result of a continuous operation (which is by definition exact).

Example: wavelet transform with Morlet wavelets, where we convolve with essentially a sinusoid windowed by a Gaussian. The wavelet decays exponentially, and with sufficient length, its amplitude will decay beneath float epsilon (smallest representable quantity). Eventually it'll decay enough that its product with input will also be beneath float epsilon; then, contributions of that portion to convolution will also be below float epsilon. The overall error is then beneath float epsilon - and the result is exact (-ly same as sampling the convolution computed analytically in continuous time).

While it's possible, it's far from trivial to attain in most applications; instead we settle for some acceptable error. Bottom line: the complete infinite representation is almost never required.