0

Sorry if this is a super easy question, I am really not a signals guy (more of a statistician, computer scientist). But I was trying to reconstruct/learn a sinusoidal function $x(t)$ with linear regression $f(x) = \langle w , \phi(x) \rangle $ (I'm adding as many polynomial features as I may need) over a fixed interval $[-1,1]$. Then I realized that there is this thing called the Nyquist-Shannon sampling theorem that might help figure out if I have enough samples. The reason that I care about it is because the performance of my regression model is really bad on the test set and I wondering if I am just not getting enough samples from my ground truth signal (which for the moment I control synthetically). I know now that I need my sample frequency $f_s$ to be larger than twice the band width $B$ of my signal:

$$ f_s > 2B$$

to my understanding bandwidth is just the range of the frequencies present in the signal $x(t)$ (I even asked to clarify that to me just in case here: What is the definition of the bandwidth of a signal?). Thus if that is correct then band width is:

$$ B = f_{max} - f_{min} $$

is there a general way to get $f_{max}$,$f_{min}$ from a signal in the time domain?

If I understand this correctly then I just need to get the frequencies that define $x(t)$? Right?

If that is correct then the only way I know is if we know the analytic form of the $x(t)$, which I do for my example, then I just read of the frequencies from the Fourier series $ f(x) = \frac{1}{2}a_0 + \sum^{\infty}_{n=1} a_n cos(nx) + \sum^{\infty}_{n=1} b_n sin(nx) $. Say $n_{max},n_{min}$ are the largest values modifying the inside of sin or cos terms then we read them off and do:

$$ B = \frac{n_{max} }{2 \pi} - \frac{n_{min}}{2 \pi} $$

right? What happens if there is only 1 single term like it only a sin like $x(t) = sin(n t)$, how do we get the smaller frequency if there is no other frequency?

Now that I understand much better what the term frequency means I assume that the answer will point me that if I don't know the analytic form of my signal (as probably happens most in practice) we need to resort to some type of transform used. Probably the Fourier transform? But a quick google search to the frequency domain yielded a list of methods:

  1. Fourier series – repetitive signals, oscillating systems
  2. Fourier transform – nonrepetitive signals, transients
  3. Laplace transform – electronic circuits and control systems
  4. Z transform – discrete-time signals, digital signal processing Wavelet transform - image analysis, data compression

I don't know when to apply each but it seems the wikipedia article links to more articles for each one...

Is this correct or am I totally off? (which might be the case since I'm not very familiar with this field). I am assuming there are probably engineering details I am totally missing like flow pass filters or something else that might be important for real signals in practice. Though its just a random guess based on the definition of low-pass filter (filters high frequencies out). Though not sure.

Further reading What is the definition of the bandwidth of a signal? makes me suspect that the actual correct answer is as follows:

  1. Consider a time domain signal $x(t)$.
  2. Consider its frequency description $X(\omega)$

the the support of $X(\omega)$ is the bandwidth. In other words, "the summations" of the points in the domain (to be more precise, the integral of the domain). So if the domain is the whole interval $[-W,W]$ the the bandwidth would be:

$$ B = \int^{W}_{-W}d \omega = 2W $$

so in general is it:

$$ B = \int_{w \in supp(X(\omega))} d\omega $$

would be my guess.

Charlie Parker
  • 121
  • 1
  • 1
  • 6
  • Your last paragraph is correct, except that by convention one only counts the positive frequencies; that is, the bandwidth would be $W$, not $2W$. Also: the bandwidth of a sine wave is zero; if you limit it to the range $[-1,1]$, however, you're multiplying it by a rectangular pulse and then its bandwidth becomes infinite. – MBaz Oct 20 '17 at 16:35
  • @MBaz but the Nyquist-Shannon sampling theorem must apply to also signals (functions in time say) that have countable frequencies like things with fourier series, right? In the simplest example is considering $x(t) = sin(kt)$. I feel there must be a way to sample it optimally, no? – Charlie Parker Oct 20 '17 at 23:10
  • 1
    Yes, the sampling theorem applies to periodic and aperiodic signals. So, a signal like a pure sine wave with frequency $f_0$ can be reconstructed from samples taken at a rate $f_s>2f_0$. There are other, simpler ways to specify that signal, though: for instance, by specifying its frequency, amplitude and phase, or by projecting it on other orthonormal bases. Is this useful? I'm not sure I understand what your question is, specifically. – MBaz Oct 20 '17 at 23:19
  • @MBaz yes that was useful. I guess my question seems to be a bit more related to the definition of bandwidth or how Nyquist-Shannon theorem changes when applied to signals with countable frequencies. So $B=\int_{w\in supp(X(\omega)} d\omega$ or $B=f_{max} - f_{min}$, or Nyquist-Shannon theorem must be something different depending on the signal. I am not sure which one it is though. – Charlie Parker Oct 21 '17 at 03:29
  • 1
    I get your question now. See chapters 6--9 of this free book: http://www.afidc.ethz.ch/A_Foundation_in_Digital_Communication/Home.html – MBaz Oct 21 '17 at 14:32
  • @MBaz thanks! I will take a look at it. Its just funny as someone from a different community reads the title as "theorem" and then it seems that the statement of the theorem depends on the context. Without much experience here but more with (mathematical) theorems, it seemed rather odd to me. I am sure it would get clarified if I went through the proof or something but I thought it was a standard easy question that could be answered quickly (specially if its a theorem), the reason my question(s) exist. Regardless, I appreciate your help. – Charlie Parker Oct 21 '17 at 19:10
  • The sampling theorem does not depend on context, and it uses $f_{max}$ to define the bandwidth. The problem is that the question "do I have enough samples" has a more complicated answer. The sampling theorem gives sufficient, but not necessary conditions for sampling, and it requires a signal of infinite duration. Under some conditions, some signals require less samples than specified by the sampling theorem. – MBaz Oct 24 '17 at 01:45

1 Answers1

1

As far as I know there is no way to get $f_{max}$ and $f_{min}$ directly from time domain (zero crossing rate will not help if the signal has a non-zero DC offset for example) you need to look at the frequency domain.

If there is just one term, the smaller frequency is just $0$

Given a time signal and its sampling rate ($Fs$), you can take the discrete Fourier Transform to study its frequency spectrum. Unfortunately you cannot use this to decide your sampling rate if your initial sampling rate ($Fs$) is lesser than the 2$*$bandwidth (2$*$[highest frequency] if you are looking for perfect reconstruction) as you have already lost the data during initial sampling(look up aliasing for further reading).

Bottom Line: You need to have an estimate in what frequency range your signal will lie so that you could sample it accordingly.

itismeghasyam
  • 152
  • 6
  • I guess what I am seeing from your answer is that in some cases bandwidth = highest frequency? Or maybe just the Nyquist-Shannon theorem statement changes depending on the input function/signal in question? – Charlie Parker Oct 20 '17 at 23:12
  • 1
    Yes and No. Bandwidth does not have a concrete definition. It can be assumed as the band of frequencies having 90% or 95% of the signal energy, or full width at half maximum etc., depending on the need. I mentioned highest frequency because if you capture that one frequency through sampling, you can capture all frequencies lower than that. Some signals might have energies in a band shifted to a higher frequency due to modulation, but we do not sample it necessarily according to the highest frequency of the band, if we can demodulate the signal and bring the frequencies to their actual range. – itismeghasyam Oct 21 '17 at 16:41
  • trivial question, is the reason if there is only 1 signal the reason the lowest is zero because a constant term (say +0) is the same as a frequency of zero, or why do we treat the smallest frequency as zero? – Charlie Parker Oct 21 '17 at 19:05
  • 1
    yes, a constant term has to be represented by a zero frequency (not true for piecewise constants, it has to be constant all throughout). A constant $c$ can be looked up as $c*cos(w_0 t)$, with $w_0 = 0$ – itismeghasyam Oct 21 '17 at 21:13