5

So I work in this domain of biophysics that has to do with a light-based detection for measuring small movement of molecules (nanometer and piconewton scale) via a Quadrant Photodiode. This signal contains lots of information but is riddled with noise. One of the challenges is denoising this signal and while conventional methods such as savitsky-golay tends to work well there are set cutoff and threshold values that go into this method which makes it not as feasible.

Time-series traces from this measurement look like a sawtooth curve and as the particle moves in space and time, the noise changes (so noise is the not the same everywhere) (Figure attached below).

My question is - I have noise measurements from this signal (I have recordings where sawtooth event never happens and only noise is left). Can I train a self-supervised learning method to denoise this signal using my known noise recordings? For example - is there a high-frequency bandpass filter that takes in some noise and can be trained to automatically smooth this curve to what we might expect the ground truth to be? Is there a better approach to it? If my question is unclear please let me know and I can provide more information.

Thanks.Figure1

Royi
  • 19,608
  • 4
  • 197
  • 238
SpaceCadet2810
  • 75
  • 4
  • Have you looked into Wiener filtering? – Jdip Sep 16 '22 at 00:00
  • Look for “wiener noise reduction”. – Jdip Sep 16 '22 at 01:05
  • Sounds good. Let me look into it. Do you have a specific link (or paper) in mind? – SpaceCadet2810 Sep 16 '22 at 01:18
  • If you are using a high pass to measure the noise alone, this implies you can just filter the signal with the complementary low pass (result will be the same). If the noise is not stationary, you will not be able to measure the noise in one interval and then subtract it in another interval. Are you able to get higher SNR samples for evaluation? If so, comparing the spectrums will be informative as to the effectiveness of filtering and the characteristics of the noise – Dan Boschen Sep 18 '22 at 13:28

2 Answers2

3

I'll assume the frequency content of the noise overlaps the frequency content of your signal of interest (otherwise a fixed filter would work):

  1. You might want to look into Weiner noise reduction. The paper Improved Signal-to-Noise Ratio Estimation for Speech Enhancement and its matlab implementation can be good starting points (it’s aimed at speech enhancement but I believe the theory would apply to your problem, you’d have to modify it to use the noise statistics at each analysis step because as is, this implementation assumes stationary noise).
  2. An other effective approach is spectral substraction: define windows where you expect the noise to be somewhat stationary, perform spectral substraction and reconstruct.
lennon310
  • 3,590
  • 19
  • 24
  • 27
Jdip
  • 5,980
  • 3
  • 7
  • 29
  • Thank you - I really like your idea of spectral subtraction. Let me look into that! – SpaceCadet2810 Sep 16 '22 at 16:05
  • Great! Let me know how that goes and if you need more detail – Jdip Sep 16 '22 at 20:16
  • Wiener works when you have a spectral information about the signals. How can it be effective in this case over any other frequency based filtering. – Royi Jun 13 '23 at 10:44
  • @Royi The op mentions having noise only measurements. So spectral information about the noise is available, which is theoretically sufficient for both methods I proposed, isn’t it? – Jdip Jun 13 '23 at 11:27
2

You may have a look at the method called JOT: A Variational Signal Decomposition Into Jump, Oscillation and Trend (You may access it in A Two Stage Signal Decomposition into Jump, Oscillation and Trend Using ADMM).

This method basically does what you're after, it decomposes the signal into 3 signals:

enter image description here

You may look on the results of a signal similar to yours:

enter image description here

The method is quite simple if you know ADMM.
In any way, they supply code.

Royi
  • 19,608
  • 4
  • 197
  • 238