Removing "spikes" in data

Question

Imagine I have a data set which gives me the following ListLinePlot

Is it possible to define a function over the data set that removes the "spikes" observed in the data and smoothens the plot, for a given threshold?

Please look at examples from the GaussianFilter docs pages under Applications. — Syed, Nov 07 '22 at 17:08
Is there a known reason why there are spikes? I ask because it doesn't appear that there's just one odd point involved but leading up to the spike and trailing the spike there are many points that seem to imply that a spike is a real thing. Providing sample data would be helpful but the appropriateness of any "solution" will depend on why the phenomenon exists. — JimB, Nov 07 '22 at 17:14
@JimB The function looks like a Fourier transform of some signal. — yarchik, Nov 07 '22 at 20:30
@yarchik I believe you. My point in my comment is that the goodness of a smoothing is not an intrinsic property of the data. Subject matter and the objective can supply that information as opposed to a "I'll know it when I see it" approach. Data is (are?) good but not completely sufficient. — JimB, Nov 07 '22 at 22:33
related https://mathematica.stackexchange.com/questions/88112/how-to-smooth-the-noise and https://reference.wolfram.com/language/guide/DataTransformsAndSmoothing.html — IntroductionToProbability, Nov 09 '22 at 10:53
Related to my first comment: Are the spikes the issue or the troughs the issue? Again, that would depend on what process is generating the data which should be described. — JimB, Mar 28 '24 at 04:33

score 3 · Answer 1 · answered Mar 28 '24 at 00:23

3

data = 
 Table[Evaluate @ Re @ FourierSeries[t^5, t, 4], {t, -3. Pi, 3. Pi, 6. Pi/128}];

Using GaussianFilter

ListLinePlot[Table[GaussianFilter[data, n], {n, {0, 6, 12, 24}}],
 PlotLegends -> {"data", 6, 12, 24},
 PlotRange -> All]

answered Mar 28 '24 at 00:23

eldo

score 1 · Answer 2 · answered Mar 28 '24 at 04:07

data = 
 Table[Evaluate@Re@FourierSeries[t^5, t, 4], {t, -3.  π, 3.  π, 6.  π/128}];

Grabbing the @eldo's data and using LowpassFilter:

smoothedData = LowpassFilter[data, 0.3];
ListLinePlot[{data, smoothedData}, 
PlotLegends -> {"Original Data", "Smoothed Data"}, PlotRange -> All]

2 Answers2