This signal represents the distance in an experimental device. As the speed is constant, the signal should be linear (untill it drops). However, for some reason the device showed interferences as shown in the figure below:

enter image description here

The question is how can I remove this high amplitude interferences, making the signal as linear as possible?

I've already tried the butterworth filter and read something about the Chebyshev, but I don't think it applies.

Any suggestions?

  • $\begingroup$ Can you share the data? $\endgroup$ Jun 21, 2016 at 18:44
  • $\begingroup$ @LaurentDuval Yes, download a sample on this link $\endgroup$ Jun 21, 2016 at 19:54
  • $\begingroup$ Same value -0.00082681 everywhere? $\endgroup$ Jun 21, 2016 at 20:02
  • $\begingroup$ @LaurentDuval This is only the beginning of the file, represented by the horizontal line near zero in thebeginning of the plot. Note that this file has 512000 values. $\endgroup$ Jun 21, 2016 at 20:36
  • $\begingroup$ Apparently this part is flat. No noise $\endgroup$ Jun 21, 2016 at 20:46

2 Answers 2


There are a few simple things that you could try:

  • sliding-window median filtering with an appropriate window size
  • linear-regression in a sliding window
  • robust linear regression to the total signal, if you know the signal duration etc well enough

EDIT: Further idea after a comment from OP on another answer

Since you want the slope at 160000 be the same as before (i.e. no flat section in your signal), you could use a robust linear regression technique such as RANSAC with an appropriate number of sample points to analyze/filter your data.

  • 2
    $\begingroup$ Especially I would like to mention bout the nonlinearity of the required filter. Some nonlinear algorithms such as a variation of median filtering is necessary. A linear filter will have quite difficulty in eliminating spike type noise which has flat spectral distribution over the whole signal spectrum. $\endgroup$
    – Fat32
    Jun 21, 2016 at 16:39
  • 2
    $\begingroup$ Yes, that is very correct. Thanks, I will remove the Fourier-suggestion since it really does not perform in this case. $\endgroup$
    – M529
    Jun 21, 2016 at 18:39

Non-linearity or non-least-energy seems a key, as stated by @Fat32.

If you want to denoise first, a sliding weighted median (with positive weights) could help you remove impulse noise, while keeping a more accurate location of transitions than a pure median. A Matlab implementation here. If this does not suffice, I'll go for more general rank filters. Median is sort of rank $1/2$. If the impulse noise is too heavy, a $\max$ or a $\min$ filter (toward rank $1$ or $0$) may better follow the slope.

If around $160000$ you want to go through following the apparent (local) slope, then a weighted-median derivative, or a sliding robust linear regression (a robust LOWESS) could produce local derivatives without prior filtering.

  • $\begingroup$ Around 160000 I'm expecting that the curve follows the pattern, that is, a diagonal line at the same slope as before. $\endgroup$ Jun 21, 2016 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.