# Which filter to use for data with high amount of errors?

I have the following data collected from a heart rate sensor.

The data is in bpm vs. milliseconds. All the data points below 80 is incorrect (errors/noise). When it plummets below 80, it's because the heart rate sensor fell off the subject.

Similarly, although on this graph it is not there, there are often spikes that go as high as 500. Those high peaks are caused by something physically tapping the heart rate sensor. The rate should ideally be between 100-160. However, I'm not sure what filter to apply to correctly display this data.

I tried applying a 100 point median filter. This got rid of majority of the low and high peaks. However, I was told that this removes too much of the detail and that I should use an average filter so that it more accurately describes the data. My argument against this is that $\frac{100 + 0}{2} = \frac{50+50}{2} = 50$. Many different values can give me the same average.

I do not come from a DSP background. Hence, how should I proceed? Are there other types of filter that will more accurately let me display this data? Ideally, I'd like to keep the general trend, but not include the sudden dips(any data that dips below 80) and spikes(any data that shoots above 200bpm).

• It may be just me, but if you have problems with your sensors, filtering the data will only result in filtered sensor errors, so taking care of the sensors, in a different manner, may be a better approach. – a concerned citizen Feb 23 '17 at 7:38
• Hmm, Good point! – Christian Feb 23 '17 at 8:29
• I would simply exclude samples that you know are erroneous so that they do not contribute any information to the result, and replace those values with NaN (not a number). – Dan Boschen Feb 25 '17 at 16:37