1
$\begingroup$

I apologize if this is not the correct forum, but I thought I'd ask.

I have an array of signal samples and I want to detect (retroactively, once I have measured the time/magnitude values) if there are any spikes / deviations during a sampled real signal How would I do this intelligently?

I've been computing the mean of all my sets and the standard deviation and seeing if any of the individual samples greatly deviate from the mean, but I'm thinking there must be a more established method or approach

Improve this question
$\endgroup$
4
  • $\begingroup$ This question might be more suitable for the DSP or the statistics stackexchanges. $\endgroup$
    – hotpaw2
    Commented Mar 14, 2014 at 0:35
  • $\begingroup$ There are many established methods for outlier detection, as a search on that term will reveal. One established method is to find points more than X standard deviations away from the mean, which it sounds like you have tried. Perhaps you could add a plot of your data or some explanation of what problems you are having with this method. $\endgroup$
    – John
    Commented Mar 14, 2014 at 0:48
  • $\begingroup$ As @John says, there are straightforward approaches. We really need to understand what the "signal model" is for the data you are seeing. If it's just a constant, and then it changes, that's one thing. If it's time-varying and you want to figure out when it's varying differently that's something else. $\endgroup$
    – Peter K. ♦
    Commented Mar 14, 2014 at 1:39
  • $\begingroup$ If you tell us what language you are using then perhaps we can already propose a library that will do the job. As you said, it is not an uncommon task. $\endgroup$
    – Lukasz Tracewski
    Commented Mar 14, 2014 at 14:41

1 Answer 1

Reset to default
4
$\begingroup$

This type of problem is part of Detection Theory. Do you have an estimate of the background noise levels? How about the statistics of the noise? If the noise is AWGN, things get a little easier. You can set a detection threshold based off your noise statistics to achieve a desired probability of detection / probability of false alarm.

Be careful about using the mean of your data to set the threshold. The "spikes" could bias your mean estimate. If the noise is not stationary (i.e. it changes levels), using the mean could result in false detections (alarms). One simple option is to compute a running average and create a threshold off this running average. If the sample of interest is above the threshold, declare a detection and do not include the sample in the running average calculation. There're a ton of variations to this simple concept depending on the situation.

If your signal is truly a flat line without noise, you could apply a highpass filter by subtract the previous sample from the current sample. The result would be an array of zeros with non-zero sample values at the spikes.

For a more formal treatment of detection theory, I highly recommend Steven Kay's book "Fundamentals of Statistical Signal Processing, Volume II: Detection Theory".

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.