I have many datasets which all looks roughly like this:enter image description here

All the datasets have this in common. First there is an artifact around 0 with an unknown (but likely very high) intensity. Then there is some signal - in this case from ~25 to ~75. Then it's quite (between 75 and 150) and then there is another signal of an unknown characteristic.

My main goal is to analyze the signal which starts at 150, but in order to do that I want to locate a point after the first signal ends, and before the real signal start. i.e. somewhere between 75 and 150.

So far I have been using a very naive approach. Smooth/Blur the entire signal, find the first local maximum, and then locate the following local minimum. This generally works well, but once in a while you get a dataset like this one where the simple naive approach fails.

The reason it fails can be seen on the next image

enter image description here

I started blurring the signal with a kernel size of 5. It worked for most signals, but some signals (like this one) would have a local minimum before the local minimum I was looking for.

I then increased the kernel size to 30, which helped a lot. However, a few datasets (like this one) still managed to have a local maximum and local minimum too much.

Increasing the kernel size to 90 fixes this particular dataset, but it doesn't feel like the correct solution. I guess it's just a question of time before I find a new dataset which have the same problem again.

What would be a better approach to locate the first signal in this case?

  • 1
    $\begingroup$ can you provide a CSV or similar of your data (optimally of several of them)? I'd suggest an energy detector that detects regions of small energy, and then choose the second one. Are there any constraints on the period before the first signal starts (i.e. is it always 25?) and the period between both signals (i.e. is it always 150-75?)? Does the first part of the signal look similar always or can it be arbitrary? $\endgroup$ Jan 19, 2017 at 8:25

1 Answer 1


Ignoring the artifact, it seems the signals model is like a bimodal distribution.

In order to make the process more robust, I'd use binary thresholding methods.
I'd choose a method based on histogram like Otsu Method.
The threshold generated by this method will be in the flat zone you're aiming.

To summarize, I'd do as following:

  1. Apply low pass filter.
  2. Normalize data as it was an histogram (Sum of 1).
  3. Apply Otsu Method.
  4. Take the threshold as the index separating the 2 signals.

If you share some data I will implement it to show results.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.