Separating 'signal' from periodic 'noise'

Question

I have data in which the signal we want is a few gaussian peaks superposed on a noisy background in which some of the 'noise' is periodic. Simply suppressing the associated fourier modes is effective (e.g., a 'notch filter') but also suppresses the signal, since it has some power associated with those frequencies.

What is a good approach for removing these periodic signals without removing the corresponding signal?

This figure shows the data (top; with X-axis labeled by pixel #) and the absolute value of its fourier transform (bottom; X-axis is pixel frequency) with relevant regions identified.

An approach I've considered is least-squares fitting a series of sine functions with frequencies limited to the identified range and the signal-containing region masked out, but this is computationally expensive and it is difficult to automatically identify the affected frequency range.

Answer 1

If you just need to detect when a gaussian peak occurs, you could try this:

Use the scipy hilbert function to determine the amplitude of the signal.

Pass the result through a high pass to remove the DC component.

Pass the result through a low pass to remove high frequency noise.

Use a threshhold function to detect peaks.

This will tell you when a peak occurred, and should be good at rejecting constant frequency and amplitude signals, as well as rejecting white noise of more or less constant amplitude.

It won't necessarily give you the shape of the peak. The peak you will see in a plot of the result (before the threshholding) may well resemble the original peak, but it won't be a perfectly accurate representation of it.

The values of the high pass and low pass filters can be juggled to get rid of noise, and to provide an appropriate pass band for the signal you are looking for.