Applying a lowpass filter to a noisy square signal leads to a shift of the signal

Question

My knowledge of analog and digital signal analysis is behind me.. thus I do not have enough confidence in my procedure to analyze my signal, and I do not know which direction to pursue to improve. I'm measuring a noisy square signal at a sampling frequency fs = 1e6 Hz. I can increase this frequency up to 1e9 Hz if needed.

At a frequency fs = 1e6 Hz, I get the following signal:

Above, you can see 2 examples with 2 square waveforms at 2 different amplitudes. As you can see, the signal is prone to overshoot, in the example below, before the square starts; and is also noisy. Thus, I wanted to clean the signal by applying a low-pass filter.

I tried the 2 following: a Butterworth and a bessel filter using the scipy library in Python.

def butter_lowpass_filter(data, cutoff, fs, order=5):
    nyq = 0.5 * fs
    normal_cutoff = cutoff / nyq
    b, a = butter(order, normal_cutoff, btype='low', analog=False)
    y = lfilter(b, a, data)
    return y

def bessel_low_pass_filter(data, cutoff, fs, order=5):
    nyq = 0.5 * fs
    normal_cutoff = cutoff / nyq
    b, a = bessel(order, normal_cutoff, btype='low', analog=False)
    y = lfilter(b, a, data)
    return y

I then call both filters on my data with the settings cutoff = 100000 # Hz, fs = 1e6 # Hz, order = 8. Why those settings? That is my first problem, I don't really know how to choose the cutoff frequency or the order. I simply tried different values, and those seem to be nice.

The result looks like this (left, low amplitude pulse; right, higher amplitude pulse):

The bessel filter seems to work better than the Butterworth, however both have the same problem. The signal is shifted to the right. If I zoom , I observe the clear shift:

The opposite shift seems to be present at the end of the square. Is it possible to correct this shift? My end goal is to measure both the amplitude and the width of the square. Do you think my approach is correct or would you recommend me another approach?

Thanks for the guidance, Mathieu

Answer 1

The only possible way that any real-world filter can work is by using past values of the signal you're filtering to get the latest filtered value. So any real-world filter will always have some delay.

So if by "correct this shift" you mean "make it go away" -- not really.

However, if you want to look at the signal and its filtered version after the fact, then yes -- calculate the filter's group delay (or just eyeball it in your data), and in your graphs shift the input over by that amount. This is easier to do if you use a symmetric FIR filter, because it'll have a constant group delay and the group delay is just half the filter length.

If your goal is to measure the duration and magnitude, then as long as you can have some delay in the measurement then just take your filtered version and feed it into whatever duration-and-magnitude measuring algorithm you've cooked up. Since the leading and trailing edges of the square wave are offset by the same amount, the measured duration should be just fine.