Applying a lowpass filter to a noisy square signal leads to a shift of the signal

My knowledge of analog and digital signal analysis is behind me.. thus I do not have enough confidence in my procedure to analyze my signal, and I do not know which direction to pursue to improve. I'm measuring a noisy square signal at a sampling frequency fs = 1e6 Hz. I can increase this frequency up to 1e9 Hz if needed.

At a frequency fs = 1e6 Hz, I get the following signal: Above, you can see 2 examples with 2 square waveforms at 2 different amplitudes. As you can see, the signal is prone to overshoot, in the example below, before the square starts; and is also noisy. Thus, I wanted to clean the signal by applying a low-pass filter.

I tried the 2 following: a Butterworth and a bessel filter using the scipy library in Python.

def butter_lowpass_filter(data, cutoff, fs, order=5):
nyq = 0.5 * fs
normal_cutoff = cutoff / nyq
b, a = butter(order, normal_cutoff, btype='low', analog=False)
y = lfilter(b, a, data)
return y

def bessel_low_pass_filter(data, cutoff, fs, order=5):
nyq = 0.5 * fs
normal_cutoff = cutoff / nyq
b, a = bessel(order, normal_cutoff, btype='low', analog=False)
y = lfilter(b, a, data)
return y

I then call both filters on my data with the settings cutoff = 100000 # Hz, fs = 1e6 # Hz, order = 8. Why those settings? That is my first problem, I don't really know how to choose the cutoff frequency or the order. I simply tried different values, and those seem to be nice.

The result looks like this (left, low amplitude pulse; right, higher amplitude pulse): The bessel filter seems to work better than the Butterworth, however both have the same problem. The signal is shifted to the right. If I zoom , I observe the clear shift: The opposite shift seems to be present at the end of the square. Is it possible to correct this shift? My end goal is to measure both the amplitude and the width of the square. Do you think my approach is correct or would you recommend me another approach?

Thanks for the guidance, Mathieu

• I think this answer could be helpful. Aug 18 '20 at 15:58
• @MattL. Yes, that looks very promising, thank you! Aug 18 '20 at 21:41

The only possible way that any real-world filter can work is by using past values of the signal you're filtering to get the latest filtered value. So any real-world filter will always have some delay.

So if by "correct this shift" you mean "make it go away" -- not really.

However, if you want to look at the signal and its filtered version after the fact, then yes -- calculate the filter's group delay (or just eyeball it in your data), and in your graphs shift the input over by that amount. This is easier to do if you use a symmetric FIR filter, because it'll have a constant group delay and the group delay is just half the filter length.

If your goal is to measure the duration and magnitude, then as long as you can have some delay in the measurement then just take your filtered version and feed it into whatever duration-and-magnitude measuring algorithm you've cooked up. Since the leading and trailing edges of the square wave are offset by the same amount, the measured duration should be just fine.

• This answer gives me confidence that the shift is at least symmetric, and thus is equal at the end of the square and at the end of the square, thus not modifying the width of the square. That's great. However, it's also transforming the abrupt, almost vertical transition, into a more gentle transition with a gentler slope. This slope makes it hard to determine the width of the square. Do you know how I could improve this? Aug 18 '20 at 21:39
• Also, are the 2 filters I used FIR filters? Aug 18 '20 at 21:40
• I'd have to pick through the scipy documentation or look at the filters to see if they're FIR or not -- it depends on scipy's implementation. If they're "real" bessel or butterworth, they're not symmetrical. But again -- scipy. Aug 18 '20 at 22:51
• That's worth another question! Edges transition quickly because of high-frequency content; noise has high-frequency content. So you've got pretty much a direct tradeoff between the sharpness of the transition and the amount of noise you let through. Aug 18 '20 at 22:52
• Thanks a lot, now I get it. Not a great news, but at least I know what I'm working with. Aug 19 '20 at 8:14