1
$\begingroup$

I have a signal that is measured by an optical sensor that is connected to an ESP-32. I am trying to apply a band pass filter "in real time" on this microcontroller. The signal is comming in batches of a certain size from a buffer.

What I am trying to do is apply a Butterworth filter on these batches in a sequential manner. I am currently testing things out in Python using scipy. I am using a band pass filter with pass-band [0.35, 10]Hz.

sos = butter(2, [0.35, 10], btype='band', fs=1000, output='sos')
zi = sosfilt_zi(sos)
batch_size = [2048] # normally testing for various batch sizes
for i in batch_size:
    stream_filtered = []
    # afe1_led1 is a complete list of measurements, to be broken up into batches
    for j in range(0, len(afe1_led1)//i):
        batch = afe1_led1[j*i:(j+1)*i]
        filtered_data, zi = sosfilt(sos, batch, zi=zi)
        stream_filtered.extend(filtered_data/1000) 
        # the division by 1000 here is due to unexpected gain added by the filter

The output of this filtering process is clearly flawed. The output signal is scaled by 3rd order of magnitude (~ 1000) and is also not resembling the expected waveform. The output signal looks more like a 10Hz sinusoidal waveform when using a 4th order Butterworth. When using a 2nd order Butterworth, the output has even higher frequency components, adding detail to the 10Hz oscilation. As seen in the image, the blue signal is the expected waveform (filtered by a Chebyshev filter normally, as 1 big batch). The orange image is the output of the scipy code snippet.

enter image description here

My question is: what is causing this behaviour? What is the part I am missing in relation to using an IIR filter with a batched input, that is causing this aparent instability?

Edit 1:

Following the suggestion given by @Hilmar, I tested the filter using simple sinusoids. It does in fact work. However, when testing with the actual data, it still does not. By @Jdip's suggestion, you can access a segment of the actual data in a numpy.ndarray here: unfiltered data. Each element is float64.

Some additional information about the blue signal (expected). It is the filtered output of a Chebyshev filter using signal.sosfiltfilt(). Note this is different to signal.sosfilt() used in the code provided above. Although I do not believe this makes a big difference.

$\endgroup$
2
  • $\begingroup$ Can you share the data somehow? $\endgroup$
    – Jdip
    Commented May 8 at 20:57
  • 3
    $\begingroup$ The initialization of your state is wrong, but otherwise your code looks fine. What type is your original data? Is this fixed point? " # the division by 1000 here is due to unexpected gain added by the filter" That should NOT be the case and I don't see that when running it with a sine wave. Try a sine wave first $\endgroup$
    – Hilmar
    Commented May 8 at 21:36

1 Answer 1

1
$\begingroup$

Your code is fine. Before doing anything to a signal, it's good to look at its frequency content. This is the power spectrum of the sample you linked to (I took DC out):

enter image description here

Your signal is highly harmonic with its fundamental at 10Hz. So your bandpass filter with cut-offs at [0.35Hz 10Hz] is getting rid of everything in your signal with significant power, except some of the fundamental that leaks through because of the filter roll-off. What you end up with is a filtered signal that is basically a 10Hz sinewave with a little bit of the first few harmonics (less of them the higher the filter order).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.