1
$\begingroup$

I need to filter some array of data before differentiating it, I do this so I don't drown the signal in noise. It's also important that no delay is applied to the signal, that's why I decided to use a forward-backward filter like scipy's sosfiltfilt and implemented the following:

order = 4
freq = 0.00025

sos = sc.signal.butter(order, freq, output='sos', fs = 1)

t = np.linspace(t0, tf, points)
data_filtered = sc.signal.sosfiltfilt(sos, data, padtype=None)

plt.figure(figsize = (10, 5))
plt.plot(data, 'k', label='Raw')
plt.plot(t, data_filtered, 'r', label='Filtered')
plt.legend(loc='best')
plt.show()

Which generates the following data: Edge artifact at the end of the filtered data

Both the original data and the filtered data can be found there.

An artifact appears where the end is flattened, this is a problem since the derivative will artificialy fall to 0 for those values.

I tried playing around with sosfiltfilt parameters, padding doesn't help and higher orders flatten more data.

Why does this happen and how can I fix it? Is there a different solution for my needs?

PS: Don't really know how to provide the data if you want to try it yourself but I'm happy to do so, thanks!

$\endgroup$
5
  • $\begingroup$ Unfamiliar with sosfiltfilt but if n_outs = n_inputs, there's always padding, even if not explicit. Yes it helps to get data, try pastebin.com or ufile.io . There's also trigdiff (Ctrl + F "trigonometric" -- code) $\endgroup$ Mar 24, 2023 at 21:02
  • $\begingroup$ Hi @Guille, from the code it looks like You are applying a low pass filter on the "data". The pass band relative width is too small and it is natural/expected to have flat characteristics in the filtered data. Flat characteristics represent presence of DC component. Can You please share more details on why this is been seen as artefact ? Is the result different using some other tool(matlab/octave)? $\endgroup$
    – SakSath
    Mar 25, 2023 at 2:02
  • $\begingroup$ Hi @SakSath yes I failed to mention it was an LP filter. I thought of it as an artifact because the filtered data follows closely the unfilitered one then abruptly separates at the end. Why would a signal with DC component show flat regions like this? Perhaps my understanding of filtering is lacking and this is not anomalous, in that case could you point me towards some resources where this behaviour is explained? Thank you! $\endgroup$
    – Guille
    Mar 27, 2023 at 14:51
  • $\begingroup$ DC Component is a term often used in electrical engineering to represent currents that "Do not change direction(sign)". In the present context DC component is that component of the signal that stays constant through out the signal's existence. The graph of constant signal is a "flat line". As far as my understanding goes, the last portions of Your signal predominantly is a combination of DC component and frequencies that fall into "stop band" region of the filter you chose. The low-pass filter attenuates the high frequencies components and allows the DC (flat line) component to pass through. $\endgroup$
    – SakSath
    Mar 28, 2023 at 7:22
  • $\begingroup$ The data isn't in an easy to parse format. If text, should be comma-separated. $\endgroup$ Mar 28, 2023 at 11:28

1 Answer 1

1
$\begingroup$

The padding length was too short, tried this:

import numpy as np
import scipy.signal as sp
import matplotlib.pyplot as plt

order = 4
freq = 0.00025
[b,a] = sp.butter(order, freq, fs = 1)
fpad = sp.filtfilt(b, a, imported_data, padlen=10000)

plt.figure()
plt.plot(imported_data, 'k-', label='input')
plt.plot(fpad, 'c-', linewidth=1.5, label='pad')
plt.show()

And it worked.

$\endgroup$
1
  • $\begingroup$ Can You share the plot of output and raw with the padding length added ? $\endgroup$
    – SakSath
    Apr 1, 2023 at 3:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.