non-consistent normalization problem in fft

Question

I have a csv file of data sampled with Ts=1ns which looks like this:

This signal is a step response of some system which responds to a step of value 1. I'm trying to get the impedance profile of the signal by using $$ Z\left( f \right)=\frac{V\left( f \right)}{I\left( f \right)}=\mathcal{F}\left\{ \frac{1}{A}\frac{\partial }{\partial t}v\left( t \right) \right\} $$ The signal is noisy and needs filtering, so i'm separating the droop part from the non-droop part and filtering them independently using savgol filter.

Than, i'm applying FFT on each of the signal parts above using the following code:

n = 1024
from scipy.fftpack import diff
frqs = np.fft.fftfreq(n, d=1.0 * 1.e-9)[range(n // 2)]
Y = np.fft.fft(diff(data_y, order=1), n=n, norm='ortho')  # fft computing and normalization
Y = abs(Y[range(len(frqs) // 2)])
return frqs, Y

Next, I take the high frequencies from the FFT applied on the droop part and the rest from the non-droop part and I stitch the results and plot them in log scale. We get a result with expected shape as below:

Now, the problem arise when we change the droop part to be longer (and thus, the non-droop part to be shorter) as seen here:

The output of this looks like this:

The noisy shape is expected, but the amplitude is changing drastically and I can't figure out why. can someone point what the problems are are help with a solution?

Thanks!

Answer 1

I believe you need to scale by the sample period $\Delta T$ properly in your differentiator (divide by $\Delta T$) if your time units are in seconds not samples.

Your differentiator can be arrived at by using the backward Euler approximation of a continuous time differentiation.

The mapping from s to z using the backward Euler

$$s \leftrightarrow \frac{1-z^{-1}}{\Delta T}$$

$1-z^{-1}$ is simply subtracting the past sample from the current sample. To maintain your time axis in seconds, divide by $\Delta T$. If your time axis is in samples (as we often do), the $\Delta T=1$ and further scaling is not needed.

Similarly an integrator which is $1/s$ would be multiplied by $\Delta T$. For all other mappings you see the same $\Delta T$ appear, just as it exists as $dt$ in the continuous time equations.

Answer 2

So I solved it eventually. The root of the problem wasn't in the fft to start with. I've changed np.diff(data) to np.gradient(data) and the amplitudes were consistent. Thanks!