I am designing a frequency lowpass filter using FFT&IFFT. My test data is a time series with a large DC component (the pink curve). Before the FFT, the time series was detrended (curve fitting) to remove the trend (the green curve).
My problem is: the detrended time series has discontinuity at the two ends which will cause severe spectral leakage. To improve the result, I tried several approaches:
1) applying windows (e.g., Hann window), it helps to force the ends to zeros. However, the windowing causes the amplitude reduced in the time domain, the frequency amplitude changes as well, as result, the restored signal using IFFT will have lower amplitude, and it cannot be recovered by multiplying certain coefficients (because the windowing procedure is not reversible).
Dividing the whole time series into small segments with 50% overlap, applying like Hann window to each sub-segment helps to solve the above amplitude change issue, but my actual project scenario requests me to do FFT using the longest data length, so this is not a way to go.
2) Instead of windowing, another approach I tried is simply align the beginning and end point to remove the discontinuity, this reduce the spectral leakage caused by discontinuity, but because of the alignment, the DC component is not zero any more, which also introduce leakage to the low frequency band.
Based on my limited knowledge, I cannot figure out a better idea, looking for experts' suggestions, thanks