Yesterday I asked a question and got the answer to detrend my time series which I think is really better than applying a highpass filter. So I read the description of scipy.signal.detrend and there I learned that there is a parameter bp:
bp : array_like of ints, optional
A sequence of break points. If given, an individual linear fit is performed for each part of data between two break points. Break points are specified as indices into data. ...
I tested this feature with this program:
import math import numpy as np from scipy.signal import detrend from matplotlib import pyplot as plt low = np.zeros((1500), dtype = float) high = np.zeros((1500), dtype = float) for i in range(500): p1 = i * math.tau / 1000 p2 = i * math.tau / 500 p3 = i * math.tau / 50 c1 = -math.cos(p1) c2 = math.cos(p2) / 2 + 1/2 c3 = math.cos(p1) c4 = math.cos(p3) / 2 low [i + 0] = c1 low [i + 500] = c2 low [i + 1000] = c3 high[i + 0] = c4 high[i + 500] = c4 high[i + 1000] = c4 raw = low + high detrended = detrend(raw, bp=[500,1000]) trend = raw - detrended fig, (ax1, ax2, ax3, ax4, ax5) = plt.subplots(nrows = 5, ncols = 1, sharex = True) ax1.plot(low, linestyle="solid") ax1.set_title('low frequency data') ax2.plot(high, linestyle="solid") ax2.set_title('high frequency data') ax3.plot(raw, linestyle="solid") ax3.set_title('to be detrended = low + high') ax4.plot(detrended, linestyle="solid") ax4.set_title('detrended data') ax5.plot(trend, linestyle="solid") ax5.set_title('trend') plt.tight_layout() plt.show()
And I got this diagrams as output:
I put the break points at exactly that two points where the low frequency curves has its maxima.
As you can see, there are jumps in the trend line at the break points and therefore also the detrended signal has jumps that didn't exist in the raw data. This is exactly what I feared and expected, but not what I hoped to get.
I hoped to get something like this:
The trendline still has sharp kinks (which is fine), but it no longer has jumps. Therefore, there are no jumps in the detrended data either.