# How to estimate the local trend in a signal?

I need to remove trend from my time-series which looks like the following images.

However, I want to estimate the trend before removing it. Hence directly removing it won't do it.

The simple polynomial models didn't work, because — I think — the signal is long and the trend is changing. Also, filtering the signal with low pass removed too much of the trend as well.

The files are available here.

• high-pass filter? or compute sliding median and subtract that from the data? Commented May 21, 2023 at 8:04
• Which simple polynomial models? References, please Commented May 21, 2023 at 8:49
• @robertbristow-johnson, highpass? it is a low frequency phenomenon, no?
– Mark
Commented May 22, 2023 at 8:04
• @RodrigodeAzevedo, i used matlab detrend mathworks.com/help/matlab/ref/detrend.html.
– Mark
Commented May 22, 2023 at 8:05
• "trend" and "local" are contradictory terms. Which trend exactly do you want to remove ? (Please a more descriptive answer than "the one on the plot".)
– user67664
Commented May 22, 2023 at 9:12

Start simple: just use a 1-D median filter of an appropriate length.

If I do that with a length of 100 samples, I get the following for your first signal.

The top plot shows the original signal (blue) and the median filtered signal (red). The bottom plot shows just the zoomed-in median filtered signal.

Another possibility is to use a DC Blocker.

## Code Below

load signals_samples.mat
clf

for k=1:10

figure(k)
clf
subplot(211)
plot(signals_samples{k})
hold on
plot(medfilt1(signals_samples{k},100),'r')
subplot(212)
plot(medfilt1(signals_samples{k},100))

end

– Mark
Commented Jul 5, 2023 at 16:30
• @Thomas D'oh! I see what you mean now. Thanks for the clarification. Done.
– Peter K.
Commented Jul 5, 2023 at 17:26

You may have a look at the method called JOT: A Variational Signal Decomposition Into Jump, Oscillation and Trend (You may access it in A Two Stage Signal Decomposition into Jump, Oscillation and Trend Using ADMM).

This method basically does what you're after, it decomposes the signal into 3 signals:

You may look on the results of a signal similar to yours:

The method is quite simple if you know ADMM.
In any way, they supply code.

You signals looks very long, so it might stress the solver for memory.
Still worth a try.

Similar to the JOT method suggested by @Royi, the BEADS algorithm decomposes a signal in three components:

• a trend,
• a sparse signal,
• a residual (which can be used as quality control).

I did a quick test on your first signal. BEADS is not (yet) good at the beginning and end, but it is quite fast: 0.25 seconds for the first signal. It scales almost linearly with signal size. Its parameters allow to control the smoothness of the trend. Here are the results. Parameters follow.

data1 = [signals_samples{1}];
% Filter parameters
fc = 0.0035;     % fc : cut-off frequency (cycles/sample)
d = 1;          % d : filter order parameter (d = 1 or 2)

% Positivity bias (peaks are positive)
r = 1;          % r : asymmetry parameter

% Regularization parameters
% amp = 0.8;
amp = 0.1;
lam0 = 1*amp;
lam1 = 5*amp;
lam2 = 2*amp;
[data1Filt, dataBaseline, cost] = beads(data1, d, fc, r, lam0, lam1, lam2);

• What can be done to improve this?
– Mark
Commented Jul 26, 2023 at 5:52
• I am not sure of what you need to improve here. There are many options. The simplest is to perform a preprocessing such that both ends are tapper to zero Commented Aug 4, 2023 at 12:25