0
$\begingroup$

I have a data signal which is generated at constant time interval. It is a real world measurement and thus exhibits some noise in its value (it fluctuates approx + or - 20 units). The noise is fairly consistent.

I obtain the measurement at a regular time interval and stash it inside a rolling buffer of sample data (done via a computer). This rolling buffer is a list of values which are located at positions (0,1,2,3,4,5... -> n) in the list. Once the nth position in this list is reached, my buffer rolls over and overwrites the value at the 0th list position followed by the value at the 1st list position and so on until the nth value is overwritten (then rollover again). The most current data point value is always tracked (so I always know the position in the list of values at which I am writing my data to). This data buffering process repeats over and over. I use the rolling sample data to calculate a moving average and a std deviation.

Now for my problem. The signal has the potential to exist in 3 states, increase at a constant rate or to decrease at a constant rate at any time t1 or to remain constant. An increase or decrease can suddenly happen at any point in my list and will continue to some time t2 after t1.

I want to detect whether the signal is Increasing, Decreasing, or is remaining Constant (within the typical noise profile) with reasonable statistical certainty. What sort of algorithm would effectively notice this?

$\endgroup$
  • $\begingroup$ i dunno what significance there is in the "rolling buffer". i presume you mean a delay line which any FIR filter has. it's memory and the index "wraps around" from the end of the block of memory to the beginning. yet a time-invariant algorithm will still be a function of the most current input sample $x[n]$, and all of the past samples in memory, $x[n-1], x[n-2], ...x[n-N]$. $\endgroup$ – robert bristow-johnson Jul 4 at 4:15
  • $\begingroup$ First thing that comes to mind is low-pass filtering, differentiating, and thresholding. Use a hysteresis-based thresholder for a little improved robustness. Have you tried that yet and if yes, why would it not satisfy your needs? $\endgroup$ – Florian Jul 4 at 8:21
  • $\begingroup$ Can I please ask if this was resolved? $\endgroup$ – A_A Sep 6 at 8:58
0
$\begingroup$

What sort of algorithm would effectively notice this?

What you are describing, is an online algorithm that uses a small set of "past" measurements to make an inference, in this case, about the slope of the signal.

Since you are keeping track of the current beginning and end of the rolling buffer, it is essentially like having access to a linear buffer of N past samples.

So, provided that the phenomenon you are trying to study (your signal) evolves slower than the length of the buffer you have chosen, you can apply a plain simple slope estimation on the values of the buffer and then use the slope estimate value (and its residuals) to decide if the signal is mostly heading downwards or upwards or remains constant. The most simple "classifier" here, would be a plain simple threshold one, possibly with the addition of hysteresis as it has been suggested already. More complex "classifiers" could be monitoring both the first and second derivative and returning additional information on whether the signal also changed direction within a buffer.

If you want to be getting one new estimate every time a new sample is received, you would have to look for iterative (or "recursive") methods, (for example, iterative least squares).

Hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.