0
$\begingroup$

I'm currently working on an application that can be used to determine when a signal changes due to a server delay. Essentially, I have an API that is used to output data to a UI. However, for reasons I have yet to discover, there is the occasional lapse in server service, resulting in my API simply transmitting old data to the UI resulting in small intermittent flat-lines.

Signal Sample

The above is a sample of one of the signals, where each color represents a days worth of data, and I'm getting a data point every 5 minutes. As you can see, in the middle, the data is being delayed. Note that all signal segments have the same number of data points; the data in the bad sections is simply repeated.

Does anybody know of an effective metric to distinguish between the bad and good signals? I need to be able to look at the data sample for a single day and determine if its bad or not. My first thought was to look at the frequency spectrum (Fourier Space) and see what the frequency distribution is like, but I cant seem to be able to isolate any one reliable metric.

Any Help is greatly appreciated!

Improve this question
$\endgroup$
7
  • $\begingroup$ I'm looking ideally for a relatively simple solution. I have thought about resorting to a Neural Network (RNN most likely) but would rather avoid it if possible $\endgroup$ – user43805 Jun 21 '19 at 13:47
  • $\begingroup$ looking closely, there are flat sections in the time series but there are also places where there is a line with a linear slope. plotting routines will draw a straight line between data points. Are your data points time stamped? $\endgroup$ – user28715 Jun 21 '19 at 14:17
  • $\begingroup$ Yes, the data is time stamped. The linear slopes occur because the server doesnt realize there is an issue and simply sends the same data value until the new data comes in. I cant actually do anything to the underlying server itself though. $\endgroup$ – user43805 Jun 21 '19 at 14:29
  • $\begingroup$ I suppose one possible approach is to simply look for the flat gradient areas and if I'm getting more than a certain number flat areas per unit time, I could throw a warning $\endgroup$ – user43805 Jun 21 '19 at 14:31
  • 1
    $\begingroup$ those sections with linear trends that aren’t horizontal, imply gaps. if the server is simply repeating the data point, why not test every new point against the last point? $\endgroup$ – user28715 Jun 21 '19 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.