1
$\begingroup$

I have a sample of graphs (more than 10000...). that look like in the image below: enter image description here

I am searching for an unsupervised learning algorithms that can help me to detect anomalous observations.

Here what I suggest for beginning: for every observation I have a collection of points $(x,y)$. With this collection, I find Fourier series with regression (I compute coefficients with the base $\{1,\sin(x),\cos(x),\sin(2x),\cos(2x)\dots\}$). Now I have a set of coefficients instead of waves.

Somebody have an idea how to detect anomaly?

$\endgroup$
  • $\begingroup$ Do you have the data the graphs were made from, or just the graphs? $\endgroup$ – JRE Dec 1 '14 at 14:39
  • $\begingroup$ I'm not sure Fourier is the best approach. How about looking at continuity? If two values of x are close but the ys are vastly different, that could be an outlier? $\endgroup$ – barrycarter Dec 1 '14 at 17:36
  • $\begingroup$ @JRE I have the data. $\endgroup$ – dmitriy Dec 6 '14 at 10:20
  • $\begingroup$ @barrycarter But what is the statistics? How I detect anomaly with your method? $\endgroup$ – dmitriy Dec 6 '14 at 10:24
  • $\begingroup$ Are you looking for anomaly in a single graph or for anomal graph compared to other graphs? $\endgroup$ – Vertex Apr 2 '15 at 12:31
1
$\begingroup$

Try Nonparametric Statistics. You may divide your random trajectories into parts and compare their statistic characteristics (mean and variance). Smth like this (but not exactly):

http://www.cscjournals.org/manuscript/Journals/SPIJ/volume7/Issue1/SPIJ-210.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.