# Changepoint Detection of a 2D signal

Consider the 2D signal below I would like to find the changepoints in the signal (using a hierarchical approach). Based on our visual observation, we can see there are 2 main changepoints where the signal 'starts' and 'stops' and we could argue there is potentially another one where the red starts to come in. We could continue to look in more and more granularity if we so wished.

My question is, how could we get a computer to detect these automatically. My intuition from the way I did it was to do as follows:

1. Select the first point as our starting point.
2. Move along the time axis with another point, finding the Fourier transform of the data between the first point and this point.
3. As long as the transformed signal is not become too much more complex (in the information sense), we know that we have not hit any big change.
4. Once the signal starts to become more complex (above some threshold level), we select this as our new starting point and repeat.

After this process is finished we now have selected an array of points where our signal changes drastically w.r.t. its spectral make up.

My question then is:

1. Does this make sense?
2. What metric should I use for the complexity of the Fourier transform?
3. Has this already been tackled successfully and if so any references?

One approach would be to use the spectral flux of a sequence of STFT frames $X(n,k)$, namely

$$\mathit{SF}(n) = \sum_{k = -N/2}^{N/2 - 1}{H(\left|X(n,k)\right| - \left|X(n-1,k)\right|)},$$

where $n$ is the frame number, $N$ is the size (in samples) of each frame, and $H(x) = \frac{x + \left|x\right|}{2}$ is the half-wave rectifier function. Intuitively, it attempts to measure how much the spectrum changes from frame to frame.

The overall area of research is called onset detection. A good overview of techniques can be found in the following paper (which also describes spectral flux):

S. Dixon. Onset detection revisited. In Proceedings of the 9th International Conference on Digital Audio Effects, volume 120, pages 133-137, 2006.

An example implementation of spectral flux can be found in the NSGToolbox.

I don't have the time to write a fully detailled answer but here's my 2 cents :

I don't think that looking for the "complexity" of the Fourier transform is a good solution.

You are looking for a magnitude change, xum your signal along the time axis. The 1D signal is simpler and it has one dominent frequency. Estimate the dominent frequency, put it in phase with you signal using correlation, demodulate your signal using a product detector and a low-pass filter. You should have a nice envelop to work on where a thresolding/clustering heuristic could land good result.

On the other hand, I think you will be able to devise more simple strategy starting from the 1d signal + dominent frequency.

I've seen such problem in non-intrusive load monitoring.