What methodology to use for discrimination of different (musical?) tones

Question

I am trying to research and figure out how best to attack this problem. It straddles music processing, image processing, and signal processing, and so there are a myriad number of ways to look at it. I wanted to inquire as to the best ways to approach it since what might seem complex in the pure sig-proc domain might be simple (and already solved) by people who do image or music processing. Anyway, the problem is as follows:

If you forgive my hand drawing of the problem, we can see the following,:

From the above figure, I have 3 different 'types' of signals. The first one is a pulse that sort of 'steps' up in frequency from $f_1$ to $f_4$, and then repeats. It has a specific pulse duration, and a specific pulse-repetition time.

The second one only exists at $f_1$, but has a shorter pulse length and a faster pulse repetition frequency.

Finally the third one is simply a tone at $f_1$.

The problem is, in what way do I approach this problem, such that I can write a classifier that can discriminate between signal-1, signal-2, and signal-3. That is, if you feed it one of the signals, it should be able to tell you this signal is so and so. What best classifier would give me a diagonal confusion matrix?

Some additional context and what I have been thinking of thus far:

As I said this straddles a number of fields. I wanted to inquire as to what methodologies might already exist before I sit down and go to war with this. I dont want to inadvertently re-invent the wheel. Here are some thoughts I have had looking from different view points.

Signal-Processing Standpoint: One thing I have looked at was doing a Cepstral Analysis, and then possibly using the Gabor Bandwidth of the cepstrum in discriminating signal-3 from the other 2, and then measuring the highest peak of the cepstrum in discriminating signal-1 from signal-2. That's my current signal-processing working solution.

Image-Processing Standpoint: Here I am thinking since I CAN in fact create images vis-a-vis the spectrograms, perhaps I can leverage something from that field? I am not intimately familiar with this part, but what about doing a 'line' detect using the Hough Transform, and then somehow 'counting' the lines (what if they are not lines and blobs though?) and going from there? Of course at any point in time when I take a spectrogram all the pulse s you see might be shifted along the time axis, so would this matter? Not sure...

Music-Processing Standpoint: A subset of signal processing to be sure, but it occurs to me that signal-1 has a certain, perhaps repetitive (musical?) quality that people in the music-proc see all the time and have already solved in maybe discriminating instruments? Not sure, but the thought did occur to me. Perhaps this stand point is the best way to look at it, taking a chunk of the time domain and teasing out those step-rates? Again, this is not my field, but I heavily suspect this is something that has been seen before... can we look at all 3 signals as different types of musical instruments?

I should also add that I have a decent amount of training data, so perhaps using some of those methods might just let me do some feature extraction which I can then utilize K-Nearest Neighbor with, but thats just a thought.

Anyway this is where I stand right now, any help is appreciated.

Thanks!

EDITS BASED ON COMMENTS:

• Yes, $f_1$, $f_2$, $f_3$, $f_4$ are all known in advance. (Some variance but very little. For example, lets say we know that $f_1$ = 400 Khz, but it might come in at 401.32 Khz. However distance to $f_2$ is high, so $f_2$ might be at 500 Khz in comparison.) Signal-1 will ALWAYS step on those 4 known frequencies. Signal-2 will ALWAYS have 1 frequency.

• Pulse repetition rates and pulse lengths of all three classes of signals are also all known in advance. (Again some variance but very little). Some caveats though, pulse repetition rates and pulse lengths of signals 1 and 2 are always known, but they are a range. Fortunately though, those ranges dont overlap at all.

• The input is a continuous time series coming in at real time, but we can assume that signals 1, 2, and 3 are mutually exclusive, in that, only one of them exists at any point in time. We also have a lot of flexibility on how much of a time chunk you take it to process at any point in time.

• The data can be noisy yes, and there might be spurious tones, etc, at bands not in our known $f_1$, $f_2$, $f_3$, $f_4$. This is quite possible. We can assume a med-high SNR just to 'get started' on the problem though.

Answer 1

Step 1

Compute the STFT $S(m, k)$ of the signal using a frame size smaller than the pulse duration. I assume that this frame size will still offer sufficient frequency discrimination between f1, f2, f3 and f4. $m$ is the frame index, $k$ is the FFT bin index.

Step 2

For each STFT frame, compute the dominant fundamental frequency using something like YIN, along with a "pitch confidence" indicator, such as the depth of the DMF "dip" computed by YIN.

Let us call $f(m)$ the dominant f0 estimated at frame $m$ and $v(m)$ the pitch confidence detected at frame $m$.

Note that if your data is not that noisy, you can get away by using the autocorrelation as a pitch estimator, and the ratio of the larger secondary peak of the autocorrelation to $r_0$ as a pitch confidence indicator. YIN is cheap to implement, though.

You can also compute the total signal energy $e(m)$ of FFT frame $m$.

Step 3

Consider a sliding window of $M$ STFT frames. $M$ is chosen to be larger than the pulse repetition time and 5 to 10 times smaller than the typical length of a signal segment (for example if an occurrence of a signal lasts for about 10s, and your STFT frame size is 20ms, you can chose $M = 50$).

Extract the following features:

• $\sigma_f(k)$ is the standard deviation of the sequence $(f(m))_{m \in [k - M, k + M], v(m) > \tau}$
• $\sigma_v(k)$ is the standard deviation of the sequence $(v(m))_{m \in [k - M, k + M]}$
• $\sigma_e(k)$ is the standard deviation of the sequence $(e(m))_{m \in [k - M, k + M]}$

Intuitively, $\sigma_f$ measure the frequency stability of the main pitched component of the signal, $\sigma_v$ measure the variability of "pitchiness" of the signal, and $\sigma_e$ the variability of the amplitude of the signal.

This will be the features to base your detection upon. Signals of type 1 will have a high $\sigma_f$ (variable pitch) and a moderate $\sigma_v$ and $\sigma_e$ (steady pitched signal strength). Signals of type 2 will have a low $\sigma_f$ (steady pitch) and a high $\sigma_v$ and $\sigma_e$ (variable strength). Signals of type 3 will have a low $\sigma_f$ (steady pitch) and a low $\sigma_v$ and $\sigma_e$ (steady strength).

Compute these 3 features on your training data and train a naive bayesian classifier (just a bunch of gaussian distributions). Depending on how good your data is you could even get away with classifiers and use hand-defined thresholds on the features, though I don't recommend this.

Step 4

To process an incoming stream of data, compute the STFT, compute the features, and classify each window of $M$ STFT frames.

If your data and classifier are good, you will see something like this:

1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 3, 3

This delimits quite well the start and end time, and type of each signal.

If your data is noisy, there must be spurious misclassified frames:

1, 1, 1, 1, 1, 2, 1, 1, 1, 2, 2, 3, 2, 2, 1, 1, 1, 3, 1, 1, 1, 3, 3, 3, 2, 3, 3, 3

If you see a lot of crap like in the second case, use a mode filter on the data over neighborhoods of 3 or 5 detections ; or use HMMs.

Take home message

What you want to base your detection upon is not a spectral feature, but aggregated temporal statistics of spectral features over windows which are at the same scale as your signal durations. This problem really calls for a processing on two time scales: the STFT frame on which you compute very local signal properties (amplitude, dominant pitch, pitch strength), and larger windows over which you peek at the temporal variability of those signal properties.

Answer 2

An alternative approach could be four heterodyne detectors: Multiply the input signal with a local oscillators of 4 frequencies and low pass filter the resulting outputs. Each output represents a vertical line in your picture. You get the output at each of the 4 frequencies as a function of time. With the low pass filter you can dial in how much frequency deviation you want to allow for and also how fast you want the outputs to change, i.e. how sharp the edges are.

This will work well even if the signal is fairly noisy.