How to recognize patterns in a noisy audio frequency spectrum?

Question

I have an audio spectrum (FFT) that looks like this:

101761.00 :                                 *                                                                                                                    
 76309.98 :                                                                                                                                                      
 57224.41 :                                                                                                                                                      
 42912.26 :       *     *                     *                                                                                                                  
 32179.65 :       *                                 *                                                                                                            
 24131.34 :                *      *       *                                                                                                                      
 18095.95 :                                                                                                                                                      
 13570.05 :          *             *     *                                                                                                                       
 10176.10 :                 *                *  *                                                                                                                
  7631.00 :                                  *  *                                                                                                                
  5722.44 :                           *     *       *                                                                                                            
  4291.23 :        *     * **               *  ** *                                                                                                              
  3217.97 :        *                       *              *                                                                                                      
  2413.13 :         *            *                   *                                                                                                           
  1809.59 :    * *    *               *               *    *                                                    *                                                
  1357.00 :                  ***    **             *       *                                                                                                     
  1017.61 :                       *     **        *                                            *        *        **                                              
   763.10 :        * * * **          *                                                                     *     **                                              
   572.24 :               *   * *   *                  *      **     *    *             *                                                                        
   429.12 :     *              *       *         *             *    *                                             *                                              
   321.80 :   *                         *              *                                                      *                                                  
   241.31 :                                *  *     *    **       * *               *                  *             *                                           
   180.96 :   *  *    *   *            * *       *                            *   *     **           *      * * *                                                
   135.70 :           **                                *             * *                *             *   *        *                                            
   101.76 :  * *                                        *     *  * *  *   *  *                       *       * *    *   *                                        
    76.31 :                  *                                                         *                 *         *                                             
    57.22 :  *                                 *   *                 *                             *   *                                                         
    42.91 :                                              *   *     *   ***       ** *     *               *            *    *                                    
    32.18 : *                                                * *         **  *       **     **           *         *  *    **   *     *                          
    24.13 :                                                                                                                                                      
    18.10 : *                   **        *                            *   ***   *              *****      *          *   *    *  ***                  *       * 
    13.57 :                                                                                                                                                      
    10.18 :     *                * *                        *   ***                *        *   ** **        **          **  *   *      *    * * **   *     *    
     7.63 :                                                                                                                                                      
     5.72 :                                                                                                                                                      
     4.29 :             *     *                       *           *   *         *    ***  *** **      *   * *          *   *          ** **     *        *  **** 
     3.22 :                                                                                                                                                      
     2.41 :                                                                                                                                                      
     1.81 :                                                                                                                                                      
     1.36 :    *                                     *      *   *          ** **** *    *  * **   *     *            *  **   ********* ****************** ** ** **
========= : ======================================================================================================================================================
          :  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
          :  0 0 0 1 1 1 2 2 2 3 3 3 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9 0 0 0 1 1 1 2 2 2 3 3 3 4 4 4 5 5 6 6 6 7 7 7 8 8 8 9 9 9 0 0 0 1 1 1 2 2 2 3 3 3 4 4 4 5 5
          :  1 5 8 2 5 8 2 5 9 2 6 9 3 6 0 3 6 0 3 7 0 4 7 1 4 7 1 4 8 1 5 8 2 5 8 2 5 9 2 6 9 3 6 9 3 6 0 3 7 0 4 7 0 4 7 1 4 8 1 5 8 1 5 8 2 5 9 2 6 9 2 6 9 3 6
          :  7 2 6 1 5 9 4 8 3 7 2 6 1 5 0 4 8 3 7 2 6 1 5 0 4 9 3 7 2 6 1 5 0 4 9 3 8 2 6 1 5 0 4 9 3 8 2 7 1 5 0 4 9 3 8 2 7 1 6 0 4 9 3 8 2 7 1 6 0 4 9 3 8 2 7

Basically, there's a broad peak near 500Hz, and a narrower peak around 1750Hz. And clearly there's a lot of noise in the data. (This is a snore, for those who are interested.)

A different person has a similar frequency distribution, only the peaks may be scaled up/down (based, presumably, on the resonant frequencies of the throat and mouth).

How would one distinguish that from a different sound that is more random in distribution? For instance, a choking sound looks about like this in one instance:

339889.00 :                                **                                                                                                                    
254880.80 :                                                                                                                                                      
191133.62 :                                                                                                                                                      
143330.00 :                                    *                                                                                                                 
107482.34 :                             *                                                                                                                        
80600.38 :                              *                                                                                                                       
60441.76 :                                                                          *                                                                           
45324.93 :                              *   *                                                                                                                   
33988.90 :                         *        *                                    *                                                                              
25488.08 :                          * *    *  *                                      *                                                                          
19113.36 :                                        *                               **                                                                            
14333.00 :                                 * * *  *                             *                                                                               
10748.23 :                                *                                          *                                                                          
  8060.04 :                               *   *        *                               **                                                                        
  6044.18 :                     ***               * *     *                  *       *                                                                           
  4532.49 :                          **              *       *                   *     *                                                                         
  3398.89 :                 * *  *       **              *                      *                                                                                
  2548.81 :                     *  *            *                      *    *  *        *                                                                        
  1911.34 :           *                                  *       *  *                       *                                                                    
  1433.30 :              *        * *                  *        *    *  *      *    *                                                                            
  1074.82 :            *       *        *                 *   * * *                     *                                                                        
   806.00 :             *  * **                                  *       ***                                                                                     
   604.42 :           *   *            *                     *    **   *      *           *                                                                      
   453.25 :                  *       *                     *       *      *                 *                                                  * ** *            
   339.89 :        *   *   *                     *  *   *             *      *                  * *   *                                         *  * *           
   254.88 :        *          *                       **       *      * ***                      *                                                *              
   191.13 :   *          *                           *         *    *                      **   **                                                     *         
   143.33 :                                                                               *            *             *                                           
   107.48 :    *    *                                                *     *                  *   *    *   *     *    *                    **      *   *     *   
    80.60 : *  *  *                                                                          *                         *                                *        
    60.44 :      *  *                                                                    *   *     *        *   **    *              *     *         **  *       
    45.32 :                                                                  *             *        ***    * *    *       **          *   *    *    *    *       
    33.99 :                                                                                                                                                      
    25.49 :      *    *   *                                    *              *                    *    * * * **            *        *    *      *               *
    19.11 : *   *           *    *               *          *                                  *     *       ***            *  ***  *    *  *             *      
    14.33 :                                                                                                                                                      
    10.75 :       *  *         *      *             *         *   *                *                *      *       *     *   * * *  *  **                  ** *  *
     8.06 :                                                                                                                                                      
     6.04 :                                                                                                                                                      
     4.53 : *** *  *    *          *              *   * *  **         *         *        *    ***  *   ****   * * ********** ** ***** ****  *****     *** *******
========= : ======================================================================================================================================================
          :  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2
          :  0 0 0 1 1 1 2 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 7 8 8 8 9 9 9 0 0 0 1 1 1 2 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 7 8 8 8 9 9 9 0 0 0 1 1 1 2 2 2 2 3
          :  1 4 7 0 4 7 0 3 6 9 2 5 9 2 5 8 1 4 7 0 4 7 0 3 6 9 2 5 9 2 5 8 1 4 7 0 4 7 0 3 6 9 2 5 9 2 5 8 1 4 7 0 4 7 0 3 6 9 2 5 9 2 5 8 1 4 7 0 4 7 0 3 6 9 2
          :  6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8 9 1 2 3 4 6 7 8

As you can see, it's a bit more disorganized (if that's possible), at least in some cases more "peaky", and, it appears, much less reproducible -- the next sound would be quite a bit different, while snores tend to look a lot alike (when standing 10 feet back, at least).

I can see attempting identification by breaking the sound up into bands and giving each band a score, then coming up with heuristics such as "band A and C should be relatively loud, while bands B and D should be relatively silent", but it would be nice if there was a more systematic approach.

Any ideas?

Until I get something better, here is a screen shot of the graph:

Here's a spectrum from Audacity of some similar data:

This is a different sound source, and I only approximated the time duration of the sample (which is 1024 samples at 8000 samples/second in the other images). The sample rate for this sound file is 44.1K, so it has much more high frequency data than is available in "real life". The first three peaks are at roughly 215Hz, 600Hz, and 2000Hz.

This next one is from a different sound file (& individual) where the sound level is much poorer. We especially have trouble with these. The peaks are about 124, 248, and 581.

Update

I added code to build a "filter" by taking snapshots of the frequency spectrum during peak snore (as determined by an existing heuristic), summing together several successive snapshots, "smoothing" a bit, and "normalizing" to an average unity gain. Did the same with snapshots taken during periods of relative quiet. I then multiply the signal by both filters and sum (over the frequency range) the difference of the two products to produce a "delta". This seems to produce a fairly sensitive indicator of snoring.

(Did this description make sense?)

To the extent that you can understand the above description does it make sense? Is a simple multiplication the most reasonable way to apply a filter of this sort?

Answer 1

This is a far wider problem than merely how to compute and plot spectra correctly; I am merely trying to answer the latter.

First things first: You've got a WAV of this stuff, right? Just download Audacity and use its spectrum/spectrogram plots. They're inadequate in several ways — in particular, it has fairly poor control over window length, it's really slow, its colormap facilities are kinda poor — but it's free, it gets a lot of things right, and it gives you results pretty quickly.

Your own spectrum analysis, like Audacity's, should be using Welch's method (with 50% overlap) for improved quality. The optimal window length and function are typically very situational, but the Hann/Hanning window is usually a pretty safe choice of window function.

Note that a power spectrum plot may not even be the best way to go here — an autocorrelation approach might be better. (Sorry, I'm not a speech guy.)

Answer 2

Speech analysis uses frequently technique called "formant analysis" that tries to extract the areas of interest out of an audio spectrogram (e.g. http://www.phon.ucl.ac.uk/resource/sfs/howto/formant.htm). This may be overkill for what you need. A simpler approach would be

• bandpass filter to frequency region of interest
• normalize spectrum to overall energy in the spectrogram
• define a threshold, everything above the threshold is a "peak"

This should give a list of peaks in terms of position and width which you can use as a "pattern".

Answer 3

It is customary to always plot audio spectra on a log-frequency scale unless you have a good reason not to do so. One thing that raises an alarm is that your corrupted speech seems to have an extremely low level, in fact it seems that it is of such low level that most of the speech is below the noise floor or what can be represented in Q15. That would also explain why your speech appear bandlimited.

Has the speech you record been processed in anyway?
Is the speech of such low level that your plot indicates?
What is the data type used to represent the samples (short, int, float)?

Answer 4

Your FFT is very poor quality, there is little point in trying to extract patterns using poor tools to gather the info.

You can check each point for it's nearest neighbour and do a dot to dot diagram.

You can use a gaussian filter to blur your spectrogram as they are not very well printed for viewing.

Your first graph has very little patterns to it at all. it is almost noise. you can perform tests to find how disparate the points are, the noisiness of it. for serious patterns you need serious detail.