Creating a spectrogram

Question

I've been trying to work out the logic for this task, and plan to use the KissFFT source package to perform the fast fourier transform. Please let me know if this looks about right:

1. Allocate an FFT structure, ie. kiss_fft_alloc(N,0,NULL,NULL) Where N is the window size I'm using. The input buffer will be an array of N elements of type kiss_fft_scalar. The output buffer will be an array of N/2 + 1 elements of type kiss_fft_cpx.
2. Decode N (window size) number of PCM samples.
3. For each PCM sample, average each channel's amplitude (unsigned samples) and scale from 0 to 2 (divide by 65536.0), storing the result into the input buffer.
4. Perform windowing (ie. Hanning) on the input buffer.
5. Perform fast fourier transform on the input buffer, storing into the output buffer. Since I am using real values as input, I can use kiss_fftr().
6. For the N/2 output values, obtain the squared magnitude of the transformed data and convert the values to the dB scale with the following formula: 10 * log10 (re * re + im * im)
7. Plot the N/2 values from step 6.
8. Discard the first half of the input buffer, decoding the next (window size / 2) PCM samples and performing scaling and windowing to the data. This should effectively slide the input window and avoid having to redo math on processed PCM samples.
9. Loop to step 5, repeating these steps until all samples are processed.
10. Free the used memory from kiss_fft_alloc().

It was suggested that I subtract a value from the input window before I perform the FFT, so that the resulting DC value has a magnitude of zero. Should I be subtracting the mean or the average from the input data?

Also, what are the things I need to consider when I choose a window size? Besides that it has to be an even number as per KissFFT's instructions, is there a benefit to using a small window size, ie. will it provide for a better graph? I assume that a large window size reduces the number of FFTs that must be performed, is that the only benefit to using a large window size?

Lastly, when I get to the point that the data is ready to plot, how do I go about plotting it? When I worked on some waveform graph logic in the past, I just plotted 3 values for each pixel along the $x$-axis (min amplitude, max amplitude, RMS amplitude), but I don't know what I'm supposed to do with spectrogram data.

Thank you in advance for any and all guidance you can provide.

Answer 1

Looks pretty good to me. In step 3, though, you actually want to scale the signal from -1 to 1, otherwise you're adding DC. You mentioned subtracting the mean -- I would not recommend doing this for a spectrogram, since that's effectively filtering out DC, which the spectrogram ought to show if it's there.

Choosing a window size is all about tradeoffs. A larger window will give you sharper frequency resolution, but blurrier time resolution. A shorter window will give you the opposite: sharper time resolution but blurrier frequency resolution. The appropriate choice of window size will depend on the data you are trying to analyze. Typically it will be a power of 2 just because FFTs tend to like powers of 2. A decent rule of thumb is that your window should be at least roughly twice as long as the period of the lowest frequency you would like to be able to accurately resolve.

You might wonder if it is possible to better deal with this tradeoff, and there are techniques for that: they generally involve computing spectrograms with several different FFT sizes at once, and combining them. There's some good visual information on this webpage: http://www.izotope.com/tech/aes_adapt/

If your window size is too small, two very close frequencies might be indistinguishable from each other since they both end up in the same FFT bin. If your window size is too large, two close events in time, might be combined, or a sharp transient might turn into a gradual attack. Check out that webpage I posted for some ways to visualize this.

A larger window size doesn't necessarily reduce the number of FFTs. You have chosen to compute a spectrogram using a short-time Fourier transform where there is an overlap of half the FFT size. You could use a higher overlap factor if you wanted. Choosing a window size is much more a matter of making the time/frequency tradeoff than how many FFTs you have to compute. In designing a spectrogram (or any STFT), you can think of choosing your window size, and hop size, the distance between blocks, as independent parameters.

When you plot it, time is typically on the x-axis, frequency is on the y-axis (usually a log scale, Mel scale, etc, rather than a linear scale), and then the magnitudes are represented with color intensity, i.e. very dark colors correspond to small magnitudes and very bright colors correspond to large magnitudes.