What is the physical significance of the Fourier coefficients in the DFT of audio signals and how can they be best displayed in a spectrogram?

Question

I am trying to display the spectrogram of an input audio signal (a newcomer to all things spectral here). The amplitudes of my audio signal can be assumed to be normalized between -1 and +1.

Suppose for the frequency $f$ in the DFT's output, the corresponding coefficient is $c$, in general a complex number. I would like to display the "amplitude" information contained in $c$ as a corresponding colored region on the spectrogram. Ideally to plot the spectrum, I would need a number that ranges from 0 to 1. 0 will represent the lowest amplitude I will record, and 1 the highest. There are a few ways I can think of to achieve this:

1. Since the $|c| \le N$, where $N$ is the number of samples, I can normalize the magnitude of $c$ with respect to $N$ ($v = |c|/N$) and display the resulting $v$ as a spectral colour.
2. Same as 1, but normalize $|c|$ wrt to $M$, the maximum magnitude of all Fourier coefficients. But this has the disadvantage of having to scan the entire data before the spectrum can be displayed, and will be a problem for large files. Also, this means that the spectrum display is relative to the intensity levels present in the current audio signal, and not valid in an "objective" sense.
3. Convert $|c|$ into decibels, say $d = 20 \log_{10}(|c|/b)$, where $b$ is a base magnitude. I will then normalize $d$ from say $l$ (where $l$ is a lowest dB value I will register) to $u = 20 \log_{10}(N/b)$. I can then use $d$ in displaying the spectrum. But in this case, what should $b$ be?

My question: what is the exact physical significance of the magnitudes of the Fourier coefficients in the DFT of audio signals and how can it be best displayed? I have tried all the above approaches, but am wondering which of them is the most proper or if I am missing any others.

Answer 1

This is a two year old question, but I think I add a little bit for those who might find it doing searches.

When you are talking about a spectrogram, there are really two steps involved that have to be considered. First is converting from the signal to DFT bin values, and the second is displaying those bin values in a pixel display.

Ideally, a spectogram of a pure tone of increasing frequency should make a solid, non varying, upward sloping curve. Inherently DFTs don't work this way. When you are at a whole integer frequency (in terms of cycles per frame), there is a single bin value, the rest are zero. When you are halfway between integer frequencies, the bin values are spread out which is commonly known as leakage. My recommendation is to use a VonHann window on your signal before you do the DFT. Depending on your scaling, this is the equivalent of subtracting the average of the two neighbor bin values from each bin value. This has the effect of "tightening up" the leakage, and "spreading out" the sharp peaks. This makes the display look a lot more consistent independent of the value of the frequency compared to the frame length, which is what you want.

The phase values in the DFT are framing dependent, and therefore don't help much with the display. Instead you should concentrate on making an intensity display based on the magnitude of the bins. You are not limited to a linear mapping, or a log mapping. You are going to want to use a lookup table anyway to speed up processing, so you can pretty much define an monotonically increasing function that works the way you want. Use can use a monochromatic color scale, or changing color one. The standard green for low, yellow for mid level, and red for intense is easily comprehended.

The second choice you need to make is the vertical axis scale. The two logical choices are linear, which the DFT is inherently. or logarithmic corresponding to musical scales. In either case, you are going to need to map a bin, or a set of bins, to a pixel, or a set of pixels, depending on the magnification at that part of the scale. Again, in practice, you will build lookup tables for this mapping.

The most proper choices in the above considerations are the ones that give you the visual results you desire. You are not going to be making calculations based on the display values so there is not problem with "distorting" the values (i.e. non-linear mappings) to give better visual results.

There actually is a physical significance of the values of the DFT coefficients. Discovering this was a major aha moment in my understanding of the DFT. A DFT bin value is actually a weighted average of a set of Roots of Unity, where the signal values are the weights and they are "wrapped" around the unit circle. The bin index is the step size in roots, and thus also then number of times the function is stretched and wrapped around the circle. For instance, suppose you have a signal with five bumps in it. When is stretched five times and wrapped around the circle five times, all the bumps align and throw the center of mass in that direction. On the other hand, when it is strectched by a factor of four, and wrapped around four times, the bumps are evenly spread and cancel each other out. This is how a DFT "plucks" out the frequencies that are in a signal.

I have written a blog article "DFT Graphical Interpretation: Centroids of Weighted Roots of Unity" to explain this.

This is long enough, hope it helps some one.

Ced

Answer 2

Your question about the meaning of the coefficients is that they indicate the amplitude and phase at each frequency. Have a look at this document.

For examples of spectrograms, have a look at this one.

Both documents are for the study of audio signals.

Answer 3

The exact physical significance of the magnitudes of the Fourier coefficients in the DFT of audio signals is that assuming you are locked in a "Ground Hog Day"-like time loop (well, not really since you cannot change anything at all) of the length of your sampling period and your magnitude is only non-zero at the times you sample it, the magnitude of the DFT coefficients will tell you how much energy the various frequency modes in the time loop have.

Expressed more succinctly: there is no physical significance as such. FFT algorithms are efficient algorithms and the actual information you are usually angling for (time-localized frequency groups changing according to some hidden state model) is washed out closer to the surface when applying DFT on some preprocessed sampled data and post-processing appropriately.

Physical significance only grows when you apply a DFT to synchronized samplings of stationary periodic processes (like rotating electric machines). For audio processing, this is basically never the case because even if you are analyzing stationary sound (like organ pipes), phase- or frequency-locking your sampling interval to an exact multiple of the sound frequency is not usually done: instead you work with constant sample rates independent of the signal.