I started learning audio dsp and the book I am working through is loaded with formulas and most of the time expects the reader to know what the relation is between these functions and why they are being used in the first place.

I'd reading a section on spectral shape, which includes sub sections on rolloff, flux, centroid, spread etc.

On what data are the functions actually applied starting from the audio source?

Im trying to get a grip on the actual process behind extracting information from audio.

The book also mentions

"It is also common to use the power spectrum instead of the magnitude spectrum"

How does one change between spectrums and when is it done in the extraction process?

Are they all applied to data in the frequency bins after running a Fourier transform?


For spectral features like roll-off, flux, centroid, etc, the process is as follows:

Before delving into the extraction process, a pre-processing phase is sometimes necessary, this includes: * Converting stereo signals to mono by summing both channels. * Resampling the signal to a target sample rate (if a collection of signal of heterogenous sample rates are used).

Now, we can assume that we are dealing with a mono signal of known sample rate.

At each time $t = 0, \Delta, 2\Delta...$ at which the feature has to be calculated (for example, $\Delta = 0.04s$ if 25 values of the feature are to be extracted per second of audio):

  • Extract the block of signal in the range $[t, t+l]$. $l$ is the analysis window duration and must be greater or equal than $\Delta$. There is a compromise to be found between using short $\Delta$ and $l$ (good temporal resolution) and using a long $l$ (good frequency resolution).
  • Multiply this signal chunk by a windowing function such as the Hann window. This prevents spectral leakage which might cause unwanted results (for example, report abnormally high values of spectral spread for a purely sinusoidal signal).
  • Eventually, pad the resulting signal with zeros. The purpose is twofold: make sure the number of samples processed at the next step is a power of 2 (in case the FFT library you use is rustic and does not handle other sizes); and in some applications, artificially boost the frequency resolution through spectral interpolation.
  • Compute the FFT of the windowed, padded, block of signal.
  • Discard the upper half of the coefficients returned by the FFT function (they correspond to "negative frequencies" and are of no interest).
  • Compute either the magnitude or square magnitude of the remaining coefficients. This gives the spectrum on which spectral features are computed.
  • From there, use whatever formula describes the feature to be extracted. The formulas usually involve $A(k)$ (squared magnitude or magnitude in the $k^{th}$ FFT bin) and $f(k) = \frac{k . \text{sample rate}}{\text{FFT size}}$ (Center frequency of the $k^{th}$ FFT bin).

There is little consensus in the audio signal processing community as to whether the magnitude or power spectrum should be used for spectral features - I have routinely seen both used in research code.

  • $\begingroup$ Thank you for your informative answer. " There is a compromise to be found between using short Δ and l (good temporal resolution) and using a long l (good frequency resolution)." Is it common to run one of each? I guess it depends on the task. "Discard the upper half of the coefficients" - What causes the negative frequencies to exist in the signal and why are they the upper half? $\endgroup$ – system Aug 2 '13 at 6:43
  • $\begingroup$ I remember having seen a few papers in which some feature extraction process is run at two scales, but this was not for the extraction of spectral coefficients. $\endgroup$ – pichenettes Aug 2 '13 at 6:53
  • $\begingroup$ Regarding negative frequencies: dsp.stackexchange.com/questions/431/… $\endgroup$ – pichenettes Aug 2 '13 at 6:55
  • $\begingroup$ The 3d spiral image from that link helped clear it up thanks. And it is on the cover of the book I ordered yesterday called "Understanding Digital Signal Processing". So my understanding now is that a signal (if real) is a balanced spiral from an origin in 3d space and the DFT detects frequencies by running at different frequency rates along the spiral and looking for clashes on the points that have been converted from 2d space (amplitude/time(sampleNumber)) to 3d space? Is this correct? $\endgroup$ – system Aug 2 '13 at 7:37

Not the answer you're looking for? Browse other questions tagged or ask your own question.