# Basics of frequency spectrum for onset detection using FFT

I'm pretty new on signal processing and I've been reading a lot the last days in order to understand a little about it. But still there are things that I'm not sure if I got well. Sorry if I don't use the terms well.

I'm developing an application that detects onsets using a frequency spectrum, using KissFFT. Basically I apply FFT to 1024 samples, so I get 513 values (one for DC and the rest are the different values for each frequency range). I analyse the variations in each frequency range (each value of the resulting array from the FFT) in order to detect the onsets (comparing them to the values' mean from the past half second, and checking that the next values are lower). I want to know in which frequency the onset has been detected. So here are my questions:

1. As I'm analysing music, it doesn't make sense to consider the whole spectrum range, am I right? I mean, based on this, I should worry to check only until the 4k range.

2. So, in case the previous sentence is right, I shouldn't take all the 512 spectrum values, but only the first ones. So if my samples are at 44100Hz, and I have a spectrum with 512 values, each one should cover a range of (approximately) 86Hz. Is that correct? In that case, I should only analyse the first 47 array values and safely ignore the rest, I guess.

3. If I want to analyse a wider frequency, would be enough to sum the values from each frequency and calculate the mean? Is that correct?

4. What I'm doing is reasonable? Or I'm just making crazy random stuff?

Sorry for making such basic questions, as I said, I'm just entering in this and the more I read, the more confused I get. Thanks

• Why are you doing this in the frequency domain ? – Paul R May 15 '13 at 15:43
• Hi Paul, I'm not sure if I understand the question, I want to detect separate different onsets depending on their frequency (so I can differentiate somehow which instrument group was, separating beats from other sounds, for example). – Dr NotSoKind May 15 '13 at 16:47

As I'm analysing music, it doesn't make sense to consider the whole spectrum range, am I right? I mean, based on this, I should worry to check only until the 4k range.

This is not right. What the chart does not show is the approximate energy drop per octave of each instrument. A kick drum, for example, will have strong energy in the low range (say below 120Hz), but often very little within the highs (say above 10kHz).

If you are analysing music, and say there is a kick drum and a distorted guitar (which has dominent frequencies from the fundamental all the way up the spectrum), you are unlikely to be able to detect the kick drum, as the distorted guitar 4k energy will overpower any 4k in the kick. If you however analyse 80Hz - you should 'see' the kick no problem.

So, in case the previous sentence is right, I shouldn't take all the 512 spectrum values, but only the first ones. So if my samples are at 44100Hz, and I have a spectrum with 512 values, each one should cover a range of (approximately) 86Hz. Is that correct? In that case, I should only analyse the first 47 array values and safely ignore the rest, I guess.

As explained, the previous sentence is not right.

In addition, a sample rate of 44100Hz can contain frequencies up to 22050Hz (aka the Nyquist frequency = half the sample rate). I'm not sure what KissFFT gives you back, most FFTs will give you a mirrored result, of which you take the first half, and you work out what x-value corresponds to what frequency by calculating the frequency gap. I hope that you get how this is done from this code (alternatively look at the code here):

[ iSamples, iSamplingRate, iBitDepth ] = wavread( '100Hz16b.wav' );

iNyquist       = iSamplingRate / 2;
iSampleCount   = length( iSamples );
iAudioDuration = iSampleCount / iSamplingRate;

iFrequencyDelta = 1 / iAudioDuration;

iFFT = fft( iSamples );

# Get half of the set
iFFT = iFFT( 1:iSampleCount/2 + 1);

iFreqRange = 0: iFrequencyDelta : iNyquist;
plot( iFreqRange, 10*log10( abs(iFFT) ) );


If I want to analyse a wider frequency, would be enough to sum the values from each frequency and calculate the mean? Is that correct?

Generally speaking this is correct, but the wider the bandwidth of your analysis is, the harder it will be to detect differences. Say we take the extreme case where you analyse the whole frequency spectrum - you would not be able to distinguish one instrument from another. More on this below.

What I'm doing is reasonable? Or I'm just making crazy random stuff?

This is a momentous challenge (PhD stuff), but with a hard effort you might be able to demonstrate some degree of success (depending on the complexity of the material being analysed).

What you have to consider is that the timbre of all sounds can be broken down into two components:

• Their spectral content (ie, what frequencies exist and how much energy in each range).
• Their dynamic envelope (how their average level changes over time).

For the latter, do some research on 'beat detection' (but try to ignore anything with DJ in the title) and envelope followers (which is a very common strategy to detect the transient nature of sounds - work better than compering RMS to peak).

While I can't share an untested strategy, you may want to consider (starting with extremely simple material, like a kick drum and a violin going slowly crescendo):

• Perform FFT.
• Perform envelope follower on each of the bands you get.
• Take it from here (think patterns and deviations).

Here are some thoughts concerning your questions:

1. I guess you have to find the relevant frequency range by experiment, because, depending on the instrument, a lot of signal energy can be in the harmonics.

2. If your sampling frequency is 44.1 kHz and you use a 1024 point FFT your frequency interval is about 43 Hz, because the 513 points you mentioned are between DC and half the sampling frequency.

3. You could indeed average some frequency bins, or simply use a shorter FFT length.

4. I would say your methodology is pretty reasonable. Have you checked the literature? You're probably not the first one the do onset detection.

• Thanks Matt for the answer, you helped me a lot on this with your answer. I indeed checked some literature, for having an idea how could I detect the onsets (in fact, a week ago I never heard about the onset term, this is how new I am in this area). – Dr NotSoKind May 15 '13 at 16:50
• Instead of the document you linked to, I suggest you to directly look at spectrograms of actual recordings - you'll see that at the note onsets, there's a lot of high-frequency content (due to the sharp transient/attack - especially for piano, percussions...) - some of it well above 4kHz.
• There's no harm in processing all frequency bands.
• If you want to work with larger frequency bands, you need to sum the magnitudes (or squared magnitude) of the FFT values - not the raw complex values as these could cancel each other.
• What you're doing is called the "spectral energy flux" method, and is indeed a classic onset detection method. The "values' mean from the past half second" you want to use are a special and not that great case of low-pass filter - you can do better in this department! If you want to go this route, I suggest you to have a look at Alonso et al.'s paper - which introduces a bunch of perceptually motivated pre- and post- processing steps in the process to make it work. A simpler and quite efficient method is based on phase and amplitude comparison (there are discontinuity both in amplitude and phase at note onsets).

Regarding your reply to Paul, what you want to achieve (assign a source to an onset) is better done as a post-processing. What you have to keep in mind is that if there is, for example, a piano and a bass playing together, the onsets of the bass will look like vertical lines in the spectrogram, extending to the high frequencies.