2
$\begingroup$

I'm working on a real time beat detection algorithm. The first step, I believe, is to detect onsets (sudden volume increases). Then, in simplified terms, I can correlate these to find the tempo and use the strongest onsets to guess the beat position.

Say we are reading the audio in with 20ms chunks. My first guess was to just subtract the volume of one chunk from the previous one. The "volume" for a chunk would be a sum over the absolute value or squared signal, or maybe just the peak value (not sure which approach would be best). Then I can threshold these values to find the biggest volume changes.

To my surprise, according to papers like these (page 3) it seems that we often do onset detection by splitting into frequency ranges. For each audio chunk, they get the power for each of 7 frequency ranges, and compare it to the power of the previous chunk at that frequency (and the two neighboring frequency ranges), taking successive differences.

Then we have onset strengths, i.e. changes in power, for the 7 distinct ranges. And sum them back up across frequency to get the overall onset strength for that time.

Why is this done instead of just using the signal (no FFT required)? This seems to me to be just deconstruct and reconstruct the same information. The only thing I can think of is that the intermediate information of the power at different frequencies could be useful (isolating a kick drum, etc.)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Your question is really quite broad.

In a nutshell, if you think of the "successive differences", called spectral flux, understood per frequency range you can start to generalize to different kinds of music where maybe the kick drum is not present but the other instrumentation is sufficient to conclude that there is still on onset. One can also better measure the "strength" (loudness) of an onset, which is useful for algorithms processing onsets to conclude something at a higher rhythmic level (meter, phrasing, other groupings)

There's a lot of literature & code out there to try:

Improve this answer
$\endgroup$
4
  • $\begingroup$ this "spectral flux" measure is needed to detect note changes when there isn't a sudden change in volume that you might find with a note "attack". like if someone is humming a tune with a continuous hum, how do you detect a new note? $\endgroup$
    – robert bristow-johnson
    Nov 29, 2016 at 22:46
  • $\begingroup$ Hi RBJ! btw, you're my hero! I think the state of the art for soft onsets (i.e. humming, operatic song, string performances, etc) is to use local group delay information to scale/weight the magnitude spectrum differences appropriately. It is called SuperFlux! reference implementation here: github.com/CPJKU/SuperFlux $\endgroup$
    – ruoho ruotsi
    Nov 29, 2016 at 23:21
  • $\begingroup$ well, i don't speak Python, but i am looking at the two papers by Böck and Widmer. so i understand the abs value of the difference of spectrum of adjacent frames to get a difference or "novelty" measure. what does the group delay information do? change w.r.t. time of group delay is a dimensionless value. what does that value represent? $\endgroup$
    – robert bristow-johnson
    Nov 30, 2016 at 0:32
  • $\begingroup$ From what I can see section 2.2 of ISMIR 2013, instead of using the local group delay (LGD) information to relocate the magnitudes of the spectrogram, Bock et al. use LGD values close to zero indicate “stable tones” and regions with absolute values greater than zero indicate a possible onset. They determine the local minimum within each band of the filterbank for the SuperFlux calculation, and use these values as a weighting function. I prolly need to take the reference impl for a spin to build better intuition here. cheers! $\endgroup$
    – ruoho ruotsi
    Nov 30, 2016 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.