I am having a problem understanding how the author gets the onset envelope of an audio song in this paper.
The entire section in question has been quoted below:
First the input sound is resampled to 8 kHz, then we calculate the short-term Fourier transform (STFT) magnitude (spectrogram) using 32 ms windows and 4 ms advance between frames. This is then converted to an approximate auditory representation by mapping to 40 Mel bands via a weighted summing of the spectrogram values [Ellis, 2005]. We use an auditory frequency scale in an effort to balance the perceptual importance of each frequency band. The Mel spectrogram is converted to dB, and the first-order difference along time is calculated in each band. Negative values are set to zero (half-wave rectification), then the remaining, positive differences are summed across all frequency bands. This signal is passed through a high-pass filter with a cutoff around 0.4 Hz to make it locally zero-mean, and smoothed by convolving with a Gaussian envelope about 20 ms wide. This gives a one- dimensional onset strength envelope as a function of time that responds to proportional increase in energy summed across approximately auditory frequency bands
My understanding of the procedure of the first part of the procedure is that our 44.1KHz audio signal is resampled to 8KHz and then we perform a overlapping widowed STFT operation on it so we get the Fourier transform coefficients.
But I really have no idea what is happening in the second part of the procedure:
I think we then alter those coefficients in such a way that the new Fourier transform coefficients are now balanced perceptually(ie the coefficients are now what a person's ear would perceive them to be).Then the so-called resultant "Mel spectrogram" is converted to dB, and the first-order difference along time is calculated in each band.Negative values are then set to zero (half-wave rectification),then the remaining, positive differences are summed across all frequency bands.
Why and how would we calculate the first order difference? What use is this? I have no extensive knowledge of the Mel scale and it's so-called "frequency bands" so this part is proving hard to understand.
This signal is passed through a high-pass filter with a cutoff around 0.4 Hz to make it locally zero-mean, and smoothed by convolving with a Gaussian envelope about 20 ms wide. This gives a one- dimensional onset strength envelope as a function of time that responds to proportional increase in energy summed across approximately auditory frequency bands
Why do we need the signal to be locally zero-mean?