1
$\begingroup$

In common literature, when generating spectrograms, mel-spectrograms, and cochleagrams, the log of the resulting filterbank energies is taken. Why is this done? I notice that my convolutional neural network does not fit very well when I don't take the log. Is this done every time a filterbank or transform is applied? I.e. in all spectrograms, cochleagrams, mel-spectrograms?

Also, if I am windowing audio, should the log be taken before or after the resulting filterbank or transform energy values are summed up?

$\endgroup$

1 Answer 1

2
$\begingroup$

Why is this done?

Mostly because human auditory perception works this way. The relationship between energy and perceived loudness is logarithmic. That is not only true for hearing but for most other senses as well (https://en.wikipedia.org/wiki/Weber%E2%80%93Fechner_law)

Also, if I am windowing audio, should the log be taken before or after the resulting filterbank or transform energy values are summed up?

The correct order is

  1. Collect frame
  2. Apply window
  3. Apply filterbank or FFT (including zero padding if applicable)
  4. Convert to energy
  5. Integrate or Average to desired resolution.
  6. Convert to dB using a suitable reference
$\endgroup$
2
  • $\begingroup$ Thank you for your response. So when we take the log of the filterbank energies, we end up with dB? Should I be taking 20*log(energy) or just the log? Also, are cochleagrams, spectrograms, and mel spectrograms all in dB in the end or just certain representations? $\endgroup$
    – Lyle
    Commented Nov 27, 2022 at 12:45
  • $\begingroup$ Also, if you could explain why we separate frames and window before the filterbank is applied, that'd be awesome. For cochleagrams, in common literatures I see the window applied after the signal is filtered with the gammatone filter. The energies in the frame are then summed, and then the log is taken at the end. How does this compare with your order, and what are the consequences of doing it in this different order? Paper: sciencedirect.com/science/article/abs/pii/S0003682X18308144 $\endgroup$
    – Lyle
    Commented Nov 27, 2022 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.