I want to do frequency analysis of audio data, basically trying to figure out what the notes are in a song algorithmically. The standard approach is to decode the MP3 into PCM data and run it through an FFT. However, notes below around middle C require too much precision for an FFT to work well. Since the MP3 already has frequency information in small time increments, is it a) precise enough to identify frequencies within about 2 - 5 Hz, and b) is there any code out that there does this conveniently? Most code (unsurprisingly) seems to convert it to the time domain, but I'd like to have just the frequency data.

  • $\begingroup$ Hi! There's a lot to unpack here: 1. figuring out what notes are in a song takes a lot more than just a frequency analysis – as far as I know, this is a problem so complex that there's not an even remotely acceptable machine-based method; start with identifying individually played notes of a single instrument under perfect clean recording methods. Identifying the note from that is hard enough. $\endgroup$ May 19 '19 at 13:03
  • $\begingroup$ Then, as you notice, methods where the spectrum is sampled at equal distances don't work well. Also note that your 2 Hz resolution requirement is also unrelated to your tone identification problem in general, but only applies to the lower frequencies. Kudos if you can tune a high violin string to 2 Hz accuracy! $\endgroup$ May 19 '19 at 13:05
  • $\begingroup$ Don't let the difficulty of my task distract from the question, though, which is how precise are the frequencies in an MP3 file, and is there an easy way of extracting it. This thing doesn't need to spit out sheet music, just having the notes in the chord is very helpful. There isn't really a problem identifying the actual note: just check if the amplitude of the frequency bin for that contains the note's frequency. This only works if the bins contain one and only one note, which is my problem for low-frequency notes. High frequency notes can tolerate wide bins, so no problems. $\endgroup$
    – prewett
    May 20 '19 at 20:00
  • $\begingroup$ OK, I'll try, but seriously, finding the notes in a chord is hard. A string plucked to produce a single note doesn't emit a single frequency, but a whole set of harmonics – that's why your approach of "finding the frequency bin with the tone" doesn't work. $\endgroup$ May 20 '19 at 20:25

The MP3 encoder works on batches of 576 time-domain samples and converts them to 576 frequency-domain samples. That means that you get a frequency resolution of $\frac{f_\text{sample}}{576}$, whatever your sampling rate is.

Notice however how short that is at usual sampling rates, for example 44.1 kHz: that's a bare 13.something milliseconds. That makes a lot of sense – the frequencies produced by an excited string, membrane or air volume change over time after the excitation.

So, no, your approach won't work: this resolution won't work to resolve your 2 Hz!

Also, not going through the MP3 route and directly FFT'ing your audio signal can't work, either:

To get a 2 Hz resolution in a linear frequency transform, you'll need 1/2 seconds of audio, no matter how you do it!

Do the math: if you have a sampling rate of 44.1 kHz, and you want to resolve 2 Hz, you'll need to transform 44.1 kHz / 2 Hz = 22050 samples. These take 22050 / 44.1 kHz = 1/2 s to accumulate at that sample rate. It's the same for every other rate, too, because the sampling rate in the equation

$$T_{\text{recording}}=\frac{\frac{f_{\text{sample}} }{f_{\text{resolution}}}}{f_{\text{sample}}}$$

always cancels.

Now, you'll notice that a note lasting half a second in its pure form rarely happens.

Furthermore, as I tried to explain in my comments:

Your hypothesis that playing a note inserts power at but a single frequency is false.

Instruments playing a note don't produce a single tone, but a mixture of multiple harmonics dampened by a changing envelope, also exposed to a drift in frequency over time. It's really not like you can go into a FFT and say, "OK, this was this and that chord played, easy to see".

Things get even more complicated since you said you wanted to detect notes played in a chord: Since the individual oscillating elements (e.g. guitar strings) mechanically couple, you typically get further frequency components and a changed temporal behaviour.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.