# Is there a way to convert audio to a list of frequencies preserving the temporal position?

I want to convert an audio file to a list of frequencies. We can apply an FFT on the audio to gain analysis of all the frequencies in a given audio sample, however, I want a list of frequencies as a whole from the audio such that if I play the frequencies in the time interval derived by the sampling rate, I would be able to have a passable reconstruction of the audio.

In simple words, assuming a 1 ms interval, I convert the audio file to a list of frequencies so that playing those frequencies 1 ms apart would reconstruct the audio.

Is there any python library using which we can accomplish given above? How exactly is it done in the industry?

• How is what you want different from what the FFT already does ? Commented Jul 11, 2021 at 17:38
• how do you do that then? I thought the FFT returns the frequencies and their count, which we can view from plotting? Like if I plot the fft of audio, I get a graph that goes up-to-down while I want a list of the individual frequencies in the audio.... Commented Jul 11, 2021 at 17:48
• @Hilmar If you know how to do it, could you post an example for clarity? Commented Jul 11, 2021 at 17:55
• you literally take the FFT of your signal "chunks". That's the example. That's the whole story. Commented Jul 11, 2021 at 19:39
• @MarcusMüller Ah, so I break it into discrete chunks and determine the most probable frequency (i.e the frequency component present the most times)? if so, would you happen to know any python lib that can accomplish that? Because I don't see how FFT-ing the chunks would give any meaningful information... Commented Jul 11, 2021 at 19:42

I think what you need here is the Short Time Fourier Transform (STFT). You set the desired window size as well as the amount of overlap (in your case I would set this to 0%) and then you perform FFT on the resulting window and calculate its magnitude. If you explicitly want a list, you can select the frequencies whose magnitude is above a certain threshold.

The process is easily invertible, especially when the window overlap is set to 0%. In any other case, you can invert the STFT by using the Overlap Add method.

Most commonly, the STFT is used to construct plots called spectrograms.

What you are looking for is the MP3 algorithm. It reduces audio to 15 millisecond blocks consisting of a list of frequencies and amplitudes. MP3 compression consists of "throwing away" frequency entries with low amplitudes. It also uses a psychoacoustic model to "throw away" frequencies you won't notice due to louder sounds before or at the same time.

The heart of MP3, though, is the discrete cosine transform (DCT.) The DCT is a relative of the fast Fourier transform (FFT.) MP3 compression runs the DCT on each block of audio. The data blocks of an MP3 file contain a list of frequencies and the corresponding amplitude as determined using the DCT.

You could implement something similar using the FFT, but as I understand it the calculation will be less efficient and the resulting data volume will be higher.

• would you happen to know any implementations already done in python? Commented Jul 11, 2021 at 21:55