# What is the proper way to implement a real time spectrogram?

I am trying to implement a real time spectrogram and I am not sure that I am implementing it the right way. First I decided to use Matlab, I was taking a short recording of lets say 0.1 seconds and then perform the FFT on that array of numbers which basically consists of the amplitude of lets say a 11025 sampling rate in that 0.1 seconds. I was then plotting the result of the FFT after computing the magnitude. After this process, I would then record another 0.1 seconds and so on. This was being done in an infinite loop in order to try to achieve a real time result. I know that this is not a spectrogram is just a different representation of this information. The problem with this is that I am not getting a very good results from this calculation. For example I was giving a certain tone using a frequency tone generator, and I was seeing a peak at just some instances which shows that somewhere the algorithm is failing. I do not have to use MATLAB, I could also use other programming languages such as Java so I prefer if I do not utilize a ready made function such as 'spectrogram' which is only present in MATLAB.

So can someone tell me on how a real time FFT of audio should be performed please? So how real time spectrum analyzer achieve that quality and they are real time so if they are taking a short sample of time it must be really short and still manage to get a good representation out of it.

• are you trying to create these "waterfall" images? with time and frequency and power (dB) on the three axes? not so sure how short the snippets of audio are in a commercial real-time analyzer, but you should know that the frameLength (which is also the window width) and the frameHop are two different numbers if there is anything better than 0% overlap. you can have a long frame (with a long window) and still advance that by short hops everytime the FFT is evaluated. Oct 1, 2018 at 23:31

Spectrograms are normally computed using the Short-Time Fourier Transform (STFT), and can be done in real-time. It consists of splitting the signal into overlapped frames, using a window function (typically scaled cosine like Hann) and then computing the FFT for the frame.

The overlap and windowing is key to avoid 'spectral leakage', and sounds like what is missing in your case.

If you want more frequency resolution, you can use longer FFTs.

In order to update with low latency (closer to "real time"), you can input short buffers often (every few milliseconds). Then combine that data with one or more previous buffers to FFT a longer data window as often as the short buffers arrive (if the CPU can keep up). Making these longer FFT buffers out of shorter incoming data buffers is called overlapping, since the longer buffers consist of partially overlapped data. Common overlaps are 50% and 75%, allowing an FFT 2X and 4X longer.

In addition, you can apply a window function as well as zero-pad the data before the FFT to remove rectangular windowing artifacts.

My understanding is that a spectrum analyzer is a graph of frequency vs magntiude and a spectrogram is a heatmap of frequency vs time. It might help to clarify which one you're after.

For audio, 1024 - 4096 is an adequate frame size to yield good results at 44.1kHz and extremly fast to compute. You will almost certainly need to window each frame.

For a spectrum analyzer, you'll have to decide if you want to skip samples for each animation frame (not to be consufsed with your FFT frame - lol) or make sure you compute multiple frames to cover all samples that have been output since the last animation frame and take, say, the max for each bin. This won't be an issue for a spectrogram as you'll decide how many lines of your heatmap you can draw since your last animation frame.

For a spectrum analyzer you'll need some ballistics so each bin can "fall".

Although you can use the STFT to calculate a spectrogram, you will have a time-frequency trade-off. The longer your frame, the better your frequency resolution at the expense of time resolution.

Alternative approaches:

• use the constant Q transform
• use a filterbank