# How to build a realtime simple audio spectogram? [closed]

I'm trying to build a spectogram view of history time of 4 seconds and I'm stuck at the part where you load/draw FFT vectors into the 2D bitmap. I just don't know how to do that, but i got the FFT in realtime.

I'm doing it in swift(iOS)

Thank you!

The Discrete Fourier Transform of a signal is a series of $$\mathbb{C}$$omplex numbers. When performed in short time segments and depicted in a spectrogram, what is really depicted is a spectrum.

That is, the strength of a particular frequency component, which is a $$\mathbb{R}$$eal number.

There are various different expressions for the strength of a component of the spectrum such as Amplitude, Power, Power Spectral Density and so on.

The most straightforward one is the Amplitude spectrum. To calculate the Amplitude of a given spectrum component, you need to get the absolute value of the complex number that represents a frequency component in your spectrum.

Programming languages such as C, Python, Julia, MATLAB / Octave and others, have native complex data types. In that case, the output from an fft() kind of function is an array of type complex and each element of that array contains a function (e.g. abs()) to obtain the absolute value of that complex number. If your programming language does not have a complex data type, then the output of your fft() is probably two arrays of real numbers. One for the real part of the complex number and one for the imaginary part. In that case, to obtain the spectrum, all that you have to do is spc[m] = sqrt(real_part[m]^2 + imaginary_part[m]^2) where ^ denotes exponentiation and m is the $$m^{th}$$ element of your DFT spectrum.

Once you do that, you will have an array of real numbers. These are going to vary wildly, because of the way the DFT sums work and for this reason you usually apply a logarithmic transformation. This transformation keeps large numbers large and makes small numbers large. You would typically apply this with something like spc[m] = log(1 + spc[m]).

At this point, you can depict spc[m] in grayscale by normalising it to 255 (usually) tones of gray with something like rendered_spc[m] = int(round(spc[m]*255.0)). int denotes an integer data type, you are likely to use something like a BYTE or whatever is more convenient for your bitmaps. The other thing you can do is to map the value of spc[m] within a palette of N colours and "paint" the pixels with that colour.

Finally, it would be probably useful to emphasise here that the spectrogram does not depict one "real time FFT" but many short time fourier transforms. So, if you want to depict 4 seconds worth of a histogram, you would have to accumulate a number of FFTs, get their amplitude spectra, transform them logarithmically, map them to colours and then depict them. Think of each FFT amplitude spectrum as one column of your bitmap.

This is a brief outline on rendering spectrograms practically. Depending on the desired accuracy, you might have to add some more complex features, such as "overlapping" DFT segments, interpolation (for smoother bitmaps with less data) and others.

Hope this helps.

• lets say i have 256x256 bitmap, how would I go about to map the accomulated FFTs? is it a nested for loop for x and y? Thank you – masaldana2 Dec 19 '18 at 19:33
• @masaldana2 The most straightforward way to think about this is to consider every one of the x columns of the image as a 256 point DFT. At a very elementary level, this would probably involve a "nested for loop" but in reality, these operations have to happen incredibly fast and for this reason the DFT is implemented in a special way (which leads to the fft). A very simple thing you could do is collect 256 FFTs of 256 points each and depict them in the image using the information provided in the answer above. – A_A Dec 20 '18 at 10:56
• oh okay, im very close... i have the spectogram being updated 60fps i just need to achieve the waterfall effect. – masaldana2 Dec 20 '18 at 19:38
• @masaldana2 You probably have the DFT updating at 60fps, not the spectrogram. The spectrogram will be the 256*256 matrix. Isn't there a spectrogram library for swift you could use? You would still need to learn about the spectrogram to use it effectively but you would not have to get into the details of its implementation of which there are many. Except if you simply want something like a VU-Animation, in which case, the theory doesn't matter a bit. – A_A Dec 21 '18 at 10:31