# How does Adobe After Effects generate its “audio spectrum” effect?

I'm trying to replicate the "audio spectrum" effect from Adobe After Effects. An example can be seen in this video:

Obviously, it has to be some variant of a fourier transform, but I've tried taking the fft of an audio sample in realtime and my implementation looks more like a big block of bars:

I imagine some kind of intermediate function is being applied to the fft, but I can't really figure out what it might be.

• Have you ever succeed on this? – arturdev Oct 25 '19 at 18:13

I've had a try at reproducing the effect, and I think these are some of the key elements:

• You need a high-resolution FFT (large size; windows overlapping in time so that this doesn't come at the expense of frame rate) and to discard all but the lowest-frequency bins. You can tell this from the video because there are only rarely any harmonics visible and then only a second harmonic. The music you're hearing contains many more harmonics, so the display must be showing only the lower frequencies, not all the way up to $f_s/2$.

• You need to dynamically adjust the vertical offset so that only the significant peaks are shown, hiding the broad-spectrum sounds whether they are continuous 'noise' or sharp drumbeats. What I did was take the median of the current frame's data and subtract it (as well as a manually set additional term), but there may be better strategies.

• If still needed once you've discarded the high-frequency components, apply a frequency-varying offset so that the levels are even across the spectrum (rather than decreasing with frequency as your picture does). Whether this is actually needed depends on the frequency content of your input signal — thus the frequency response of the microphone it was recorded with, and so on.

• The nice rounded rather than spiky peaks should be apparent once you've zoomed in on the low frequencies, but they are also dependent on your choice of window function. Use a window function with a large main lobe.

Pseudocode:

fft_length = 32768
display_length = 1024
freq_varying_offset = fft_length * 0.000005
for frame from 1 to ...:
let start = frame / video_frame_rate * audio_sample_rate
let samples = audio_data.subarray(start, start + fft_length)

let buf = log_fft(window(samples), fft_length).subarray(0, display_length)
let offset = fixed_offset - median(buf)
for i from 0 to buf.length:
buf[i] = buf[i] - offset + i * freq_varying_offset

// now draw the bars on the video frame using the contents of buf