5
$\begingroup$

I'm trying to replicate the "audio spectrum" effect from Adobe After Effects. An example can be seen in this video:

(screenshot of video)

Obviously, it has to be some variant of a fourier transform, but I've tried taking the fft of an audio sample in realtime and my implementation looks more like a big block of bars:

(big block of bars)

I imagine some kind of intermediate function is being applied to the fft, but I can't really figure out what it might be.

$\endgroup$
4
$\begingroup$

I've had a try at reproducing the effect, and I think these are some of the key elements:

  • You need a high-resolution FFT (large size; windows overlapping in time so that this doesn't come at the expense of frame rate) and to discard all but the lowest-frequency bins. You can tell this from the video because there are only rarely any harmonics visible and then only a second harmonic. The music you're hearing contains many more harmonics, so the display must be showing only the lower frequencies, not all the way up to $f_s/2$.

  • You need to dynamically adjust the vertical offset so that only the significant peaks are shown, hiding the broad-spectrum sounds whether they are continuous 'noise' or sharp drumbeats. What I did was take the median of the current frame's data and subtract it (as well as a manually set additional term), but there may be better strategies.

  • If still needed once you've discarded the high-frequency components, apply a frequency-varying offset so that the levels are even across the spectrum (rather than decreasing with frequency as your picture does). Whether this is actually needed depends on the frequency content of your input signal — thus the frequency response of the microphone it was recorded with, and so on.

  • The nice rounded rather than spiky peaks should be apparent once you've zoomed in on the low frequencies, but they are also dependent on your choice of window function. Use a window function with a large main lobe.

Pseudocode:

fft_length = 32768
display_length = 1024
freq_varying_offset = fft_length * 0.000005
for frame from 1 to ...:
    let start = frame / video_frame_rate * audio_sample_rate
    let samples = audio_data.subarray(start, start + fft_length)

    let buf = log_fft(window(samples), fft_length).subarray(0, display_length)
    let offset = fixed_offset - median(buf)
    for i from 0 to buf.length:
        buf[i] = buf[i] - offset + i * freq_varying_offset

    // now draw the bars on the video frame using the contents of buf
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.