Combing FFTs with multiple batch sizes

Question

When processing audio with DFT, a common strategy is to keep a buffer of many samples (maybe 8192) which is continuously updated, but compute the DFT of this buffer more quickly (say every 1024 samples). The idea being that you can get higher frequency resolution, but still get new data at a rapid rate. However, this has the obvious downside of effectively averaging the data out. Sharp transients will not result in correspondingly sharp spikes in the output of the DFT.

Is there some way to address this problem? I know that we have some inherent tradeoff between frequency resolution and time resolution, but I'm wondering if there is some way to combine the benefits of small and large buffers.

I had an idea when considering this problem: what if you compute the DFT of every 1024 samples, and also compute the DFT of the large buffer (8192 samples). Then, perhaps you can look at the differences between these two DFTs, and use the transient information from the less accurate one to influence the more accurate one.

Is this concept familiar to anyone? I can try to implement it myself, but it would be nice if such a thing has already been implemented, or proven to not work.

Answer 1

There is the fundamental uncertainty principle of the Fourier Transform and there is no way around it: higher resolution in time causes lower resolution in frequency and vice versa.

There are many methods to mitigate the effects (hop size, window shape, FFT length, wavelets, etc) but there isn’t a one size fits all method. These all involve complicated trade offs which are typically optimized to the requirements of your specific application.

Your method of combining high and low resolution spectra can certainly be useful (and has been done before) but whether it’s useful for your specific problem depends on your problem

Answer 2

As you have different windows, you can analyze simultaneously both spectrum, the fast low res spectrum with $n$ samples, $n$ frequency points, allowing you to take mean $\bar\mu_f=\frac1m\sum_{i=1}^m f_i$ and deviation $\bar \sigma_f=\frac1{m-1}\sum_{i=1}^m f_i^2$ for each frequency point, and the slow high res with $mn$ samples, $mn$ frequency points.

You can overlap all these information in a single plot, including the fast spectrum, the mean and deviation spectrum bands, and the slow spectrum, for your evaluation in your specific application to analyze with display is the better.

But be aware most people out there do not wish to see everything, just the precise information they need.