9
$\begingroup$

I'm a chemical engineer, not an EE, so this is a bit difficult.

I'm trying to figure out how to take amplitude vs time data and transform it into frequency vs time. My first instinct is to slice my data into chunks, perform the FFT on each chunk, and then plot that. Unfortunately, as the time duration of each slice approaches zero, there is no longer enough information to get accurate frequency information (low frequencies require more than a very small time slice). So... how do I do this? I'm sure this is some sort of famous problem that someone's already solved.

Here is the kind of transform I'm looking for, illustrated with a sound wave (piano note G). As you can see, this graph as three axes, with the third one being represented by color.

Thanks!

enter image description here

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

The time vs frequency resolution is a well-known problem, and there are indeed approaches to overcome it. For audio signals, some of the commonly used techniques include: parametric methods ; adaptive resolution (analyze with various time/frequency configurations and patch the results together - Wen X. and M. Sandler, "Composite spectrogram using multiple fourier transforms") ; wavelets/decompositions on overcomplete bases ; and use of phase information to extract the precise location of frequency peaks (IFgram).

However, it appears that the graph you have shown does not use some of these techniques ; so I suspect this is not what you might be looking for. There appears to be some "smearing" on the horizontal axis (for example at t=1.2s) and this is a sure sign that the analysis has been done with a high overlap between chunks.

Indeed, the chunk duration and the number of analysis frames per second do not have to be linked to each other if you allow frames to overlap. So if you want to use 40ms long analysis frame, your grid does not have to be:

frame 1: t=0..t=40ms ; frame 2: t=40ms..t=80ms

It could very well be:

frame 1: t=0..t=40ms ; frame 2: t=10ms..t=50ms

This overlapping can give the illusion of a higher temporal resolution without reducing too much the FFT window size. Note that this can only help in accurately locating an event on the time axis - it will not help resolving two events close in time... Just like increasing the FFT size might help with identifying the location of a frequency peak, but not with the resolution of two adjacent frequency peaks.

Improve this answer
$\endgroup$
1
  • $\begingroup$ How would you go the other way (transform from spectrogram to audio signal) $\endgroup$
    – pete
    Commented Apr 28, 2015 at 17:56
5
$\begingroup$

Yes, many people have worked on time-frequency analysis.

The approach of "slice my data into chunks, perform the FFT on each chunk" is a good idea. Applying a "window function" on each chunk, just before performing the FFT, helps avoid many artifacts. Allowing chunks to overlap also helps. After those tweaks, you end up with the Gabor transform, which seems to be the most popular short-time Fourier transform (STFT).

As you've already pointed out, and as the Wikipedia article points out, all short-time Fourier transform techniques have a tradeoff:

  • when you cut the time-series into very short pieces, you get highly precise time information as to exactly when a tone starts and stops, but the frequency information is very blurry.
  • When you cut the time-series into very long pieces, you get highly precise frequency information as to the exact frequency of a tone, but the exact time it starts and stops is blurry.

This is a famous problem, but alas, not only has it not been solved, it's been proven that the uncertainty between the two is inevitable -- the Gabor limit, the Heisenberg–Gabor limit, the uncertainty principle, etc.

If I were you, I would start with one of the many off-the-shelf libraries to calculate the Gabor transform, and experiment with cutting the time series into various lengths. There's a pretty good chance you'll be lucky and you will end up with some length that gives adequate time localization and adequate frequency discrimination.

If that doesn't work for this application, then I would move on to other approaches to time–frequency representation and time-frequency analysis -- wavelet transforms, chirplet transforms, fractional Fourier transform (FRFT), etc.

EDIT: Some source code to generate spectrograms / waterfall plots from audio data:

Image to Spectrogram goes in the reverse direction from the above utilities.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Gabor transform is with Gaussian windows only. If you use another window, it's just an STFT. (And true Gaussian windows don't exist in digital because they taper off to infinity.) $\endgroup$
    – endolith
    Commented Jun 15, 2012 at 18:45
  • $\begingroup$ @endolith: You are right. The Gabor transform uses a "Gaussian window function" that is truncated to finite length -- it's pretty close, but not mathematically identical to, an ideal Gaussian. $\endgroup$
    – David Cary
    Commented Jun 17, 2012 at 4:53
  • $\begingroup$ I think the Gabor transform is a continuous transform, using an integral, so it can have a non-truncated Gaussian as the window? $\endgroup$
    – endolith
    Commented Jun 17, 2012 at 17:04
  • $\begingroup$ @endolith: Yes, in principle, a person could use a non-truncated Gaussian as a window. In practice, since practically all the energy of the Gaussian is within a few sigma of the center hump, using a truncated window practically always makes no perceptible different in the output chart. Since software that produces spectrogram waterfall plots repeatedly applies the Gaussian window and then does a FFT for every column of the chart, "not truncating" would make that software unbearably slow. $\endgroup$
    – David Cary
    Commented Jun 21, 2012 at 4:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.