2
$\begingroup$

Spectrograms convert an amplitude-time signal into a frequency-time signal using Short Time Fourier Transform (STFT). The window size L used in STFT is a design choice. As L increases, you get higher frequency resolution and lower time resolution and vice versa.

If you performed STFT with a large L to get spectrogram A with high frequency resolution then performed STFT with small L to get spectrogram B with high temporal resolution, can you combine spectrogram A and B somehow to get high temporal and frequency resolution?

My gut feeling is no, as changing L is simply "changing how you interpret the amplitude-time data."

$\endgroup$
  • $\begingroup$ You could even take it to the next level and analyze the data for chirps so that you have high resolution on rising and falling frequencies. Those are smudged by STFT of any window size. $\endgroup$ – Olli Niemitalo Oct 31 '17 at 9:05
2
$\begingroup$

Improving the time/frequency resolution balance remains a standard research issue, trying to adapt the STFT, inventing novel/better time-frequency distributions (Wigner, Cohen), using together or combining the above. For instance in music or speech source separation, a couple of STFTs is often used with both short and long windows, to be able to separate transient from larger signals.

The ways you can combine such SFTF depend a lot on the exact processing or features you want to extract, as detailed by @hotpaw2. Here are a few pointers to references suggesting methods. The first one suggests a point-wise combination of two spectrograms as $$S_c(f,t) = (|S_1(f,t)||S_2(f,t)|)^{1/2},$$ and the second generalizes it to the geometric mean, the minimax or the reciprocal average. The third encompasses it in a more generic merging approach.

$\endgroup$
1
$\begingroup$

Yes. Of course you can combine them. The question is how to choose which way to combine them, and whether the combined result provides any useful information gain (highlighting certain types of spectra), or value (e.g. looks "nicer", more colorful, etc.)

One method might be to upsample or resample each STFT set, one in the time dimension and one in the frequency dimension, until you get a matching set of 2D matrix sizes and grid points. Then combine time-frequency grid points using some algorithm chosen depending on your purpose. For instance you can give each matrix a different color look-up table, and then mix the two colors (with a 2D LUT, or do 3 STFTs and mix RGB). Or pick the max or min or average or use some non-linear 2D filter kernel, such as a threshold gate, or median filter, or highest contrast decision mux, or best looking color scheme chooser, and etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.