Increasing STFT resolution by repeating the window? Ways to improve STFT resolution?

Question

In theory of FT(Fourier transform) and STFT(Short Time Fourier Transform) it is said that "A narrower window gives good time resolution but poor frequency resolution".

My question is, can we increase the frequency resolution without pooring the time resolution, by taking the samples in the narrow SFTF window, and repeating them X times until the desired size, and calculating the FT on a large array of samples? Or will it not work? If not, can something else be done with changing how we window to improve the resolution?

EDIT:

For example if the samples in the narrow window are:

[1, 2, 3, 4]

we will not do FT on it, but first we will repeat the window 2 or more times, so that it will look like this:

[1, 2, 3, 4, 1, 2, 3, 4]

And only then will we do FT.

Answer 1

Linear = Heisenberg-bound.

No amount of fancy tinkering of a linear method can ever break the bound, and STFT is linear.

OP suggests fft([window * segment, window * segment, ...]) to compute each column, as opposed to fft(window * segment). This simply inserts zeros into the existing result, between the bins. Information-wise, it's purely duplication, no change of capability is possible. It's also no longer STFT, nor a valid time-frequency representation.

If an operation can be expressed by its Fourier transform, it's linear. In this case, it's inserting zeros into FT. If $\text{op}(x) + \text{op}(y) = \text{op}(x + y)$ for all $x, y$, it's also linear.

A fundamental limitation of any method, is that time-frequency decomposition is non-unique. It's not that we can't "resolve", but that "resolve" isn't absolutely defined. Of course, we can just define it and there's no problem; it's a "limitation" in practical sense in that improvements in joint resolution must concern themselves in what they mean by "improved".

Improving STFT's resolution

1. Only a different window or its length can change resolution.
2. Higher nfft and noverlap (i.e. lower hop_size) can, each, improve both time and frequency resolution.

These aren't contradictions, rather mean different things by "improve". The short version is, as a flawed analogy, lower nfft/noverlap is like taking fft but 5% of bins at random aren't returned. The coffee you ordered was good quality, it just never arrived.

It's understood by realizing that STFT is equivalently convolutions (rather, cross-correlations) - and not just equivalently but it's the more faithful interpretation as a time-frequency representation. hop_size is the subsampling factor along time, 1 / nfft along frequency, which are lossy, and the losses can be measured and compensated for - see STFT: why overlapping the window?.

3. The DPSS window has the highest joint resolution, above Gaussian's (by design for the discrete case). But often we want another tradeoff, more time or more frequency. I heard good things of Kaiser (@DanBoschen) but haven't used it - comparison. DPSS is tunable by more than window_size, and it's what I personally use, but when that doesn't cut it, Hanning/Hamming.
4. Synchrosqueezing / nonlinear modifications: discussed below.

Alternatives / extensions

• Continuous Wavelet Transform, which is multi-resolution, but the joint resolution is approximately same and never breaks Heisenberg. Avoid scipy, PyWavelets. MATLAB is OK in magnitude, I'm unsure of complex-valued. In Python I recommend ssqueezepy (am author).
• Synchrosqueezed STFT (also CWT). This breaks the mathematical definition of Heisenberg's resolution for a wide variety of signals, ideally attaining perfect joint resolution. However, there's no improvement in the general case, and ability to resolve remains the same$^1$. Despite this, the "pseudo-gain" in resolution is very useful for many applications. See Synchrosqueezed Wavelet Transform explanation, which directly generalizes to SSQ_STFT.
• Adaptive SSQ: the resolution is varied also along time. I don't know whether it improves "ability to resolve". Paper.
• Many STFTs with different windows, nonlinearly combine. I've heard of such methods, I don't know what they are. If we must do many STFTs instead of something else entirely, odds are good that they also have "if's" and "but's" or are limited to certain kinds of signals. Note this is not CWT, nor is there any improvement by linear combinations of said STFTs.
• Super Resolution: injecting a "prior" (assumption), as neural networks do, can improve ability to resolve. I'm not familiar with these methods, but an example is provided in this answer. Perhaps it can be adapted to STFT.
• Deep learning: that deep nets can solve the cocktail problem suggests that their latent space goes above and beyond Heisenberg's bound also in ability to resolve. This is achieved with tremendous amounts of nonlinearities and dimensional projections. Recovering the data from said latent space in a usable format is another story. Once the components are separated and we get single component in time-frequency sense, SSQ can map it out perfectly. I'm not claiming DNNs achieve perfect unmixing, but perhaps "practically perfect".
• Wavelet scattering: it's an improvement in another important sense, but again ultimately Heisenberg-bound. I explain the method here, and the resolution improvement here under "Increased frequency resolution" (should say "joint")

---

1: I'm not certain about this, I've not studied the math. What I do know is, SSQ can't help with insufficient separation in time-frequency, and "insufficient" is controlled by the kernel (window/wavelet); if there's any improvement, from my experience, it's very limited.

Answer 2

"...can we increase the frequency resolution without pooring the time resolution,...?"

Yes and no.

With only one run of the STFT, the answer is no. There is this thing called the "Uncertainty principle" that spells out the tradeoff between resolution in time vs. frequency.

However, using the same original data, you can run the STFT several times, each time with an increasingly wider window, to get increasing frequency resolution at the expense of time resolution.

If you can somehow integrate the results of all of the different runs of the STFT, you could have a composite picture of this with higher resolution in both time and frequency. This is sorta how I view the continuous Wavelet Transform.

Answer 3

It will not work.

If you repeat the buffer M times the nominal frequency resolution will indeed be M times higher but the newly created frequencies values will be all 0.

The standard way of increasing the frequency resolution is to zero pad. That doesn't create new information but it interpolates correctly between the independent values of the FT which can still be useful.