# Increasing STFT resolution by repeating the window? Ways to improve STFT resolution?

In theory of FT(Fourier transform) and STFT(Short Time Fourier Transform) it is said that "A narrower window gives good time resolution but poor frequency resolution".

My question is, can we increase the frequency resolution without pooring the time resolution, by taking the samples in the narrow SFTF window, and repeating them X times until the desired size, and calculating the FT on a large array of samples? Or will it not work? If not, can something else be done with changing how we window to improve the resolution?

EDIT:

For example if the samples in the narrow window are:

[1, 2, 3, 4]

we will not do FT on it, but first we will repeat the window 2 or more times, so that it will look like this:

[1, 2, 3, 4, 1, 2, 3, 4]

And only then will we do FT.

• What do you mean by "taking the samples in the narrow STFT window, and repeating them X times"? Concatenating the window to itself, or inserting samples between existing samples (upsampling)? Jun 12, 2023 at 14:06
• Joint resolution can be increased with synchrosqueezing (but it's not absolute). Jun 12, 2023 at 14:14
• @OverLordGoldDragon I mean Concatenating the window to itself. Jun 12, 2023 at 17:20
• So repeat(window * x_segment), or repeat(window) * x_segment? The current answer is for former. Latter yields nonsense and shouldn't be done; the resolution is worsened in both domains. Either way, edit this information into your question rather than leaving it in comments. Jun 12, 2023 at 17:27
• I've extended the scope of your question a bit in a way I think you'll find useful and relevant to original. In general though I suggest such extensions carefully phrased as they risk becoming too broad or making it unlikely for people to answer. Answers going beyond the said requirement is of course allowed, but what matters is to not require it. (This comment isn't entirely directed at you.) Jun 12, 2023 at 19:16

Linear = Heisenberg-bound.

No amount of fancy tinkering of a linear method can ever break the bound, and STFT is linear.

OP suggests fft([window * segment, window * segment, ...]) to compute each column, as opposed to fft(window * segment). This simply inserts zeros into the existing result, between the bins. Information-wise, it's purely duplication, no change of capability is possible. It's also no longer STFT, nor a valid time-frequency representation.

If an operation can be expressed by its Fourier transform, it's linear. In this case, it's inserting zeros into FT. If $$\text{op}(x) + \text{op}(y) = \text{op}(x + y)$$ for all $$x, y$$, it's also linear.

A fundamental limitation of any method, is that time-frequency decomposition is non-unique. It's not that we can't "resolve", but that "resolve" isn't absolutely defined. Of course, we can just define it and there's no problem; it's a "limitation" in practical sense in that improvements in joint resolution must concern themselves in what they mean by "improved".

### Improving STFT's resolution

1. Only a different window or its length can change resolution.
2. Higher nfft and noverlap (i.e. lower hop_size) can, each, improve both time and frequency resolution.

These aren't contradictions, rather mean different things by "improve". The short version is, as a flawed analogy, lower nfft/noverlap is like taking fft but 5% of bins at random aren't returned. The coffee you ordered was good quality, it just never arrived.

It's understood by realizing that STFT is equivalently convolutions (rather, cross-correlations) - and not just equivalently but it's the more faithful interpretation as a time-frequency representation. hop_size is the subsampling factor along time, 1 / nfft along frequency, which are lossy, and the losses can be measured and compensated for - see STFT: why overlapping the window?.

1. The DPSS window has the highest joint resolution, above Gaussian's (by design for the discrete case). But often we want another tradeoff, more time or more frequency. I heard good things of Kaiser (@DanBoschen) but haven't used it - comparison. DPSS is tunable by more than window_size, and it's what I personally use, but when that doesn't cut it, Hanning/Hamming.
2. Synchrosqueezing / nonlinear modifications: discussed below.

### Alternatives / extensions

• Continuous Wavelet Transform, which is multi-resolution, but the joint resolution is approximately same and never breaks Heisenberg. Avoid scipy, PyWavelets. MATLAB is OK in magnitude, I'm unsure of complex-valued. In Python I recommend ssqueezepy (am author).
• Synchrosqueezed STFT (also CWT). This breaks the mathematical definition of Heisenberg's resolution for a wide variety of signals, ideally attaining perfect joint resolution. However, there's no improvement in the general case, and ability to resolve remains the same$$^1$$. Despite this, the "pseudo-gain" in resolution is very useful for many applications. See Synchrosqueezed Wavelet Transform explanation, which directly generalizes to SSQ_STFT.
• Adaptive SSQ: the resolution is varied also along time. I don't know whether it improves "ability to resolve". Paper.
• Many STFTs with different windows, nonlinearly combine. I've heard of such methods, I don't know what they are. If we must do many STFTs instead of something else entirely, odds are good that they also have "if's" and "but's" or are limited to certain kinds of signals. Note this is not CWT, nor is there any improvement by linear combinations of said STFTs.
• Super Resolution: injecting a "prior" (assumption), as neural networks do, can improve ability to resolve. I'm not familiar with these methods, but an example is provided in this answer. Perhaps it can be adapted to STFT.
• Deep learning: that deep nets can solve the cocktail problem suggests that their latent space goes above and beyond Heisenberg's bound also in ability to resolve. This is achieved with tremendous amounts of nonlinearities and dimensional projections. Recovering the data from said latent space in a usable format is another story. Once the components are separated and we get single component in time-frequency sense, SSQ can map it out perfectly. I'm not claiming DNNs achieve perfect unmixing, but perhaps "practically perfect".
• Wavelet scattering: it's an improvement in another important sense, but again ultimately Heisenberg-bound. I explain the method here, and the resolution improvement here under "Increased frequency resolution" (should say "joint")

1: I'm not certain about this, I've not studied the math. What I do know is, SSQ can't help with insufficient separation in time-frequency, and "insufficient" is controlled by the kernel (window/wavelet); if there's any improvement, from my experience, it's very limited.

It will not work.

If you repeat the buffer M times the nominal frequency resolution will indeed be M times higher but the newly created frequencies values will be all 0.

The standard way of increasing the frequency resolution is to zero pad. That doesn't create new information but it interpolates correctly between the independent values of the FT which can still be useful.

• This shouldn't be called "increasing frequency resolution". Only a longer or wider window does that. It's "increasing frequency granularity". Also "doesn't create new information" isn't necessarily true for greater nfft. Jun 12, 2023 at 14:13
• @OverLordGoldDragon good point, I use the term “frequency precision” to make this distinction. It equals $f_s/\texttt{nfft}$ with $\texttt{nfft}$ the size of the window after zero-padding, as opposed to $f_s/N$ which is the standard “frequency resolution”, where $N$ is the length of the window before zero-padding.
– Jdip
Jun 12, 2023 at 14:34
• @Jdip Resolution cannot be measured in samples alone, it's about the window, but $f_s / N$ looks correct as a proportionality relation, though it's inversely defined (greater $N$ is greater resolution). This is not a minor distinction. $f_s / \texttt{nfft}$ is also reciprocally formulated, and it makes sense and can be defined with samples alone. I've not seen these metrics before, good to know. Jun 12, 2023 at 14:53
• Anyway the first two paragraphs are correct, but "nominal res higher" is correct but tricky to interpret. Assuming that's what OP means by "repeating the window". Jun 12, 2023 at 15:18
• @Jdip A retraction, FP also isn't window-independent: if the window is unit impulse, then nfft=1 == nfft=9999. It at first struck me as good-enough for most windows, but they're both too different and it promotes false thinking to imply independence: we shouldn't say equals. Both $\texttt{nfft}/f_s$ and $N/f_s$ remain correct as proportionalities, and that's already very useful. Jun 13, 2023 at 17:23

"...can we increase the frequency resolution without pooring the time resolution,...?"

Yes and no.

With only one run of the STFT, the answer is no. There is this thing called the "Uncertainty principle" that spells out the tradeoff between resolution in time vs. frequency.

However, using the same original data, you can run the STFT several times, each time with an increasingly wider window, to get increasing frequency resolution at the expense of time resolution.

If you can somehow integrate the results of all of the different runs of the STFT, you could have a composite picture of this with higher resolution in both time and frequency. This is sorta how I view the continuous Wavelet Transform.

• "somehow integrate" that's a big "somehow", and no, the suggested connection to CWT is very erroneous and should be disclaimed. Heisenberg's resolution is inherent to linearity - said integration must be done non-linearly, and isn't at all trivial. Jun 12, 2023 at 18:10
• ayup .......... Jun 12, 2023 at 18:12