Role of window length and overlap in uncertainty principle?

Question

I am trying to predict epilepsy using spectrograms and a convolutional neural network.

So far I have achieved a validation accuracy of 86% which i feel like is pretty good. Lots of the papers doing similar deep learning are using an very high frequency resolution in their spectrograms.

However I keep reading that when creating spectrogram one should be aware of the uncertainty principle. Is it correct that the frequency resolution is dependent on the length of the window? and how does the overlap then affects this. Assume I have a very large window but also a high overlap? Do I get high time and frequency resolution?

I have a sample plot showing two identical spectrograms but different window lengths. The sampling rate is 500 Hz and i have 2000 samples = 4 seconds of data:

#However we can create some manual configuration which increases the visibility of each frequency.

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10,5), sharey=True)


normalize_color= matplotlib.colors.Normalize(vmin=new_min + (new_min/4), vmax=new_max)
ax1.set_title("Increase in frequency resolution")
f, t, Sxx = signal.spectrogram(np.array(sz_win[0][0]), fs=500, nperseg=500, noverlap=int(500*0.99), nfft=1024, scaling='density', return_onesided=True)
pcm = ax1.pcolormesh(t, f, nanpow2db(Sxx), cmap='jet', norm=normalize_color)

ax2.set_title("Increase in time resolution")
f, t, Sxx = signal.spectrogram(np.array(sz_win[0][0]), fs=500, nperseg=int(500/7), noverlap=int((500/7)*0.99), nfft=1024, scaling='density', return_onesided=True)
pcm2 = ax2.pcolormesh(t, f, nanpow2db(Sxx), cmap='jet', norm=normalize_color)

fig.suptitle('Time vs Frequency domains')
fig.tight_layout()
plt.savefig("timevsfreg")
plt.show()

Answer 1

Overlap is and isn't related to time resolution: in sense of the uncertainty principle, only the window width plays a role.

However, any overlap other than maximum (hop_size = len(window) - 1) will alias the STFT and lose analysis information: this loss is for both time and frequency. Depending on hop_size, only a part of STFT is affected, however (see linked post).

Put simply, STFT takes similarity of a windowed sinusoid at each frequency and time point, and if we skip some time points ... we skip some time points: no information captured there. However, often this is fine, as neighboring points are sufficiently similar.

Measuring loss of information

I recommend applying Parseval's theorem and computing the percentage of energy lost by subsampling (hop_size):

Sx = np.abs(STFT(x))
# frequency-domain energy for each bin
# omit DC since it doesn't measure variation, only offset
E = np.sum(np.abs(rfft(Sx, axis=-1))**2, axis=0)[1:]
# fraction of energy lost by subsampling
hop_size = 4
loss = E[len(E)//hop_size:].sum() / E.sum()

If loss (which is between 0 and 1) is low < 0.1, not much is lost.

Example: loss of analysis info

The spectrogram hence wrongly suggests a pure sine (despite hop_len=64 being perfectly invertible), where we have an F.M., degrading feature extraction. Code at Github.

Answer 2

Is it correct that the frequency resolution is dependent on the length of the window?

yes.

and how does the overlap then affects this.

The higher the overlap the more information is "repeated" from frame to frame. Consecutive frames are not independent but become more and more the same as the overlap increases.

Assume I have a very large window but also a high overlap? Do I get high time and frequency resolution?

yes and no.

It's helpful to distinguish between data and information. You can certainly increase the amount of data that you get this way, but you don't create more information. A standard way of increasing the frequency resolution is to simply zero pad in the time domain. That creates a higher frequency resolution, not by adding information but simply by interpolation. If you start with $N$ points and zero pad to $2N$ and then do an FFT you end up with $2N$ frequency values, $N$ of which are independent and $N$ of which are linearly dependent.

That still can be very useful thing to do, you just have to be aware of what it is and what it isn't.

Let's look at a simple (if contrived) example. Let's say you choose a window length of 1000 and a hop size of 1 (i.e. overlap of 999). Technically you have a time resolution of 1 but let's see what happens if we analyze a unit step:

Before the unit step the spectrogram is all zeros and the final state is that the spectrogram being all ones. However, the transition will be quite slow. It will take a full 1000 frames to get to the final stage. The length of your time domain window "smears out" any time domain transient. Increasing the overlap does not change that smearing at all, it just increases the resolution of it's representation.

Answer 3

You are right, reducing increasing the overlap you increase the time resolution, and increasing the duration you increase the frequency resolution.

You can increase the resolution as much as you can, by increasing the overlap, what you can't do is to observe fast variations in frequency.

Think of each frequency bin as a moving average, sudden variations of the input will take a longer time to affect the output. As you increase the window length you are smoothing the spectrogram on the time axis, this is why, even with a small overlap you can't perceive quick variations of the spectrum.

This type of tradeoff appears in many situations, for instance, here. A very short answer will not cover the topic, a very extense answer will be difficult to grasp. I hope this length was appropriate for this question.