2
$\begingroup$

I'm very new to signal processing and attempting to implement a spectrogram generator to learn more about DSP in general. So far, I have built up an understanding of how I might do that.

  1. Get input signal
  2. Window the data (e.g. split up the samples into 1024 sample long chunks)
  3. Apply a windowing function
  4. Zero-pad the data (optional)
  5. Compute the FFT of each window
  6. Get the magnitude for each frequency
  7. Plot a spectrogram by laying each window side-by-side

I've also read that to reduce spectral leakage and other artifacts, one should apply an overlap between each window. Most recommend an overlap of 50%. This means that the first half of each window's data is copied from the tail end of the previous window. If my understanding is correct, this results in about twice as many windows, but the length of the window remains the same (e.x. 1024 samples with a 512 sample overlap).

When plotting these windows side-by-side, won't it also double the length of the time axis, since each window now refers to an interval covered by a previous window?

For example, if I have a 2s long signal where each window is 1s long, assuming a 50% overlap, I'll have a windows on the interval [0-1], then [0.5-1.5], then [1-2]. Laying these windows out side-by-side would result in a 3s long time axis.

I'm sure that I've misunderstood the transformation of the windowed FFT information into a spectrogram. Can someone clarify my understanding?

$\endgroup$

3 Answers 3

2
$\begingroup$

You're understanding of the spectrogram is good! Some additional steps could be taking the magnitude squared $\lvert X_k \rvert^2$, and then converting to log scale: $10\cdot\text{log}_{10}(\lvert X_k \rvert^2)$. Note that's equal to $20\cdot\text{log}_{10}(\lvert X_k \rvert)$

Here what confuses you: once you "lay them out side-by-side", you should assign each window their center-time. With your example, this would yield: \begin{align} &\texttt{Window edges} &\quad \quad &\texttt{time-axis}\\ &[0 \quad1] &\quad \quad &0.5\texttt{s}\\ &[0.5 \quad1.5] &\quad \quad &1\texttt{s}\\ &[1 \quad2] &\quad \quad &1.5\texttt{s}\\ &\cdots \end{align}

Some implementations start with the first sample at the center (using a half window for the first and last frames of the input data). This would yield: \begin{align} &\texttt{Window edges} &\quad \quad &\texttt{time-axis}\\ &[0 \quad0.5] &\quad \quad &0\texttt{s}\\ &[0 \quad1] &\quad \quad &0.5\texttt{s}\\ &[0.5 \quad1.5] &\quad \quad &1\texttt{s}\\ &\cdots \end{align}

$\endgroup$
8
  • $\begingroup$ Thanks! This was what I was looking for. Could you clarify why computing the log scale is useful? Is it related to decibels? $\endgroup$
    – acd
    Feb 3, 2023 at 2:57
  • $\begingroup$ Good question. It’s useful depending on the data you’re visualizing. For example, if you’re visualizing audio data, that’s because humans perceive sound on a logarithmic scale. By taking the log you get closer to a loudness representation. $\endgroup$
    – Jdip
    Feb 3, 2023 at 3:17
  • $\begingroup$ I strongly disagree with this answer. "1st window 0s" is incorrect, and it's not that arbitrary. $\endgroup$ Feb 3, 2023 at 11:23
  • $\begingroup$ @OverLordGoldDragon My point was that the time axis should be labelled by the OP and that’s, by construct, arbitrary. However the correct way is the first mentioned. I’ll edit to reflect. $\endgroup$
    – Jdip
    Feb 3, 2023 at 15:42
  • $\begingroup$ What is the point of suggesting a wrong approach? We "could" do 10s, 20s, 30s. If there is a "wrong" then it's no longer arbitrary, especially since OP appears concerned with attributing meaning and physical interpretation. I can see a way in which 0s makes sense, but that shouldn't be recommended without explaining. $\endgroup$ Feb 3, 2023 at 15:56
2
$\begingroup$

It's not 3s because they're overlapped. [0-1] and [0.5-1.5] labels 0.5s to 1.5s, not 0s to 2s. Every output column maps a single time point - a centering of a windowed complex sinusoid at an input time instant, and rows are the different frequencies of said sinusoid. See this visual: it's that except windows are same width, frequencies are spaced linearly, and our "hop size" is greater than 1.

The correct labeling of individual slices is [0.5, 1, 1.5], and their slices' temporal extents span [0-1], [0.5, 1.5], and [1, 2]. Like so:

An alternate labeling, 0, 0.5, 1, wouldn't fit in this picture. The motivation of STFT is to jointly localize in time and frequency, and the time instant most pertinent to a given windowing is its center. If we double the overlap, now there's windows actually centered at 0 and 1. If we eliminate hopping altogether, such labeling would share no time instances with the input's. It's analogous to labeling the frequency axis by k + 0.5, where k is a DFT bin.

To get STFT whose labeling starts at 0s and ends at 2s, we need windows that are centered at those points, which requires padding.

Lastly, a suggestion - the "windowed Fourier" is but one perspective on STFT, and not necessarily the best. Fundamentally STFT is a time-frequency representation formed by bandpass filtering - convolution with complex windowed sinusoids, and "hop size" is just stride. Also the standard STFT implementation isn't best.

$\endgroup$
2
  • $\begingroup$ +1 for the important last paragraph $\endgroup$
    – Jdip
    Feb 3, 2023 at 18:30
  • 1
    $\begingroup$ @Jdip We're told "use this symmetry window for zero phase"... yet we're far from zero phase no matter the window, with that implementation. It looks so obviously botched up if we plot the actual convolving filters. $\endgroup$ Feb 4, 2023 at 8:51
1
$\begingroup$

The need for 50% overlap is really a synthesis or resynthesis issue. It has to do with complementary windows for perfect reconstruction with the STFT in processing. For analysis, you can use any window, it doesn't have to be 50% overlapping Hann windows. I liked to use Gaussian windows because of their mathematical similarity to the chirp function when I am trying to characterize each sinusoidal component.

$\endgroup$
4
  • $\begingroup$ What do you mean by "mathematical similarity to the chirp function ... [in characterizing] each sinusoidal component"? $\endgroup$ Feb 3, 2023 at 11:12
  • $\begingroup$ In this sense: $$ x(t) = e^{-\alpha t^2} e^{j \beta t^2} $$ Intraframe time-scaling of nonstationary sinusoids within the phase vocoder $\endgroup$ Feb 3, 2023 at 15:50
  • $\begingroup$ hmm ok though that seems packed with fairly specific meaning that's not evident from the statement. Without clarification it's suggestive of a chirp-like time-frequency basis, but that's subject of the oscillations under the window in addition to the window itself - e.g. chirplets. Your comment reference should help though. $\endgroup$ Feb 3, 2023 at 16:06
  • $\begingroup$ @OverLordGoldDragon , what is the difference between $-\alpha t^2$ and $j \beta t^2$? Same $t^2$. $\endgroup$ Feb 3, 2023 at 16:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.