What's a good physical description of the following matlab code?

Question

I've inherited some matlab code and I can't really understand what the resultant signal really means. The input is audio and the code first performs an STFT/spectrogram (sliding window FFT of size 1024 with 50% overlap, hanning window applied). The output is then filtered to the range (using Matlab notation) to the range (2:N/2). In other words, the first bin is dropped and the negative half frequencies too. Just to be clear, for each 1024 window of audio data (audio_chunk below) the code does:

y = abs(fft(hann_win.*audio_chunk));
filt_y = y(2:length(y)/2);

The value filt_y is then summed and that becomes the output signal:

output = sum(filt_y);

So, for every window's FFT we get a single output value. The output sampling frequency is audio_sampling_frequency/512 because we slide the 1024 sample window along by 512 samples at each step.

My question is: what would you call that output signal? What sort of physical significance does it have, or what use is it for analysis?

Answer 1

The input is audio and the code first performs an STFT/spectrogram (sliding window FFT of size 1024 with 50% overlap, hanning window applied). The output is then filtered to the range (using Matlab notation) to the range (2:N/2). In other words, the first bin is dropped and the negative half frequencies too.

. . .

The value filt_y is then summed and that becomes the output signal:

What you are describing thus far is equivalent to a boxcar bandpass filter followed by integration similar to the way a moving average filter works.

So, for every window's FFT we get a single output value. The output frequency is audio_frequency/512 because we slide the 1024 sample window along by 512 samples at each step.

It would not be straightforward to define "output frequency" here. Most of the times, when you refer to "frequency" you might implicitly be referring to sinusoids. However, here, there is no sinusoidal output. But, because of properties of the DFT, your output will indeed fluctuate at a frequency that depends on the length of the FFT window.

This is because, the FFT assumes that the signal at its input is periodic. In other words, it assumes that whatever lands in its input frame is played again and again in a loop to infinity on either sides (both positive and negative). But this is not what is happening in reality. Because of this, your "running sums" from frame to frame have discontinuities. If you were to listen to that signal, IT!WOU!'D.CO!NTAI!NDIS!CON!TIN!UIT!IES!.

There are however two elements in your code that reduce the effect of those discontinuities. The first, is the use of the hann window and the second is the fact that the summation drops the DC component (and low frequencies depending on NFFT). With the use of the window, the discontinuities between the end of the frame and its beginning are reduced (remember, that is the start and end of the "loop"). By dropping the DC, you drop the result of partial summation that could end up having very high values.

Imagine for example that you are summing a zero mean sinusoid with a "frame" whose length just so happens to be half its period. Then in that case, during the positive phase of the sinusoid you would get a high positive value and then during the negative phase you would get a negative value. If the length of the "frame" was one period of the sinusoid, you would get a zero mean.

If you left the DC in and tried to "listen" to the signal you would be getting a soft "flutter" sound in the background, like holding a microphone close to a fan because of these discontinuities that are produced at each frame.

My question is: what would you call that output signal?

Some sort of band-limited integration. As mentioned previously, this is an integrator with a crude band-pass filter at its input.

What sort of physical significance does it have, or what use is it for analysis?

It returns the total amplitude of the frame of the signal minus its average value. It would be related to the average amplitude in a "moving average filter" sense.

It is difficult to say what its use would be for analysis without knowing what else is going on in the processing pipeline, but something like this could be used to get a sense of how "loud" is the current frame.

If this was a compressor for example, this signal could be used to trigger it at a sharp transition from soft sound to louder sound. In fact, because of the crude band-pass filter it contains at its input, this would then become a de-esser.

Hope this helps.

Answer 2

This quantity is proportional to the Manhattan or taxicab norm (aka the $\ell_1$ norm) of the windowed and zero-averaged magnitude spectrum.

Avoiding the y(1) removes the mean from the windowed audio chunk (plus some low frequencies). It is thus a sort of sliding activity or onset detector, from one frame to another. A related concept is the spectral flux:

a norm of the difference between two consecutive frames of the magnitude spectrum

I do not know of a better word for the integral of a Fourier spectrum, or the sum of harmonics