Interpreting dataset of a power spectrum

Question

I am currently working with kaldi, and are trying to make spectrogram plots of different audio files. Kaldi is a speech recognition toolkit.

Kaldi frames its input in frames of 25 ms with a hop_length of 10 ms.

Kaldi have some code that makes spectograms of an audio files, but not much information is given on how things are stored... The sample rate used for the audio files is 16 KHz.

An example of dataset could be fash-b-an251:

which is an audio file of duration 1 sec. the matrix has 98 rows, and 257 columns.

Plotted: A different dataset could be fash-b-an253

which is an audio file of duration duration: 0.70s The matrix has 69 rows and 257 columns. Plotted: 257 must be the size of the fft length, since it is consistent. What is the rows - how it is related to the length of the audio file. And can the length of audio file be determined using number of rows(Probably), and if,then how?..

I am also a bit unsure about the unit of the values in the plots.. Is is a log power spectrum, or just a power spectrum?

Spectogram of the First audio file with matlab:

So the unit are not the same, what is it stored as??

Answer 1

1 second at 10ms "hops" gives 100 hops, which correspond to the rows.

In DSP parlance, the frame (the 25ms of samples) is called window size, and is the sample interval that is fed into the DFT (FFT). These samples are then multiplied by a window function prior to calculating the DFT.

25ms with $f_s = 16 \rm{kHz}$ gives 400 samples per frame. From your data, the DFT is performed with a vector size of 512, so probably the frame is then 0-padded from 400 to 512 samples, then fed to the DFT.

The DFT will then produce its output as a vector of frequency samples of size 512, where each frequency corresponds to multiples of $f_s/512 = 31.25 \rm{Hz}$. So your first column corresponds to frequency 0 Hz, then second column to 31.25 Hz, third to 62.5 Hz, etc.

Only half of the output is shown; since your signal is real, the other half of the DFT output (from 8000 to 16000 Hz) will be symmetrical to the first half. Also, since the sampling frequency is 16 kHz, then you cannot represent signals beyond 8 kHz.

Why 257 columns? Probably because the last column corresponds to 8 kHz (inclusive), so you have 256 frecuency intervals, therefore 257 frequency samples.

The plot looks like log of the magnitude of the DFT. The exact units you can deduce studying your audio signal. Try yourself with Octave / Matlab, using the spectrogram functionality there.

The DFT of 1 frame gives you 1 row of your spectrogram. Step forward 10ms, get another frame, repeat.

P.S. Sometimes spectrograms are easier to visualize with time on the horizontal axis.