Hot answers tagged

21

Just to make things clear - this property is not fundamental but important. It is the fundamental difference when it comes to using DCT instead of DFT for spectrum calculation. Why do we do Cepstral Mean Normalisation In speaker recognition we want to remove any channel effects (impulse response of vocal tract, audio path, room, etc.). Providing that ...


18

Let me start from the beginning. The standard way of calculating cepstrum is following: $$C(x(t))=\mathcal{F}^{-1}[\log(\mathcal{F}[x(t)])] $$ In the case of the MFCC coefficients case is a bit different, but still similar. After pre-emphasis and windowing, you calculate the DFT of your signal and apply the filter bank of the overlapping triangular ...


16

By long shot it is doable - to what extend? You will see. This task of environmental sound classification is not very well studied. Also choice of machine learning paradigm is crucial - statistical approach or maybe binary classifier? You can start with GMM's, ANN's and SVM's - I opt for GMM's and ANN's. Yes, most of people are using MFCC's because they are ...


11

The way MFCC's are always used is by feeding them into the classifier. This can be done on a frame-by-frame basis (12x1 vector), or by concatenating (12xN) - same as a spectrogram. Thus for DTW, you must perform the classification by calculating the distance between 12D vectors. It's a Dynamic Time Warping, so the difference must be calculated between ...


6

You could collect a large collection of voice signals, extract the MFCCs, and compute the mean and standard deviation of each coefficient. Keep these numbers and use them once and for all. Then, you can normalize the sequence of vectors from an utterance by subtracting the mean and dividing by the standard deviation you have pre-computed (do not recompute ...


6

First of all there is some serious "cheating" in the MFCC reconstruction experiment you linked to: not only the MFCCs are used, but also the voiced/unvoiced bit and the pitch. MFCC are not speaker independent. In fact, they are used for speaker identification/verification tasks! The speaker "idiosyncracies" are both in their prosody (preserved by this ...


6

Mel filter bank is important due to following reasons: It applies the Mel-frequency scaling, which is perceptual scale that helps to simulate the way human ear works. It corresponds to better resolution at low frequencies and less at high. Using the triangular filter-bank helps to capture the energy at each critical band and gives a rough approximation of ...


5

Ideally you would only see that frequency as a narrow peak but because of the finite-length window you also get that other artificial crud: Windowing equals time domain multiplication by the window function (here rectangular or Hamming). Time domain multiplication equals frequency domain convolution: All frequencies will be replaced by the Fourier transform ...


5

Number of filter banks One of the last steps in the MFCC's calculation is measuring the energy in the filter banks. We do that because want to reduce the dimensionality of our input vector (amplitude spectrum), as well as capture its envelope. Those triangular filters are spaced over the Mel scale: This means that we have very good resolution in low ...


5

No, liftering is never applied before computing MFCC because liftering is defined as a windowing operation in the cepstral domain. So you need cepstral coefficients in order to be able to apply liftering. The effect of liftering is to smoothen the corresponding log magnitude spectrum, which can result in more robust recognition / classification results. ...


4

It's better to copy first frame and last frame values to extend vector sequence beyond boundaries than to assign 0. This could be implemented just by adjusting indexes: if (index1 < 0) index1 = 0 if (index2 > N - 1) index2 = N - 1 delta = v[index1] - v[index2]


4

The rationale behind it is to separate the correlation in the log-spectral magnitudes (from the filterbank) due to the overlapping of the filters. Essentially, the DCT smooths the spectrum representation given by these log-spectral magnitudes. This is incorrect. There is correlation between the log-spectral magnitudes not just because they overlap, but also ...


4

You get a vector of 39 coefficients for each short frame of the signal; not for the entire signal. A speech signal contains segments of voice and silence, and of course when voice is present, there are different phonemes spoken in sequence - so the signal is not stationary and the MFCCs will fluctuate. Extracting the coefficients on a single frame spanning ...


4

Non-verbal Audio (let alone environmental) seems to be the little brother to main stream machine learning media types like images, speech, text. To answer your question is it possible to train a network to identify a given sound? Yes, it is! But it's hard for all the same reasons machine learning is hard. However what's really holding Audio back, and why ...


4

This paper$^1$ suggests: that the reason is because the filtered versions give less weight to the higher coefficients which provide less discrimination than the lower ones. That paper references this one$^2$ which shows the plot below indicating how the application of window reduces the variability. References K.K. Paliwal "DECORRELATED AND LIFTERED ...


4

To stand on @hotpaw2's answers, think of Mel as one kind of pyscho-acoustic scale, derived from a set of experiments on human subjects, others are Bark & ERB Why have such a scale? Imagine a 100 Hz sine wave playing in your head, .... wait for it to stick ...okay... Good. Imagine now, a 200 Hz sine wave playing ... marinate in it. Good work. Now ...


4

I will answer your questions in reverse order. 4: DTW (Dynamic Time Warping) is not a library but an algorithm. It allows aligning two sequences by warping them in time. You can use it for pretty much any kind of sequential data, for which a metric (distance) can be defined. Generally, you calculate a distance between each and every point of both sequences ...


3

It's a very bad idea to stack together consecutive frames in a single vector. By doing so, you are training a classifier to recognize the exact same sound as the one in your training database, pronounced at the exact same speed. For your application (which appears to be speaker identification), you should consider each individual frame as a training ...


3

Here is a solution for sound classification for 10 classes: dog barking, car horn, children playing etc. It is based on tensorflow library using neural networks. Features are extracted by converting sound clips to spectrogram


3

You can have as many MFCC coefficients as you want, 12 is just only a widely used number. As you probably know (or if not then please refer to the old answer), coefficients are being obtained via fitting of cosinusoids to log energies in your filter banks. Usually 12 is enough, more is not improving recognition rate too much. It is better to calculate their ...


3

More than smoothing the DCT reduces the number of dimensions needed to represent the spectrum. DCT is good for dimensionality reduction because it tends to compact most of the energy of the spectrum in the first few coefficients.


3

It would be more accurate to say that the MFCC "capture" or "represent" the spectral envelope - in the sense that two signals with a similar spectral envelope will have a similar sequence of MFCC. The MFCC coefficients themselves are not supposed to "look" like the spectral envelope when plotted. After all the spectral envelope is a "big" vector (as long as ...


3

Yes, this should be enough for a basic isolated word recognition system. Probably not something for a commercial product, but good enough for a university project or demo... It would be better to ask the user to record a word and match against this, rather than attempt to match against a large database of utterances of the same word by different speakers. ...


3

You say: I stream this file via VLC-Player and Icecast2-Server, receive it in Java (with the Player of Javazoom-Lib) When you receive the stream in Java, what is the stream format? You say that it is the "same" because the waveform and/or spectrogram look similar, but the MFCCs will come out different if the stream format (sample rate, bit depth, etc)...


3

I don't believe there is a right answer. DCT or IDCT will achieve the same purpose: decorrelating to put most energy in fewer coefficients. Whatever you do next with your MFCC (compression, feature extraction) will work with either approach. Similarly, I think there are implementations that do an FFT instead of an IFFT for the first stage of the MFCC ...


3

Have a look at these two python libraries that provide a number of audio features easily from WAV files, including MFCC. Librosa: MFCC docs, github Madmom: MFCC docs, github Good luck!


3

Binning is an averaging operation on the (squared) magnitudes of the DFT. You would maybe have $256$ DFT bins but only around $20$ outputs of the filter bank. So you need to average groups of DFT bins to reduce the dimension from $256$ to $20$. For a mel-scaled filter bank, the averaging functions (kernels) are usually triangular, i.e. the center DFT bins ...


2

You are misunderstanding how DTW works. You can actually adjust the frame deletion/insertion costs to adjust how much temporal mismatch you can tolerate. We would have: DTW("Puuuum pum tchak", "Puuum pum tchak") < DTW("Puuuum pum tchak", "Pum pum tchk") < DTW("Puuuum pum tchak", "Purr purr purr"). That is to say, if there is a sample with a perfectly ...


2

Speech - along with the sounds produced by most musical instruments - can be described by a source-filter model. In the case of speech, the source is the glottis - producing a periodical pulse train - and the filter is the vocal tract - acting like a filter with several narrow peaks (formants) shaping the pulse train. When articulating different phonemes (...


2

As you mention, linear prediction attempts to estimate the next signal sample from a linear combination of the previous P outputs. Mathematically, we can then express the $n^{th}$ sample of our signal $x[n]$ like this: \begin{equation} x[n] = \sum^{P}_{k=1} a_k x[n-k]+e[n] \end{equation} where $e[n]$ is the error signal. Applying linear prediction to a ...


Only top voted, non community-wiki answers of a minimum length are eligible