Hot answers tagged

26

Step by step... 1. & 2. This is correct. Note that the frames are usually overlapping, for example, frame 0 is samples 0 to 440; frame 1 is samples 220 to 660; frame 2 is samples 440 to 880 and so on... Note also that a window function is applied to the samples in the frame. 3. The Fourier transform is done for each frame. The motivation behind this ...


19

Just to make things clear - this property is not fundamental but important. It is the fundamental difference when it comes to using DCT instead of DFT for spectrum calculation. Why do we do Cepstral Mean Normalisation In speaker recognition we want to remove any channel effects (impulse response of vocal tract, audio path, room, etc.). Providing that ...


17

Let me start from the beginning. The standard way of calculating cepstrum is following: $$C(x(t))=\mathcal{F}^{-1}[\log(\mathcal{F}[x(t)])] $$ In the case of the MFCC coefficients case is a bit different, but still similar. After pre-emphasis and windowing, you calculate the DFT of your signal and apply the filter bank of the overlapping triangular ...


16

By long shot it is doable - to what extend? You will see. This task of environmental sound classification is not very well studied. Also choice of machine learning paradigm is crucial - statistical approach or maybe binary classifier? You can start with GMM's, ANN's and SVM's - I opt for GMM's and ANN's. Yes, most of people are using MFCC's because they are ...


13

First, you will have to correct for differences in timing. For example, if one utterance is "--heeelloooo---" and the other "hellooooooo----" (- representing silence), a direct pairwise comparison of MFCC frames will show differences simply because the two samples are not aligned. You can use Dynamic Time Warping to find the best alignment between the two ...


11

There's a lot of literature on MFCCs on the web, so it would be a bit easier if you could be more specific as to which part of the processing you don't understand. But I'll give an overview of what needs to be done, hoping this is helpful for you: compute the squared magnitudes of the FFT bins weigh the bins using triangular windows; usually the windows are ...


11

The logarithm serves to transform a multiplication into an addition. It is part of the computation of the cepstrum. The basic idea is as follows: Assume a source signal $x$ is convolved by some impulse response $h$. The resulting magnitude spectrum is $$|Y(\omega)| = |X(\omega)||H(\omega)|$$ By applying the logarithm we get $$\log |Y(\omega)| = \log |X(\...


10

The way MFCC's are always used is by feeding them into the classifier. This can be done on a frame-by-frame basis (12x1 vector), or by concatenating (12xN) - same as a spectrogram. Thus for DTW, you must perform the classification by calculating the distance between 12D vectors. It's a Dynamic Time Warping, so the difference must be calculated between ...


6

How should I combine these groups of 12 coefficients to get some value(s) that describe the whole signal? I don't think "flattening" the sequence of $215 \times 12$ coefficients into a $1 \times N$ vector, and then computing the distance between these vectors to compare two sounds is a good idea. This approach collapses the temporal information - the way ...


6

First of all, FT and IFT of a discrete sequence are the same up to complex conjugation/scaling - these two operations will thus yield qualitatively the same result (in the sense that subsequence processing/statistical modeling will make abstraction of these differences anyway). The textbook definition of cepstrum requires a Fourier transform. People ...


6

First of all there is some serious "cheating" in the MFCC reconstruction experiment you linked to: not only the MFCCs are used, but also the voiced/unvoiced bit and the pitch. MFCC are not speaker independent. In fact, they are used for speaker identification/verification tasks! The speaker "idiosyncracies" are both in their prosody (preserved by this ...


6

You could collect a large collection of voice signals, extract the MFCCs, and compute the mean and standard deviation of each coefficient. Keep these numbers and use them once and for all. Then, you can normalize the sequence of vectors from an utterance by subtracting the mean and dividing by the standard deviation you have pre-computed (do not recompute ...


5

Main difference is the frequency of the fundamental, which is about an octave higher for women with the split point around 160 Hz or so. A fundamental lower than 160 Hz is most likely a male and a fundamental higher is most likely a female. A good overview over a number of studies on the topic can be found at http://www2.ling.su.se/staff/hartmut/f0_m%26f.pdf


5

Mel filter bank is important due to following reasons: It applies the Mel-frequency scaling, which is perceptual scale that helps to simulate the way human ear works. It corresponds to better resolution at low frequencies and less at high. Using the triangular filter-bank helps to capture the energy at each critical band and gives a rough approximation of ...


5

Ideally you would only see that frequency as a narrow peak but because of the finite-length window you also get that other artificial crud: Windowing equals time domain multiplication by the window function (here rectangular or Hamming). Time domain multiplication equals frequency domain convolution: All frequencies will be replaced by the Fourier transform ...


5

Number of filter banks One of the last steps in the MFCC's calculation is measuring the energy in the filter banks. We do that because want to reduce the dimensionality of our input vector (amplitude spectrum), as well as capture its envelope. Those triangular filters are spaced over the Mel scale: This means that we have very good resolution in low ...


5

No, liftering is never applied before computing MFCC because liftering is defined as a windowing operation in the cepstral domain. So you need cepstral coefficients in order to be able to apply liftering. The effect of liftering is to smoothen the corresponding log magnitude spectrum, which can result in more robust recognition / classification results. ...


4

It's better to copy first frame and last frame values to extend vector sequence beyond boundaries than to assign 0. This could be implemented just by adjusting indexes: if (index1 < 0) index1 = 0 if (index2 > N - 1) index2 = N - 1 delta = v[index1] - v[index2]


4

Do we really need 10 samples of each word? or can we "combine" them into a single template? and if so, how? You can use single template, 10 templates will make result more accurate. Should we run the SVM directly on the MFCCs vectors? On the MFCC vector of vectors (for each word)? On the DTW values? or should we combine all the MFCC vectors into a single ...


4

Non-verbal Audio (let alone environmental) seems to be the little brother to main stream machine learning media types like images, speech, text. To answer your question is it possible to train a network to identify a given sound? Yes, it is! But it's hard for all the same reasons machine learning is hard. However what's really holding Audio back, and why ...


4

You get a vector of 39 coefficients for each short frame of the signal; not for the entire signal. A speech signal contains segments of voice and silence, and of course when voice is present, there are different phonemes spoken in sequence - so the signal is not stationary and the MFCCs will fluctuate. Extracting the coefficients on a single frame spanning ...


4

To stand on @hotpaw2's answers, think of Mel as one kind of pyscho-acoustic scale, derived from a set of experiments on human subjects, others are Bark & ERB Why have such a scale? Imagine a 100 Hz sine wave playing in your head, .... wait for it to stick ...okay... Good. Imagine now, a 200 Hz sine wave playing ... marinate in it. Good work. Now ...


4

I will answer your questions in reverse order. 4: DTW (Dynamic Time Warping) is not a library but an algorithm. It allows aligning two sequences by warping them in time. You can use it for pretty much any kind of sequential data, for which a metric (distance) can be defined. Generally, you calculate a distance between each and every point of both sequences ...


3

It would be more accurate to say that the MFCC "capture" or "represent" the spectral envelope - in the sense that two signals with a similar spectral envelope will have a similar sequence of MFCC. The MFCC coefficients themselves are not supposed to "look" like the spectral envelope when plotted. After all the spectral envelope is a "big" vector (as long as ...


3

Yes, this should be enough for a basic isolated word recognition system. Probably not something for a commercial product, but good enough for a university project or demo... It would be better to ask the user to record a word and match against this, rather than attempt to match against a large database of utterances of the same word by different speakers. ...


3

Regarding your existing steps: Do you have some overlap between adjacent blocks? It is common in feature extraction systems to have some overlap between adjacent blocks, so that a short transient event can be correctly captured (if it is right at the end of a block, it will be right in the middle of the next one). Your idea of discarding blocks is dangerous....


3

DTW is not proportional to anything, including frame length. It's not a good idea to compare DTW values between two different pairs like you are doing. The issue is that DTW is not just the number but also an alignment. If alignment does not match, you can not relate numbers. You only can relate numbers if you align to the same thing. Normalization is ...


3

More than smoothing the DCT reduces the number of dimensions needed to represent the spectrum. DCT is good for dimensionality reduction because it tends to compact most of the energy of the spectrum in the first few coefficients.


3

The rationale behind it is to separate the correlation in the log-spectral magnitudes (from the filterbank) due to the overlapping of the filters. Essentially, the DCT smooths the spectrum representation given by these log-spectral magnitudes. This is incorrect. There is correlation between the log-spectral magnitudes not just because they overlap, but also ...


3

Here is a solution for sound classification for 10 classes: dog barking, car horn, children playing etc. It is based on tensorflow library using neural networks. Features are extracted by converting sound clips to spectrogram


Only top voted, non community-wiki answers of a minimum length are eligible