I have been experimenting with speaker independent automatic speech recognition. At this point, I understand that the signal is typically segmented into overlapping frames and MFCCs are extracted as features for each frame.

A lot of the texts I read talk about reducing the dimensionality of the features using (for example) Linear Discriminant Analysis or Primary Component Analysis. While I think I understand the concepts involved in reducing dimensions in the abstract, I am not understanding how this feature reduction is supposed to be applied to the MFCC vectors.

I am interested primarily because several texts seem to suggest that reducing the dimensionality produces less variant features, which make recognizing patterns easier. As a bonus, there is also less computational overhead.

Is the dimension reduction supposed to be applied on a per-frame basis? For something like LDA, don't you need to know what class the data belongs to?

Thanks, Carl

  • $\begingroup$ See nonnegative matrix factorisation. $\endgroup$
    – Deniz
    Jan 16, 2014 at 1:10

1 Answer 1


You have one feature vector per frame. The collection of feature vectors for all the frames in the training data form a matrix. Since this is training data, you know the true class for each frame. Use this matrix and the training data labels to build your LDA matrix. Alternatively, just use this matrix (without the labels) to find the PCA matrix.

Now you use this matrix to reduce the dimensionality of the feature vector for each frame in your test data when you're doing the speech recognition.

PCA doesn't produce less variant features. I think what you meant to say was that the features are less correlated with each other. This is called whitening and it can help some classifiers better learn from the training data.

  • $\begingroup$ The PCA explanation makes sense to me. However I am not clear how one might determine what the classes should be for LDA. What are the labels supposed to represent in the training data? It is my understanding that the number of classes ought to be less than the number of dimensions. $\endgroup$
    – Carl
    Jan 16, 2014 at 17:57
  • $\begingroup$ What is it that you are trying to predict with your features? $\endgroup$
    – Aaron
    Jan 16, 2014 at 23:41
  • $\begingroup$ I'd like to be able train a database of template phonemes and the recognize the phonemes from an untrained speech sample. I use DTW to compare the distance between each segment in the untrained signal against each template and emit the closest. $\endgroup$
    – Carl
    Jan 17, 2014 at 1:09
  • $\begingroup$ If you are trying to recognize phonemes then phonemes will be your feature for LDA $\endgroup$
    – Aaron
    Jan 17, 2014 at 1:10
  • $\begingroup$ There are 13 MFCCs (39 if you take deltas and delta-deltas). This is fewer than the number of phonemes in English. So if the number of classes is so big, does LDA help? Am I misunderstanding it? $\endgroup$
    – Carl
    Jan 17, 2014 at 2:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.