Given MFCC values for some wav’s, how can I easily tell if all speakers are the same?

Question

Context: for a research project I have at regular times a batch of wav files with recordings of voice. The hypothesis is that all speakers in a batch are the same, but there might be 1 or a few “others” in it. There are 5 to 10 files in a batch.

My goal is to obtain a similarity matrix for all wav files (speakers) in a batch.

What I got so far: I calculate MFCC (16 values) with an existing tool, I have between 2000 and 4000 frames per wav file. Simply taking the average of MFCCx is not meaningful. Next I perform a GMM clustering on each wav and obtain something like this: (MFCC0 discarded)

Wav1    
Class   Mean (MFCC1)    Mean (MFCC2) …     Proportion
        1       -44             6               0,09
        2        46           -23               0,37
        3        20           -26               0,10
        4        43           -43               0,06
        5        57           -52               0,06
        6        66           -55               0,18
        7       108           -46               0,15

And here I am stuck, what do I do next to get a similarity matrix, can I calculate some “distance” between the different wavs and how?

Voice is natural talk. Recordings are under various channel conditions (sometimes background noise) and can be with varying emotional status (eg laughter, some excitement,..). Speakers are of random nature (man, woman, old, young, …).

I would like to avoid more difficult algorithms for speaker recognition because that is beyond my current knowledge. Given that speakers are random, it will be rare that the other(s) is very similar to the normal speaker, and some error rate is acceptable, so a basic method should be ok.

Answer 1

Given MFCC values for some wav’s, how can I easily tell if all speakers are the same?

In general, you would deal with this kind of problem using some form of Clustering.

The fundamental premise behind this idea is that the features of similar data items would have similar values.

How you assess similarity, how you assess "a cluster" and what this means for the particular application you are dealing with are very much context specific issues.

In this particular example, the features are the MFCC coefficients. If these are similar between two frames we would tend to think that the frames contain more or less signals produced from the same source.

The simplest clustering to apply would be k-means clustering but this is not going to help too much here, especially if you don't know how many speakers you might get in a recording. So, long story short, you will most likely gravitate to something like hierarchical clustering.

You don't have to extract the means of each feature to cluster them. Doing so would be equivalent to training and using a minimum distance classifier. The only thing that is needed here to complete that is the "ground truth" knowledge that will tell you which centroids belong to which speakers.

Note here that if you have two speakers talking in a mostly silent room, the "feature space" of the clustering is not necessarily expected to be divided into two distinct "point clouds". It could, for example, contain separate regions for background silence, one speaker talking, the other speaker talking, both speakers talking at the same time. This last class might contain further grading depending on how loudly was each speaker talking while interrupting the other.

Hierarchical clustering allows you to focus on details with varying degrees of "attention", provided that the MFCC clusters the feature space with sufficient accuracy.

The other thing you can do, is a Principal Component Analysis on the MFCC matrix which (hopefully) will return one component for each one of these "major events" happening in the file.

Hope this helps.

EDIT: Forgot to add, if you are looking for the terminology that could return to you some useful results in varying levels of complexity, this is "Speaker Diarisation". See for example here.