I am working with phoneme clustering. I have been using $k$-means clustering have done $k$-means on 2 different data: spectrogram vector and MFCCs. I would like to know how I can evaluate the result of clustering quality? I mean I need to know standard ways to validate the clustering. It would great if someone explains their experience.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ You should have the groundtruth first! $\endgroup$– user13107Commented Aug 26, 2013 at 13:30
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
One standard way to evaluate the quality of such techniques is to look at the distribution of correct clusterings versus incorrect clusterings.
This can be quantified by looking at the precision, recall and quality of the clusterings... but assumes that you have a "ground truth" (i.e. that you know the real cluster the data point belongs to).
Precision: $p = \frac{TP}{TP + FP}$, the percentage of positives that are true positives (and not mis-classified negatives).
Recall: $r = \frac{TP}{TP + FN}$, the percentage of true positives that were correctly identified.
Quality (AKA accuracy) = $q = \frac{TP + TN}{TP + TN + FP + FN}$, the percentage of correctly classified items from all classified items.
Here $TP$ is the number of true positives, $TN$ the number of true negatives, $FP$ the number of false positives, $FN$ is the number of false negatives.
If you have many clusters and need to evaluate their interaction, then a confusion matrix may be more useful.
If you want to go into this in even more detail, I'd suggest reading the paper:
Andrew P. Bradley, The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition, Volume 30, Issue 7, July 1997, Pages 1145-1159, ISSN 0031-3203, http://dx.doi.org/10.1016/S0031-3203(96)00142-2. (http://www.sciencedirect.com/science/article/pii/S0031320396001422) Keywords: The ROC curve; The area under the ROC curve (AUC); Accuracy measures; Cross-validation; Wilcoxon statistic; Standard error
