Mel Cepstral Distortion

I am working on a speech synthesis model and I am looking to evaluate my synthesized speech. I found that most people use the Mel Cepstral Distortion (MCD) which can be calculated by the formula:

$$\frac{10\sqrt{2}}{\ln10}\frac{1}{T}\sum_t\sqrt{\sum^{25}_i 2||mc(t,i)-mc_{synth}(t,i)||^2}$$

I have ground-truth speech and the synthesised version so I calculate the MFCCs for both (taking 25 coefficients) and then apply the formula. The result I get is around 300, which seems orders of magnitude larger than what I should get. I tried doing cepstral mean normalization which brings the number down to around 30 but I notice that most of the time this number is <10. The speech I synthesize is not so bad so I must be doing something wrong. Does anyone have any ideas?

P.S. The way I produce the speech ensures that it be aligned with the original(more or less). I wonder if a slight misalignment could cause this issue and if so should I use DTW to align the audios?

Edit Also If anyone knows of a library that calculated MCD when given an audio file then I could use that to compare my results.

• Is the segmentation the same across audio streams? If you can generate the timestamps of the frames the coefficients are calculated on then you might be able to compare the corresponding frames more accurately. – A_A Apr 2 '19 at 6:52
• I think the two signals are aligned. Do you think that misalignment is the problem I'm experiencing? Also I'm using frame length 20ms and stride 10ms for the mfccs. Could this be the problem – MrHat Apr 2 '19 at 7:22
• OK, another way to cross check the error quantity is to subtract one waveform from another. Is the error as small as you would expect in that case? – A_A Apr 2 '19 at 7:24
• when subtracting one from another I get an average difference of 35. My signals are 16-bit integers so they have values range [-32767, 32767]. To me this seems reasonable but I have no idea what others get when they synthesize. Another problem is I can't seem to find two reference signals for which the MCD is known so that I can try to reproduce that. – MrHat Apr 2 '19 at 7:32
• I am having similar issues in computing MCD. I need to compare results with a paper - which seems to have mcds of order 1-10, but in my computations we get a number that is 10 times larger. I am aligning my sequences (this is for speech waveforms) using the dtwalign tool, and then compute the MCD using the same formula above, with librosa to create mels and mfccs. There seems to be two ways of getting the mfccs, either through the raw waveform or through the mel spectrogram with librosa's api. Could you describe how you managed to reduce the disparity, if possible? – kakrafoon Jun 18 '19 at 21:49

Ok so here is what I found. The distance is dependent on the way that the mfccs are calculated. This makes sense to me and also explains why cepstral mean normalization affects the values of the MCD.

I found this implementation (https://github.com/MattShannon/mcd), which unfortunately did not support .wav files. I ran this and it gives results that are in the correct range for its test examples. I looked into the code and found that the the difference was not so much in the calculation of the MCD (which is straight forward) but rather the mfccs. The library I was using gave mfccs at a different scale (I tried python_speech_features and pyspeech). Using the mfccs from Matlab gave me mfccs in a similar range to that of the repo so I opted to use those.

A noteworthy point is that the first coefficient is quite large which is why it is ignored in (https://github.com/MattShannon/mcd) so I did the same. There are some artifacts in the speech I synthesise which likely accounts for why my MCD is still above 10 but its still lower than 30 now.

I'm not sure that the formula posted in the question is accurate. According to the paper which introduced the measure (eq. 1) and to this paper (eqs. 1a and 1b), the sum over the cepstral coefficients (denoted by $$i$$) should be inside the square root; that is:

$$\frac{10 \sqrt{2}}{\ln 10} \frac{1}{T} \sum_{t=1}^T \sqrt{\sum_{i} \left(C_{ti} - \hat{C}_{ti}\right)^2}.$$

This formula can be implemented in Python (in vectorized form using NumPy) as follows:

import numpy as np

def mcd(C, C_hat):
"""C and C_hat are NumPy arrays of shape (T, D),
representing mel-cepstral coefficients.

"""
K = 10 / np.log(10) * np.sqrt(2)
return K * np.mean(np.sqrt(np.sum((C - C_hat) ** 2, axis=1)))


But as you point out, the MCD between two WAV samples depends on the underlying MFCC implementation. The second paper that I've mentioned gives more information about the MFCC extraction process (§2.1), but in my (limited) experience these details do not suffice – different implementations give different MFCC coefficients:

For TTS applications, typical parameters are 25-D mel frequency-scaled cepstral coefficients with a frame step size of 5 ms. [...] In this paper, [...] the power term is ignored. We adopt this choice so that the distortion measure is not influenced by the speaker's loudness [...].

• Yes sorry that was a typo. I have corrected it on the original question. – MrHat Mar 4 at 15:42