Mel Cepstral Distortion

Question

I am working on a speech synthesis model and I am looking to evaluate my synthesized speech. I found that most people use the Mel Cepstral Distortion (MCD) which can be calculated by the formula:

$$\frac{10\sqrt{2}}{\ln10}\frac{1}{T}\sum_t\sqrt{\sum^{25}_i 2||mc(t,i)-mc_{synth}(t,i)||^2}$$

I have ground-truth speech and the synthesised version so I calculate the MFCCs for both (taking 25 coefficients) and then apply the formula. The result I get is around 300, which seems orders of magnitude larger than what I should get. I tried doing cepstral mean normalization which brings the number down to around 30 but I notice that most of the time this number is <10. The speech I synthesize is not so bad so I must be doing something wrong. Does anyone have any ideas?

P.S. The way I produce the speech ensures that it be aligned with the original(more or less). I wonder if a slight misalignment could cause this issue and if so should I use DTW to align the audios?

Edit Also If anyone knows of a library that calculated MCD when given an audio file then I could use that to compare my results.

Answer 1

Ok so here is what I found. The distance is dependent on the way that the mfccs are calculated. This makes sense to me and also explains why cepstral mean normalization affects the values of the MCD.

I found this implementation (https://github.com/MattShannon/mcd), which unfortunately did not support .wav files. I ran this and it gives results that are in the correct range for its test examples. I looked into the code and found that the the difference was not so much in the calculation of the MCD (which is straight forward) but rather the mfccs. The library I was using gave mfccs at a different scale (I tried python_speech_features and pyspeech). Using the mfccs from Matlab gave me mfccs in a similar range to that of the repo so I opted to use those.

A noteworthy point is that the first coefficient is quite large which is why it is ignored in (https://github.com/MattShannon/mcd) so I did the same. There are some artifacts in the speech I synthesise which likely accounts for why my MCD is still above 10 but its still lower than 30 now.

Answer 2

I'm not sure that the formula posted in the question is accurate. According to the paper which introduced the measure (eq. 1) and to this paper (eqs. 1a and 1b), the sum over the cepstral coefficients (denoted by $i$) should be inside the square root; that is:

$$ \frac{10 \sqrt{2}}{\ln 10} \frac{1}{T} \sum_{t=1}^T \sqrt{\sum_{i} \left(C_{ti} - \hat{C}_{ti}\right)^2}. $$

This formula can be implemented in Python (in vectorized form using NumPy) as follows:

import numpy as np

def mcd(C, C_hat):
    """C and C_hat are NumPy arrays of shape (T, D),
    representing mel-cepstral coefficients.

    """
    K = 10 / np.log(10) * np.sqrt(2)
    return K * np.mean(np.sqrt(np.sum((C - C_hat) ** 2, axis=1)))

But as you point out, the MCD between two WAV samples depends on the underlying MFCC implementation. The second paper that I've mentioned gives more information about the MFCC extraction process (§2.1), but in my (limited) experience these details do not suffice – different implementations give different MFCC coefficients:

For TTS applications, typical parameters are 25-D mel frequency-scaled cepstral coefficients with a frame step size of 5 ms. [...] In this paper, [...] the power term is ignored. We adopt this choice so that the distortion measure is not influenced by the speaker's loudness [...].