I've been reading bits and pieces online but I just can't piece it all together. I have some background knowledge of signals / DSP stuff which should be enough prerequisites for this. I'm interested in eventually coding this algorithm in Java but I don't understand it fully yet which is why I'm here (it counts as math, right?).
Here's how I think it works along with the gaps in my knowledge.
Start with your audio speech sample, say a .wav file, that you can read into an array. Call this array x[n], where n ranges from 0, 1, ... N-1 (so N samples). The values correspond to audio intensity I guess - amplitudes.
Split the audio signal into distinct "frames" of 10ms or so where you assume the speech signal is "stationary". This is a form of quantization. So if your sample rate is 44.1KHz, 10ms is equal to 441 samples, or values of x[n].
Do a Fourier transform (FFT for computation's sake). Now is this done on the entire signal or on each separate frame of x[n]? I think there's a difference because in general the Fourier transform looks at all elements of a signal, so F(x[n]) =/= F(x1[n]) joined with F(x2[n]) joined with ... F(xN[n]) where xi[n] are the smaller frames. Anyways, say we do some FFT and end up with X[k] for the rest of this.
Mapping to the Mel scale, and logging. I know how to convert regular frequency numbers to the Mel scale. For each k of X[k] (the "x-axis" if you'll allow me), you can do the formula here: http://en.wikipedia.org/wiki/Mel_scale. But how about the "y-values" or the amplitudes of X[k]? Do they just remain the same values but shifted to the appropriate spots on the new Mel (x-) axis? I saw in some paper there was something about logging the actual values of X[k] because then if X[k] = A[k]*B[k] where one of those signals is presumed to be noise you don't want, the log operation on this equation turns the multiplicative noise into additive noise, which hopefully can be filtered (?).
Now the final step is to take a DCT of your modified X[k] from above (however it ended up getting modified). Then you take the amplitudes of this final result and those are your MFCCs. I read something about throwing away high frequency values.
So I'm trying to really iron out how to compute these guys step by step, and clearly some things are eluding me from above.
Also, I've heard about using "filter banks" (an array of band pass filters basically) and don't know if this refers to making the frames from the original signal, or maybe you make the frames after the FFT?
Lastly, there's something I saw about MFCCs having 13 coefficients?