First of all, FT and IFT of a discrete sequence are the same up to complex conjugation/scaling - these two operations will thus yield qualitatively the same result (in the sense that subsequence processing/statistical modeling will make abstraction of these differences anyway). The textbook definition of cepstrum requires a Fourier transform. People sometimes use the inverse Fourier transform to emphasizes that the X axis on the cepstrum plot can indeed be interpreted as a time (rather than as a quefrency).
The MFCC extraction process found in speech and audio analysis systems uses the DCT since it has the advantage of being a real -> real transform, and because the resulting coefficients are in practice decorrelated from each other - which is not the case if you take the sequence of real/imaginary part of a FT or IFT. This makes the resulting data very good for subsequent modeling with gaussian models with diagonal covariance matrix.
Another view on this: take thousands of hours of audio, extract log mel frequency coefficients (through a bank of filters with triangular responses, with center frequencies distributed on the Mel scale) for each analysis frame. You'll get a big dataset... Do a PCA on it to extract the dimensionality reduction transform that captures most of the variance of the mel coefficients. The transform matrix the PCA will produce will be uncannily close to the DCT matrix.
