8
$\begingroup$

I have a set of songs for which I extracted the STFT (Short-Time Fourier Transform) and used the magnitude spectrum $|S|$ to calculate the mel spectrogram by using a mel filterbank matrix $M$, so $X=\log(M\times |S|)$. I want to know is there any method to reverse this process, i.e. convert from the mel spectrogram back to the spectrogram. I performed some dimensionality reduction on the mel spectrogram, and reconstructed the mel spectrogram from lower dimensions. Now I want to regenerate the audio signal from the reconstructed mel spectrogram, so I guess first reconstruct the spectrogram and then the audio signal.

The problem is that the mel filter bank matrix is not a square matrix, since we the reduce the no of frequency bins, so inverse of $M$ cant be used like this : $ \hat{S}=M^{-1}\exp(X)$. So is there any way to generate the inverse mapping, like some inverse transfer function that can convert from $X$ to $S$?

$\endgroup$
  • $\begingroup$ This is simply not possible unless you have a very strong prior model of what your sources sounded like. Without any prior information on your sources, there are indeed many different spectra that can be "collapsed" into the same set of mel frequency coefficients. $\endgroup$ – pichenettes Jul 29 '13 at 10:41
  • $\begingroup$ @pichenettes : Then how do reconstruct the original song, I hope you have understood the problem. I have a mel-spectrogram, and a reconstructed version of it from lower dimensions. Instead of reporting that "the error was " in terms of numeric figures, if there is a way I could somehow "hear" the reconstructed mel-spectrogram , then I can say that after the dimensionality reduction, by retaining less components and reconstructing the mel-spectrogram, this is how the audio sounds. $\endgroup$ – user76170 Jul 29 '13 at 14:01
  • 1
    $\begingroup$ It is simply not possible in the general case. It is like having 5 numbers and asking how to "reconstruct" the numbers form their sum. $\endgroup$ – pichenettes Jul 29 '13 at 14:20
  • $\begingroup$ We wanna make voice control system on the matlab but we don't know how we can do this.We have voice data but how can we put this data in the Matlab. e.g (eight1.mel....eight.mel) files . We want this audio english numbers recognition in matlab . 1-Must i convert ".mel-waw" files? 2-Which can i use codes? If you help us,we'd appreciate Thanks Regards $\endgroup$ – user21133 May 22 '16 at 15:36
5
$\begingroup$

Both taking a magnitude spectrogram and a Mel filter bank are lossy processes. Important information needed to reconstruct the original will have been lost. Thus you need to go back and use the original audio samples to do the reconstruction by determining a time or frequency domain filter equivalent to your dimensionality reduction.

You can make assumptions about the lost information, but those assumptions themselves usually sound inaccurate, artificial and/or robotic. Or you can use only specially synthesized input, where the assumptions will be correct by design of that input.

$\endgroup$
  • $\begingroup$ Is there an equivalent reverse transformation for Mel fiter bank ? I understand that it is a lossy process. and we cannot find $M^{-1}$ since M is not square. However, is it possible to skip that step, and directly perform PCA on magnitude spectrogram |M| ? Also store the phase information. Then reconstruct |M|, combine the phase information, and get back the original audio ? $\endgroup$ – user76170 Nov 4 '13 at 10:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.