24
$\begingroup$

I have a set of songs for which I extracted the magnitude spectrogram using a Hamming Window with 50% overlap. After extracting the spectrogram, I did some dimensionality reduction using Principal Components Analysis (PCA). After reducing it to lower dimensionality, I reconstructed the spectrograms from lower dimensions. So now, there would be some error between the original spectrogram and the reconstructed spectrogram. I would like to convert this spectrogram back to the audio signal and play it, so that I would be able to know when reconstructed from lower dimensions, how does the audio sound.

Is there any function available in say Matlab. to convert a magnitude Spectrogram to an audio signal ??

Improve this question
$\endgroup$
8
  • 3
    $\begingroup$ You really want the STFT and inverse STFT. "Spectrogram" is just a name for a heat map of the magnitude of the STFT, and magnitude alone is not enough to reconstruct a signal. Look at mathworks.com/matlabcentral/fileexchange/12902-dafx-toolbox/… ? $\endgroup$
    – endolith
    Commented Jul 9, 2013 at 13:44
  • 5
    $\begingroup$ To expand on @endolith's comment, what you are missing when you go from STFT to spectorgram is the phase information, a vital component of the frequency domain representation of your signal. $\endgroup$
    – Bjorn Roche
    Commented Jul 9, 2013 at 16:19
  • $\begingroup$ so that means if I want to reconstruct the original audio signal, I require both the amplitude as well as phase of the STFT ? But genereally to build audio features, the |S| amplitude of complex no is what is used and the phase information is discarded. I have performed PCA on the mel spectrogram which I calculated as $X= log( M |S|)$ M, is the mel filterbank multiplying matrix. so how do you reconstruct the audio signal given $\hat{X}$, the approximation to X got after PCA ? $\endgroup$
    – user76170
    Commented Jul 9, 2013 at 17:05
  • $\begingroup$ @endolith : I tried using the link that you gave mathworks.com/matlabcentral/fileexchange/12902-dafx-toolbox/… I used the phase and amplitude information both ie (complex $S$ as input to the function above). using this I tried playing the signal and it sounded choppy. Why does this happen ? I then computed the norm between the original signal and the one got from inverse STFT procedure as above and it showed a huge value of 3.46*10^3. Any idea why does this happen ?? $\endgroup$
    – user76170
    Commented Jul 15, 2013 at 13:03
  • 1
    $\begingroup$ @user76170: because the spectrogram command doesn't do the same thing as the stft command. you could probably adjust all the parameters to get the same output from them, but why bother? use the stft command and then the istft command to get the reverse. $\endgroup$
    – endolith
    Commented Aug 9, 2013 at 6:58

4 Answers 4

Reset to default
18
$\begingroup$

If the spectrogram was computed as the magnitude of short time fourrier transforms from overlapping windows, then the spectrogram contains implicitly some phase information.

The following iterations do the job :

$$x_{n+1} = \text{istft}(S\cdot\exp(i\cdot\text{angle}(\text{stft}(x_n))))$$

$S$ is the spectrogram, $\text{stft}$ is the forward-short time Fourier transform, $\text{isft}$ is the inverse-short time Fourier transform.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I would like to comment straight at @edouard , but I do not have enough reputation. Does anyone know what $\text{i}$ in his answer is? Also how would I initialize $x_0$? Just random? Is $x_n$ the complete reconstructed signal at iteration $n$ or just the $n^{\text{th}}$ coefficient of $x$? Thanks. $\endgroup$
    – P.R.
    Commented Jun 14, 2014 at 19:46
  • 1
    $\begingroup$ @P.R. It's the unit imaginary number, $\sqrt{-1}$. $\endgroup$
    – Peter K. ♦
    Commented Sep 13, 2017 at 10:51
4
$\begingroup$

I had a bit of a hard time to understand the answer of @edouard, which is doing the right thing. Compare to https://dsp.stackexchange.com/a/3410/9031 , which I used to implement my reconstruction.

Note that $i$ is the imaginary number, and $x_n$ is the reconstructed signal at the $n^{\text{th}}$ iteration. Start with $x_0$ being a random vector of length of the audio signal. For me a few iterations were sufficient to get a result that sounded alright. The absolute error to the original signal was nevertheless quite high. Also the generated spectrogram I generated from the reconstructed signal, although showing the same structures in general, had quite different magnitudes.

Improve this answer
$\endgroup$
2
$\begingroup$

You can use the reconstructed spectrogram versus the original spectrogram to design a filter whose magnitude response transforms one spectrogram to the other. You can then apply this filter to the original time domain data, or to the original FFTs for overlap add/save fast convolution filtering.

Improve this answer
$\endgroup$
5
  • $\begingroup$ @ hotpaw2 : I did not understand your response, why would I want to convert one spectrogram into the other? I want to reconstruct the audio signal given a spectrogram matrix $|S|$. What is the requirement of designing a filter that transforms from one spectrogram to another, and the overlap add/save fast convolution filtering ? I want to reconstruct the audio from $|S|$ so that I can see how effective PCA is. So say I can play two clips, one original audio signal, and the other one reconstructed from lower dimensions $|\hat{S}|$ $\endgroup$
    – user76170
    Commented Jul 10, 2013 at 15:44
  • 2
    $\begingroup$ A spectrogram matrix is lossy, so can't be used for reconstruction. But if you can reverse engineer a transform to produce your desired spectrogram, you can apply it to the original non-lossy time domain data or possibly to the original complex result FFT. $\endgroup$
    – hotpaw2
    Commented Jul 10, 2013 at 16:13
  • $\begingroup$ ok so what you mean to say is that I can reconstruct the audio signal from the complex result $S$ but not just using the magnitude of it $|S|$. Because I have used $|S|$ for my further processing and dimesnionality reduction, then reconstruction of original signal is not possible then, I guess. $\endgroup$
    – user76170
    Commented Jul 13, 2013 at 19:50
  • 1
    $\begingroup$ @user76170 The long and short of it is that you require the complex STFT before you can reconstruct your signal. If you just have the magnitude STFT, that is not enough. There are exceptions to this rule, but generally, you need the complex STFT, not just the magnitude. $\endgroup$
    – Tarin Ziyaee
    Commented Sep 7, 2013 at 23:49
  • $\begingroup$ @hotpaw2 : Is it possible to do it like this, I store the phase information (imaginary part of complex FFT), then take magnitude spectrum $|S|$, then apply dimensionality reduction and then reconstruct from lower dimensions to obtain ${|\hat{S}|}$, add the phase information and do an inverse FFT to obtain the audio signal? $\endgroup$
    – user76170
    Commented Oct 10, 2013 at 23:07
-1
$\begingroup$

Use Griffin-Lim algorithm to invert the audio signal from spectrogram, if you are not worried about computation complexity.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Can you please expand your answer a little bit? Perhaps adding a representative paper or link to the algorithm and a brief explanation of how it is relevant to this question (?) $\endgroup$
    – A_A
    Commented Sep 13, 2017 at 9:26
  • $\begingroup$ If I understand your question correctly, in brief, you want to reconstruct the audio signal from a spectrogram without using the original phase information. Griffin-Lim algorithm requires a spectrogram matrix as input and reconstruct phase iteratively. You can refer the paper ieeexplore.ieee.org/document/1164317 $\endgroup$
    – Jitendra Dhiman
    Commented Oct 23, 2017 at 5:12
  • $\begingroup$ Thank you for letting me know. Just to clarify this point. This response popped up on my review queue as "low quality". The options that I have in terms of a "review" include providing comments for "improvement". For this answer to come in line with the sort of answers commonly encountered in DSP.SE, it would have to go a little bit into the Griffin Lim algorithm to show how it is relevant to what the OP is asking. Any future edits, you can apply directly to your answer. The point of this is not to satisfy "me" in particular, it is to have a meaningful set of illuminating answers to a question $\endgroup$
    – A_A
    Commented Oct 23, 2017 at 8:58
  • $\begingroup$ Best answer is here (using Griffin-Lim) in case you don't have original FFT information. timsainb.github.io/… $\endgroup$
    – Artemi Krymski
    Commented Nov 22, 2018 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.