19
$\begingroup$

I have a set of songs for which I extracted the magnitude spectrogram using a Hamming Window with 50% overlap. After extracting the spectrogram, I did some dimensionality reduction using Principal Components Analysis (PCA). After reducing it to lower dimensionality, I reconstructed the spectrograms from lower dimensions. So now, there would be some error between the original spectrogram and the reconstructed spectrogram. I would like to convert this spectrogram back to the audio signal and play it, so that I would be able to know when reconstructed from lower dimensions, how does the audio sound.

Is there any function available in say Matlab. to convert a magnitude Spectrogram to an audio signal ??

$\endgroup$
  • 2
    $\begingroup$ You really want the STFT and inverse STFT. "Spectrogram" is just a name for a heat map of the magnitude of the STFT, and magnitude alone is not enough to reconstruct a signal. Look at mathworks.com/matlabcentral/fileexchange/12902-dafx-toolbox/… ? $\endgroup$ – endolith Jul 9 '13 at 13:44
  • 4
    $\begingroup$ To expand on @endolith's comment, what you are missing when you go from STFT to spectorgram is the phase information, a vital component of the frequency domain representation of your signal. $\endgroup$ – Bjorn Roche Jul 9 '13 at 16:19
  • $\begingroup$ so that means if I want to reconstruct the original audio signal, I require both the amplitude as well as phase of the STFT ? But genereally to build audio features, the |S| amplitude of complex no is what is used and the phase information is discarded. I have performed PCA on the mel spectrogram which I calculated as $X= log( M |S|)$ M, is the mel filterbank multiplying matrix. so how do you reconstruct the audio signal given $\hat{X}$, the approximation to X got after PCA ? $\endgroup$ – user76170 Jul 9 '13 at 17:05
  • $\begingroup$ @endolith : I tried using the link that you gave mathworks.com/matlabcentral/fileexchange/12902-dafx-toolbox/… I used the phase and amplitude information both ie (complex $S$ as input to the function above). using this I tried playing the signal and it sounded choppy. Why does this happen ? I then computed the norm between the original signal and the one got from inverse STFT procedure as above and it showed a huge value of 3.46*10^3. Any idea why does this happen ?? $\endgroup$ – user76170 Jul 15 '13 at 13:03
  • $\begingroup$ @user76170: choppy is because the STFT chops up the signal into frames, sometimes overlapping, and you have to deconstruct them in the same way they were constructed or there will be discontinuities at each one. Did you use the STFT and ISTFT functions from that link? Look at the waveform of the choppy signal so you can see what the problem is. $\endgroup$ – endolith Jul 15 '13 at 14:51
15
$\begingroup$

If the spectrogram was computed as the magnitude of short time fourrier transforms from overlapping windows, then the spectrogram contains implicitly some phase information.

The following iterations do the job :

$$x_{n+1} = \text{istft}(S\cdot\exp(i\cdot\text{angle}(\text{stft}(x_n))))$$

$S$ is the spectrogram, $\text{stft}$ is the forward-short time Fourier transform, $\text{isft}$ is the inverse-short time Fourier transform.

$\endgroup$
  • $\begingroup$ I would like to comment straight at @edouard , but I do not have enough reputation. Does anyone know what $\text{i}$ in his answer is? Also how would I initialize $x_0$? Just random? Is $x_n$ the complete reconstructed signal at iteration $n$ or just the $n^{\text{th}}$ coefficient of $x$? Thanks. $\endgroup$ – P.R. Jun 14 '14 at 19:46
  • 1
    $\begingroup$ @P.R. It's the unit imaginary number, $\sqrt{-1}$. $\endgroup$ – Peter K. Sep 13 '17 at 10:51
3
$\begingroup$

I had a bit of a hard time to understand the answer of @edouard, which is doing the right thing. Compare to https://dsp.stackexchange.com/a/3410/9031 , which I used to implement my reconstruction.

Note that $i$ is the imaginary number, and $x_n$ is the reconstructed signal at the $n^{\text{th}}$ iteration. Start with $x_0$ being a random vector of length of the audio signal. For me a few iterations were sufficient to get a result that sounded alright. The absolute error to the original signal was nevertheless quite high. Also the generated spectrogram I generated from the reconstructed signal, although showing the same structures in general, had quite different magnitudes.

$\endgroup$
2
$\begingroup$

You can use the reconstructed spectrogram versus the original spectrogram to design a filter whose magnitude response transforms one spectrogram to the other. You can then apply this filter to the original time domain data, or to the original FFTs for overlap add/save fast convolution filtering.

$\endgroup$
  • $\begingroup$ @ hotpaw2 : I did not understand your response, why would I want to convert one spectrogram into the other? I want to reconstruct the audio signal given a spectrogram matrix $|S|$. What is the requirement of designing a filter that transforms from one spectrogram to another, and the overlap add/save fast convolution filtering ? I want to reconstruct the audio from $|S|$ so that I can see how effective PCA is. So say I can play two clips, one original audio signal, and the other one reconstructed from lower dimensions $|\hat{S}|$ $\endgroup$ – user76170 Jul 10 '13 at 15:44
  • 2
    $\begingroup$ A spectrogram matrix is lossy, so can't be used for reconstruction. But if you can reverse engineer a transform to produce your desired spectrogram, you can apply it to the original non-lossy time domain data or possibly to the original complex result FFT. $\endgroup$ – hotpaw2 Jul 10 '13 at 16:13
  • $\begingroup$ ok so what you mean to say is that I can reconstruct the audio signal from the complex result $S$ but not just using the magnitude of it $|S|$. Because I have used $|S|$ for my further processing and dimesnionality reduction, then reconstruction of original signal is not possible then, I guess. $\endgroup$ – user76170 Jul 13 '13 at 19:50
  • 1
    $\begingroup$ @user76170 The long and short of it is that you require the complex STFT before you can reconstruct your signal. If you just have the magnitude STFT, that is not enough. There are exceptions to this rule, but generally, you need the complex STFT, not just the magnitude. $\endgroup$ – Tarin Ziyaee Sep 7 '13 at 23:49
  • $\begingroup$ @hotpaw2 : Is it possible to do it like this, I store the phase information (imaginary part of complex FFT), then take magnitude spectrum $|S|$, then apply dimensionality reduction and then reconstruct from lower dimensions to obtain ${|\hat{S}|}$, add the phase information and do an inverse FFT to obtain the audio signal? $\endgroup$ – user76170 Oct 10 '13 at 23:07
-1
$\begingroup$

Use Griffin-Lim algorithm to invert the audio signal from spectrogram, if you are not worried about computation complexity.

$\endgroup$
  • $\begingroup$ Can you please expand your answer a little bit? Perhaps adding a representative paper or link to the algorithm and a brief explanation of how it is relevant to this question (?) $\endgroup$ – A_A Sep 13 '17 at 9:26
  • $\begingroup$ If I understand your question correctly, in brief, you want to reconstruct the audio signal from a spectrogram without using the original phase information. Griffin-Lim algorithm requires a spectrogram matrix as input and reconstruct phase iteratively. You can refer the paper ieeexplore.ieee.org/document/1164317 $\endgroup$ – Jitendra Dhiman Oct 23 '17 at 5:12
  • $\begingroup$ Thank you for letting me know. Just to clarify this point. This response popped up on my review queue as "low quality". The options that I have in terms of a "review" include providing comments for "improvement". For this answer to come in line with the sort of answers commonly encountered in DSP.SE, it would have to go a little bit into the Griffin Lim algorithm to show how it is relevant to what the OP is asking. Any future edits, you can apply directly to your answer. The point of this is not to satisfy "me" in particular, it is to have a meaningful set of illuminating answers to a question $\endgroup$ – A_A Oct 23 '17 at 8:58
  • $\begingroup$ Best answer is here (using Griffin-Lim) in case you don't have original FFT information. timsainb.github.io/… $\endgroup$ – Artemi Krymski Nov 22 '18 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.