# STFT to spectrogram

I would like to know whether I am correct in my understanding of going from STFT to a spectrogram. My goal is to convert a spectrogram back to a wav file.

If I have my STFT:

audio, _ = librosa.load(f, sr=Fs)
stft = librosa.stft(samples, n_fft=NFFT, hop_length=HOP_LENGTH, win_length=window_length_samples)


and I wish to display a spectogram, I have to do:

spec = librosa.specshow(np.abs(stft))


However, since I have taken the modulus, it must be impossible to go from spec back to audio correct? So does that mean that librosa.istft does NOT convert a spectrogram to a wav file? My confusion arises because I have seen many answers to "spec to wav" questions suggesting the use of librosa.istft.

I am wishing to use a visual representation of audio that can be fed to a convolutional neural network, but still be converted back into audio. As an extension to the question, does anyone have experience using the stft numpy array as the input to a CNN?

• if you don’t change your STFT results, it’s a lot easier to just keep a copy of the wave file and process it the way you want.
– user28715
Dec 11, 2019 at 14:30
• I am actually using generative adversarial networks to try and produce an stft array. So the reason I am wondering about the conversion between STFT and wav is because I’d like the network to generate unique, unseen stft arrays. Dec 12, 2019 at 8:27
• You are right in that it's impossible to get the time-domain signal from the spectrogram. (There are algorithms that aim at estimating it, e.g., the algorithm proposed by Griffin and Lim mentioned in an answer, but that's a different topic.) The reason for that is that all phase information is lost by taking the modulus. What you could try to do is to pass a second matrix, which contains the phase of the short-time DFT, to the CNN and see whether that brings you a step further. Apr 15, 2021 at 22:53

## 1 Answer

However, since I have taken the modulus, it must be impossible to go from spec back to audio correct?

You need to resort to some approximations like the Griffin-Lim algorithm or various vocoders (WaveNet,WaveGlow,etc.).

So does that mean that librosa.istft does NOT convert a spectrogram to a wav file?

It's a bit more complicated than that. An FFT ideally gives you the magnitude and phase spectra. If I'm not mistaken, what causes the confusion is that np.stft gives you an ndarray with a complex dtype, that is, it includes the magnitude and phase representation in one single ndarray. When you take the absolute value it tosses away the phase information, and that's an irreversible operation (even in the real domain np.abs would be irreversible). That's why you need a phase approximation scheme to get back to audio.

As an extension to the question, does anyone have experience using the stft numpy array as the input to a CNN?

You can do it. Actually, most CNNs use Mel spectrograms as their audio representation, so it is entirely possible. The best way to do it depends on your dataset, sadly.