0
$\begingroup$

I am from an ML background and I am working on MP3 audio reconstruction and I have managed some amount of spectrogram reconstruction but I have noticed that the higher frequency end of the generated Power Spectral Density plot(Wave2) isn't matching with the original audio's plot (Wave1).

There is a similar distribution in the magnitude spectrum which is obvious since the two are linked. I wanted to know if there is any way I could regenerate the higher frequency power (or magnitude) because I cannot map the input PSD data to output PSD data as there is no way to convert the PSD data back to an audio clip even if I am successful in mapping them. The real and imaginary part re-constructions for the spectrogram reconstruction are visually pretty accurate but I am still unsure about how to approach this issue. Any guidance would be great!

$\endgroup$
8
  • $\begingroup$ well, that's lossy compression to you: it doesn't contain all the original information. What's surprising about that? $\endgroup$ – Marcus Müller Dec 16 '20 at 13:05
  • $\begingroup$ And why would you reconstruct audio from the spectrogram instead of directly from the MP3? A spectrogram isn't uniquely mappable to audio (i.e. many very different pieces of audio have the same magnitude spectrum), whereas the MP3 data as is has a single optimum interpretation -> kick the spectrogram out of your signal processing chain, it loses info. $\endgroup$ – Marcus Müller Dec 16 '20 at 13:07
  • $\begingroup$ @MarcusMüller I was working with Spectrograms because I had to perform a spectrogram reconstruction as well (I initially thought just re-creating the higher frequencies would work but I was wrong). I know they are lossy but they were the best option I had. I managed to get the reconstructed spectrogram to match the original audio but the magnitude spectrum is still off. I was hoping that there could be a specific way of representing the audio data through which I could reconstruct the magnitude information too, so that the spectrogram changes would actually be a bit more humanly audible. $\endgroup$ – Darshan Deshpande Dec 16 '20 at 15:31
  • $\begingroup$ nope, you really want to go MP3->audio directly, I promise. $\endgroup$ – Marcus Müller Dec 16 '20 at 15:53
  • $\begingroup$ @MarcusMüller Could you explain a bit more specifically on what exactly "MP3->audio directly" means? $\endgroup$ – Darshan Deshpande Dec 16 '20 at 16:00
0
$\begingroup$

A few observations

  1. Your pictures looks odd. "Wave 1" doesn't look like an original audio file but something that has already been encoded/decoded and aggressively lowpass filtered in the process. Your reconstructed audio (Wave 2) actually as MORE high frequency energy, which I suspect is simply just quantization noise from the other bands. The expected spectrum of music is roughly pink (3dB/octave) up to about 10 kHz and then a steeper decline (maybe 10dB octave) to 20 kHz and above. There shouldn't be a step function in the spectrum.
  2. The standard method of recovering for lost high frequency spectrum is Spectral Replication. That's been already implemented in some of the more modern codec standards (e.g. AAC) although not in MP3. See for example http://www.stockis.se/AES112th_SBR.pdf
  3. If you want to do any commercial work in this area, I highly recommend studying the patent situation carefully. A massive amount of work has already been done in this field and since the license revenue is quite substantial, the work has been well protected by patents and there is incentive to enforce them.
$\endgroup$
1
  • $\begingroup$ Yes the Wave 1 is also an MP3 file so it has been passed through an encoder beforehand. Thanks for the link to the paper but I have managed to recover most of the frequency spectrum through a spectrogram but turns out that only spectrogram reconstruction isn't enough to make it sound like the original sample. Magnitude and power spectrums are the only spectrums not matching (phase matches satisfactorily) and since both are correlated, I figured they are the key to the audio actually sounding similar to the original. Any idea how I can work on the magnitude or power spectrum specifically? $\endgroup$ – Darshan Deshpande Dec 16 '20 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.