generating spectrograms in python with less noise

Question

I'm trying to do something similar to Shazam (paper here) in Python. Basically the idea is to use FFT with sliding window to transform both WAV files and recorded audio to the same spectral representations for recognition purposes.

The only problem is the spectrograms I extract from certain songs just seem to generate lots of useless peaks that become false positives when heard in other songs. They look a lot like noise. Examples below.

Spectrogram of SONG_1 [channel 1]:

Spectrogram of SONG_2 (dominates and generates WAY more peaks) [channel 1]:

Spectrogram of SONG_3 [channel 1]:

Spectrogram of SONG_1, recorded through my laptop mic (looks like noise) [channel 1]:

Here's the code I use to generate the spectrograms:

import pylab
from scipy.io import wavfile

fs, frames = wavfile.read("song1.wav")

channels = [
    np.array(frames[:, 0]),
    np.array(frames[:, 1])
]

# generate specgram
Pxx, freqs, t, plot = pylab.specgram(
    channels[0],
    NFFT=4096, 
    Fs=44100, 
    detrend=pylab.detrend_none,
    window=pylab.window_hanning,
    noverlap=int(4096 * 0.5))

I've implemented the hashing, storage, recognition, etc as well but they are rather useless unless the extracted peaks are distinct to the song heard.

I should mention my peak extraction algorithm is not just a threshold - I find points in Pxx that are the highest value in a neighborhood of around 15-20 adjacent points.

What am I doing wrong? How can I get more distinctive spectrograms out of audio sample channels?

Answer 1

None of these spectrograms show music. I would validate your algorithms by substituting a known signal, e.g. a 440 Hz pure tone. That should be a single line in your spectrogram. Audacity can generate that as a .wav file, but you can also numerically generate the signal in Python. Next, mix two known tones and check if your spectrogram is lineair (i.e. showing two lines).

Audacity can also show you what the spectrogram should be for your inputs.

Answer 2

I think there's an underlying issue with how you're generating the spectogram. From the looks of things you are FFTing data that has a disconinuity, or jump, resulting in large amounts of broadband noise. It's subtle in SONG_1, but very prominant in SONG_2/3. This would be equivalent to either having a sequence of zeros somewhere in your data. Something like fft([data (20 zeros) data]). I know it's a lot of data to plot (44.1KHz), but you can either scroll through the raw audio and look for patches of zeros or compute derivative of the audio, plot, and look for peaks. This is equivalent to y[n] = x[n] - x[n-1] and plot y[n]. That is also a pretty cheap lowpass filter, but I digress.

Another easy way to test this would be to record about 10 seconds of audio of you saying something like "Test one two three" etc. When plotting the spectogram (depending on your window size and overlap) you should see somewhat discrete areas of frequency content. Here's a pretty good image of what the sample of your voice should look like. If you again see broadband spectral leakage, that just so happens to look periodic, something else is going on.

Answer 3

There are several things you can do to alleviate or characterize the noise you see, both in generating the spectrogram and in cleaning it up afterward. Changing your widowing function and size, adding multitapers, various normalizations, etc., might help uncover or mitigate the saturation and noise you see. Can you share the files you used to generate the spectrograms for us to play with?

Answer 4

It seems like your data file is corrupt. Can you post that somewhere?

This script:

import numpy as np
f = 10.
w = 2. * np.pi * f
time_interval = 100
samples = 5000
t = np.linspace(0, time_interval, samples)
y1 = np.sin(w * t )
y2 = np.sin(2. * w * t)


y1c = np.array(y1[:])
y2c = np.array(y2[:])

yc = np.concatenate(y1c,y2c)


Pxx, freqs, t, plot = pylab.specgram(
    yc,
    NFFT=128, 
    Fs=44100, 
    detrend=pylab.detrend_none,
    window=pylab.window_hanning,
    noverlap=int(128 * 0.5))

show()

generates this plot:

which is doing the right thing, as far as I can see.

Answer 5

Once you get the spectrogram part working, I would recommend using scipy.signal.find_peaks_cwt for the peak finding; its pretty good. It uses user defined thresholds for the SNR and a wavelet transform and ridge tracking to distinguish real peaks from noise.

You do have to give it a range for the peak widths you expect, but that's not too much of a hassle. For standard music with a 4:4 beat you can just pick the SFT time to match the beat.