3
$\begingroup$

I'm trying to do something similar to Shazam (paper here) in Python. Basically the idea is to use FFT with sliding window to transform both WAV files and recorded audio to the same spectral representations for recognition purposes.

The only problem is the spectrograms I extract from certain songs just seem to generate lots of useless peaks that become false positives when heard in other songs. They look a lot like noise. Examples below.

Spectrogram of SONG_1 [channel 1]:

song 1

Spectrogram of SONG_2 (dominates and generates WAY more peaks) [channel 1]:

song 2

Spectrogram of SONG_3 [channel 1]:

song 3

Spectrogram of SONG_1, recorded through my laptop mic (looks like noise) [channel 1]:

recorded song 1

Here's the code I use to generate the spectrograms:

import pylab
from scipy.io import wavfile

fs, frames = wavfile.read("song1.wav")

channels = [
    np.array(frames[:, 0]),
    np.array(frames[:, 1])
]

# generate specgram
Pxx, freqs, t, plot = pylab.specgram(
    channels[0],
    NFFT=4096, 
    Fs=44100, 
    detrend=pylab.detrend_none,
    window=pylab.window_hanning,
    noverlap=int(4096 * 0.5))

I've implemented the hashing, storage, recognition, etc as well but they are rather useless unless the extracted peaks are distinct to the song heard.

I should mention my peak extraction algorithm is not just a threshold - I find points in Pxx that are the highest value in a neighborhood of around 15-20 adjacent points.

What am I doing wrong? How can I get more distinctive spectrograms out of audio sample channels?

$\endgroup$
  • $\begingroup$ What do you mean by a lot of useless frequencies? The intensity of the spectrograms seem to be around the Lower frequencies which is reasonable enough I think. I'm assuming you need to look for signatures across particular frequencies for this particular application. One more thing is, Laptop Microphones are not too good generally, also you need to know how linear the Microphone's transfer function is if you're looking for high-frequency signatures to be able to use thresholding across those frequencies. The last spectrogram seems is completely off; probably your microphone recorded nothing. $\endgroup$ – Sudarsan Sep 16 '13 at 4:01
  • $\begingroup$ I don't think that the laptop mic is the problem - shazam does it on cell phone mics just fine. $\endgroup$ – lollercoaster Sep 16 '13 at 4:04
  • $\begingroup$ I just mentioned it because if you want to peak detect, you need to use different thresholds for different frequencies depending upon the Frequency response of the microphone. You can try seeing the Power Spectral density of a reasonable number of samples of wav files before coming to a conclusion. Also Peak detection is not trivial, you might want to take a look at available literature depending upon the computational complexity you can afford. $\endgroup$ – Sudarsan Sep 16 '13 at 4:07
  • $\begingroup$ I have no idea what you just said - I have no DSP experience at all. I think you are saying that certain microphones tend to overamp certain ranges of frequencies. But I don't think peak detection in the Pxx 2d array (amplitude as function of freq bin and time window) I have above is hard - just finding peaks using scipy ndimage package can find peaks in arbitrarily sized neighboorhoods. $\endgroup$ – lollercoaster Sep 16 '13 at 4:13
  • 1
    $\begingroup$ I don't know python, but your spectrograms look way off to me. $\endgroup$ – Bjorn Roche Sep 16 '13 at 16:12
4
$\begingroup$

None of these spectrograms show music. I would validate your algorithms by substituting a known signal, e.g. a 440 Hz pure tone. That should be a single line in your spectrogram. Audacity can generate that as a .wav file, but you can also numerically generate the signal in Python. Next, mix two known tones and check if your spectrogram is lineair (i.e. showing two lines).

Audacity can also show you what the spectrogram should be for your inputs.

$\endgroup$
  • 1
    $\begingroup$ None of these spectrograms show music: Actually, the first and third appear quite normal to me, like lossily compressed audio where almost everything above 16 kHz is filtered out. The second has massive clipping, and the fourth is simply (loud) white noise. $\endgroup$ – chirlu Sep 17 '13 at 14:43
  • $\begingroup$ Might be a lack of detail, but I'd expect more variation and less structure. The third one is a bit of a toss-up, but that seems like a mix of a 16 kHz-filtered and an 18 kHz-filtered sound. $\endgroup$ – MSalters Sep 17 '13 at 14:53
  • 1
    $\begingroup$ This is common for variable bitrate lossy compression. $\endgroup$ – chirlu Sep 17 '13 at 15:20
  • $\begingroup$ SONG_1 and SONG_3 look fine to me. SONG_2 is probably recorded too hot so that it's clipping, laptop recording is either heavily clipping or being read in the wrong format? Look at the time domain waveforms before doing the spectrogram. $\endgroup$ – endolith Sep 18 '13 at 20:54
2
$\begingroup$

I think there's an underlying issue with how you're generating the spectogram. From the looks of things you are FFTing data that has a disconinuity, or jump, resulting in large amounts of broadband noise. It's subtle in SONG_1, but very prominant in SONG_2/3. This would be equivalent to either having a sequence of zeros somewhere in your data. Something like fft([data (20 zeros) data]). I know it's a lot of data to plot (44.1KHz), but you can either scroll through the raw audio and look for patches of zeros or compute derivative of the audio, plot, and look for peaks. This is equivalent to y[n] = x[n] - x[n-1] and plot y[n]. That is also a pretty cheap lowpass filter, but I digress.

Another easy way to test this would be to record about 10 seconds of audio of you saying something like "Test one two three" etc. When plotting the spectogram (depending on your window size and overlap) you should see somewhat discrete areas of frequency content. Here's a pretty good image of what the sample of your voice should look like. If you again see broadband spectral leakage, that just so happens to look periodic, something else is going on.

$\endgroup$
1
$\begingroup$

There are several things you can do to alleviate or characterize the noise you see, both in generating the spectrogram and in cleaning it up afterward. Changing your widowing function and size, adding multitapers, various normalizations, etc., might help uncover or mitigate the saturation and noise you see. Can you share the files you used to generate the spectrograms for us to play with?

$\endgroup$
1
$\begingroup$

It seems like your data file is corrupt. Can you post that somewhere?

This script:

import numpy as np
f = 10.
w = 2. * np.pi * f
time_interval = 100
samples = 5000
t = np.linspace(0, time_interval, samples)
y1 = np.sin(w * t )
y2 = np.sin(2. * w * t)


y1c = np.array(y1[:])
y2c = np.array(y2[:])

yc = np.concatenate(y1c,y2c)


Pxx, freqs, t, plot = pylab.specgram(
    yc,
    NFFT=128, 
    Fs=44100, 
    detrend=pylab.detrend_none,
    window=pylab.window_hanning,
    noverlap=int(128 * 0.5))

show()

generates this plot:

enter image description here

which is doing the right thing, as far as I can see.

$\endgroup$
1
$\begingroup$

Once you get the spectrogram part working, I would recommend using scipy.signal.find_peaks_cwt for the peak finding; its pretty good. It uses user defined thresholds for the SNR and a wavelet transform and ridge tracking to distinguish real peaks from noise.

You do have to give it a range for the peak widths you expect, but that's not too much of a hassle. For standard music with a 4:4 beat you can just pick the SFT time to match the beat.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.