16
$\begingroup$

I have been experimenting some weeks to find a way to match/find similar Songs in a Library containing different Genres of Music.

My first try was to Detect Features like Tempo or how much Bass there is in Songs to form groups, but I didnt get far with this approach (Volume Changes Based Beat Detection) since on about 20% of the Songs Beat dont have to be counted always, sometimes 1/2 or 1/3 of them and I couldnt implement that.

After some weeks of failed trying I got a new idea which is described later in this Post. Simply put it works by taking Spectrum Samples of Files, making something like an "Average Spectrum" of Files to compare them. The Idea behind was that for example Hardstyle has a lot more Bass than average Rock Music, I also verified this by looking at some Spectrums in Audacity.

  • File 1: Take full File FFT Spectrums (2048 Sample Size atm,Amplitudes Log. scaled)
  • Sum all Spectrum Arrays, take Averages of each Bin
  • Do the same some other Files, store all Results
  • Make List of FFT Values Differencies between File 1 and other Files
  • Make Average of Differencies between File 1 and File X
  • Sort Ascending by these Averages
  • Songs with the Lowest "Difference Value" are considered to be Similar.

Can some of you who have good knowledge tell me if this would be the right/good way to implement my Idea?

$\endgroup$

migrated from math.stackexchange.com Feb 7 '12 at 16:28

This question came from our site for people studying math at any level and professionals in related fields.

  • 1
    $\begingroup$ If you're trying to detect tempo, you might want to try squaring the signal and then taking a Fourier transform. Frequencies on the order of 1 Hz will not be present in an ordinary (unsquared) FFT spectrum, because they're filtered out. A related idea, used for pitch detection, is called the "cepstrum;" you can find out about it by googling. To differentiate pop and jazz from classical, you could try detecting the sounds of a drum kit, which are unpitched. Vibrato should be machine-detectable. There are measures of dissonance that can be machine-computed. $\endgroup$ – Ben Crowell Feb 7 '12 at 1:52
  • 1
    $\begingroup$ Maybe you should ask the moderators to move this to dsp.SE $\endgroup$ – Dilip Sarwate Feb 7 '12 at 1:55
  • $\begingroup$ I flagged my question with request to move it to DSP of SE. You mean I could detect if there is a Drumkit Present or Not to classify the Input? Can you explain how the squared signal leads to Tempo? $\endgroup$ – gfg Feb 7 '12 at 14:13
  • 3
    $\begingroup$ Music is recorded and mastered in such ways to maximize their spectral spread, especially these days. I don't think that full length spectra will give you a good criterion for classifying music. $\endgroup$ – Phonon Feb 7 '12 at 16:47
  • 2
    $\begingroup$ Instead of spectrums, you should be looking at spectrograms. Spectrums only show you the frequency content of the entire song at once. Spectrograms show how the frequency content changes over time. $\endgroup$ – endolith Mar 26 '12 at 17:45
17
$\begingroup$

What you are trying to do has been tried over and over by hundreds of researchers and there is quite a large body of work about this. Check the proceedings of the ISMIR conference. Even if it is not up to date, read Elias Pampalk's thesis : http://www.ofai.at/~elias.pampalk/publications/pampalk06thesis.pdf

To quickly orient you on the right track:

Music can be similar according to many dimensions: a) timbre/texture/genre ; b) rhythmic pattern ; c) melody/chord progression... and more! From your message it's not clear what you want to measure!

  • If you are interested in a) the features you might want to look at are MFCC (Mel Frequency Cepstrum Coefficients), since they somehow capture the way human hearing works (frequency warping, log scale), since they are decorrelated (making modeling easier), and since they have lower dimensionality (13 coefficients vs 2048).
  • If you are interested in b), look at a feature called "Fluctuation Patterns" (Pampalk, in short autocorrelation of the signal in the 0.1 .. 10 Hz range over a few bands) ; or Whitman's "Penny" features (FFT of the MFCC along the time axis).
  • If you are interested in c), look at chromagrams. Start with Ellis' chromagram code (http://labrosa.ee.columbia.edu/matlab/chroma-ansyn/) then move up to Mauch's implementation if you need something more robust (http://isophonics.net/nnls-chroma).

That's for the features. Now you'll have to think of a better way to compare your songs once they have been represented as a sequence of those features. Computing pairwise differences between sequences is not very smart - eg: comparing a song and the same song offset by some silence will yield a difference while it is exactly the same! You'd rather compare the distribution of those features ; for example compute the mean / standard deviation of the features over song A and the mean / standard deviation of the features over song B and then take a probabilistic distance (KL, Bhattacharyya over those).

Last point, but which will matter later: computing the distance between a song and the rest of the corpus to find the nearest matches is quite inefficient. When dealing with large collections, techniques like LSH or Ball trees allow such nearest neighbors queries to be performed without explicit comparison with the whole corpus.

As an aside, tempo detection is an entirely different matter. If you want to look into it, best performance / accessibility paper on the topic is Ellis' Beat Tracking by Dynamic Programming. http://www.ee.columbia.edu/~dpwe/pubs/Ellis07-beattrack.pdf . It's incredibly simple but is close to state of the art algorithms.

$\endgroup$
  • $\begingroup$ Thanks for your detailed Answer, I already heard of MFCC multiple times in this context and it seems reasonable not to use bare FFT Results. It seems pretty complex to implement with my current "state of knowledge" and development environment (C#,Bass Library's FFT Results) but I will try. $\endgroup$ – gfg Feb 9 '12 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.