how can I analize music as spotify/musixmath/shazam does?

Question

I mean, does exists some state of the art algorithm able to extract music features, classify them and identify similar music or find lyrics?

I know that there are many topics or steps involved on my question, like digital signal processing, feature extraction, machine learning, pattern recognition, etc. I'm a computer science student and I would like to know if there is a methodology to identify music as it does musixmatc. For example, this program is able to find the lyrics of a song from a very short audio sample, even with some noise. As I know, [1] is the unique related paper that is available.

1 An Industrial-Strength Audio Search Algorithm

Answer 1

The general idea for the algorithm is well presented in the linked paper. As is the case with applied machine learning and related fields, the secret sauce (which features to select, how to tweak your algorithms, etc.) is found by trial and error by people who know things about audio/audio signal processing (see for example, Julius O. Smith III's page at CCRMA @ Stanford) and by just trying a ton of things. Unfortunately, to become (and stay) the state of the art, the secret sauce has to be kept, well, secret.

That being said, the wikipedia article on acoustic fingerprinting is a decent place to look for some alternatives (and some links to some open source ones like Acoustid).

Answer 2

The paper "An Industrial-Strength Audio Search Algorithm" is really a nice example of how extract audio fingerprints, the proposed algorithm works extremely well, you can realy hit audio with a lot of noise because you are always looking for the spectrogram peak and it can survive better in noisy conditions!

Of course this paper show the BASIC idea behind shazam, there are many unrevealed secrets, but I can tell you that the BASIC works well because I did the steps described in this paper, and I can match a lot of songs in noise conditions, using samples of about 10 seconds was enough to get good results in my tests, for audio sampled at 44100hz I just used a window size of 4096 samples overlapping in 50% and I can match altered sounds with echo, reverb, WahWah, etc.

Another nice algorithm is an proposed by Philips A Highly Robust Audio Fingerprinting System, This algorithm show you how build fingerprints at every 3 seconds of sound, this mean that maybe you need just 3 seconds of sample to match one song, the paper show you how split the signal in 33 log spaced Frequency band and compute the Energy Difference from each Band to extract a 32-bit sub-fingerprint. In my tests got results similar to shazam, works especially if you want to have hits into small samples like 3 seconds.

Answer 3

you could explore the possibilities of the essentia C++ audio analysis library

see http://essentia.upf.edu/

this is not a complete solution to your problem, but rather provides some of the building blocks you'd need to build a solution to your problem