I am developing an application that requires audio fingerprints. I have been reading a lot of articles and PDFs, now i think i have gotten myself confused. Based on my present understanding, I have some questions

  1. After decoding the audio to its raw format, resampling and extracting the mono channel. Do i convert the bytes to integers or floats?

  2. What is convolution and why is it necessary to convolve the audio samples

  3. What is a window and its length

  4. FFT transforms a signal from time domain to frequency domain. Am i correct? If yes, does the frequency components of an audio determine the contents/sound/noise/volume?

  5. What operation can be carried out on two audios of same content but different bitrate to normalize the bitrate and get the data that they share

  6. What are low pass and high pass filters and how are the derived. What are their uses in relation to audio fingerprint operations

  7. In the text Computer vision for music identification. After convolving the signals with a low pass filter and extracting every 8th sample. Then, a short term fourier transform with a window size of 2048 samples with successive windows offset by 64 samples. Also divide the power between 300Hz and 2000Hz in 33 logarithmically spaced bands.

    • Will these operations be applied to each of the samples retrieved after the convolution process? Can i get a simpler explanation?
  8. 32 learned filters and thresholds are applied to get a 32-bit descriptor for every time step (11.6 ms) of signal. This series of signals are known as the signature.

    • what are these learned filters and thresholds?

Wow :), that's a lot of questions. I am sure a lot of beginners will find the answers helpful.


  • 7
    $\begingroup$ Many of those are very basic questions. Audio fingerprinting is a pretty complex signal processing problem. You might be better off by trying to bite off a smaller problem instead. Also, you'll find better success if you focus a posted question on one specific problem. The above could be broken into a number of separate questions, and if you search this site, you might find answers for some of them already. $\endgroup$
    – Jason R
    Aug 22, 2012 at 3:46
  • 5
    $\begingroup$ Given that you apparently have no background in signal processing, it will be very hard to answer your questions in terms that you will understand, yet be precise enough to allow you to turn this into code. Maybe you could try looking into open-source solutions, eg. Acoustid Chromaprint or Echoprint for music. $\endgroup$ Aug 22, 2012 at 8:15
  • $\begingroup$ @pichenettes its true i have no background in signal processing and i have the source codes to Chromaprint and Echoprint. I am not only looking to get fingerprint of audio, I really want to know how its done. Answering any of the questions will help alot. I've being doing a lot of reading on signal processing $\endgroup$
    – Kennedy
    Aug 22, 2012 at 8:25
  • $\begingroup$ voted to close. you need to break this up into separate questions, some of which you'll find are already answered here. $\endgroup$
    – endolith
    Aug 22, 2012 at 15:52
  • $\begingroup$ Audible Magic has a multi-platform SDK available for not much money. developers.audiblemagic.com $\endgroup$
    – user3726
    Jan 31, 2013 at 22:06

1 Answer 1

  1. There are rare situations when processing is done with integers (embedded systems with no floating point units), but usually, audio signal processing is done with floats.

  2. (Handwaving) Convolution is the operation through which a filter is applied to a signal. If the authors of the paper you're looking at found that applying a specific filter to the data (for example to remove noise, any information not processed by the human auditory system, etc.) yielded good results, then you have to trust them on this!

  3. Since audio signals are not stationary, they are processed in short blocks, referred to as "windows" or "frames". The analysis/processing is thus done on short overlapping segments. For example you will process segment 0.00s:0.02s ; then segment 0.01s:0.03s ; then 0.02s:0.04s etc. In this example, the window size is 20ms and the overlap 50%. There is no "universal" window size for audio applications, look at what is used in the paper describing the technique you're implementing.

  4. Not really, but going from the time domain to frequency domain is "one step closer" to what is ultimately happening in the human auditory system.

  5. No operation could do that. If there was one, we would send all our files with the lowest quality and apply this magical operation to recover the highest quality. What this means is that the features used for fingerprinting must be robust enough to capture only the overall "shape" of the sound rather than focus on details - which would be lost by an increased amount of MP3 encoding.

  6. Look at a signal processing textbook. These are very basic signal processing operations and can be used for many things inside or outside the context of a fingerprinting system - reducing sample rate, smoothing temporal sequences, emphasizing details, removing noise...

  7. Let's say you have 800000 samples. You apply the convolution, you get 800000 samples out of it. Now you extract every 8th sample, you get 100000 samples. Then you apply your FFT on samples 0..2047 to get your 33 coefficients ; then on samples 64..2111 etc... so in the end you have a list of roughly 100000 / 64 = 1562 vectors of 33 coefficients.

  8. (Handwaving). I am assuming you are referring to the Ke et al system (later refined in the works of Baluja et al), which is itself inspired by the Viola-Jones face detector. Once you have converted the signal into a sequence of vectors of 33 coefficients, you compute weighted sums of the coefficients in adjacent bins or along the time axis (a filter), and you compare the result to a threshold. This gives you a binary "feature". There are 32 such binary features, each of them computed with a different filter and a different threshold. They are "learned", because they were obtained the following ways: the authors built a database with millions of pairs of songs, some of them identical (but maybe at difference bitrate, volume, noise level etc.), some of them different. Using machine learning techniques (boosting on decision stumps), they evaluated which filters were the best at binning together identical content and separating unrelated content, and which threshold were being used for that. You can find the list of those filters/thresholds in the source code of an open-source implementation of lastfm's Ke-style audio fingerprinter.

  • $\begingroup$ Thank you for taking your time to answer my questions. Now, i need to go read a good book $\endgroup$
    – Kennedy
    Aug 22, 2012 at 10:48
  • $\begingroup$ This was really well written with a balance of programming and dsp techniques. If you ever start a blog or write more about this knowledge please comment back here. Thanks :) $\endgroup$
    – some_id
    Jun 27, 2013 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.