For a large amount of musical audio tracks I am doing the following with every single track:

I am slicing the track into very small pieces whereas each piece represents a note onset event. So the piece begins at a note onset event and ends before the next event starts. The result is having hundreds of very small, typically sub-2-second slices for each track.

Now I want to take one of these slices - a totally random pick - and search through all the remaining slices to find the slice that is most similar to the picked one.

Most similar means that it "sounds" like the picked one. More specific this means that it is having the same key, pitch - not timbre and not rhythmically (since rhythm is cut off by slicing at note onsets)

The Problems

To find the slice that is most similar to / that most "sounds-like" a randomly chosen slice I have to determine its "sound". I have been reading about MFCCs but I am not sure if this will help determine the sound, which in this case will be key, pitch.

Next, when having a way to determine the "sound" in numerical way I need to find a way how to compare these results. There are things like euclidian distance or consine-similarity.

Last but not least the slices are of different length.

What I have done so far

Slicing based on note onset-events is done with the help of the onset-feature extraction methods of the librosa python library. This function returns a list of onset-events represented as time-stamps. Each time-stamp is used as a "cut-mark" to slice the audio tracks.

I have been playing with the fastDTW python library which does a dynamic time warping analyzation, compares the result with using an euclidian distance function and fed it with slices. But I am not sure if the result is the distance in MFCCs

The question

To summarize the above into a single question:

How to match a piece of very short audio based on key and pitch to find a piece in a large database that is most similar to it?

  • 1
    $\begingroup$ Welcome to SP.SE! Your response to rb-j's answer was very rude, so it's not clear to me what you're after. Please modify your question to make it clearer what answers / information you seek. $\endgroup$
    – Peter K.
    Aug 17 '16 at 19:58
  • $\begingroup$ Ok. That's not how it was meant so I try to describe my question better ... Sorry for this ... $\endgroup$ Aug 17 '16 at 19:59
  • $\begingroup$ @PeterK. I have edited my question and put it in another way. I hope this will help getting a better picture of what I am trying to do!? $\endgroup$ Aug 17 '16 at 21:09
  • $\begingroup$ Nicely done! That seems like a tall order, but let's see if someone has some suggestions for you. $\endgroup$
    – Peter K.
    Aug 17 '16 at 21:50
  • $\begingroup$ Question is ambiguous. Possibly even self contradictory. Maybe the reading of some textbooks on audiology and/or psycho-acoustics of music would help clarify the meaning of "similar" and "sounds like" to S. Kemper. $\endgroup$
    – hotpaw2
    Aug 18 '16 at 6:39

well, the "sound" of something depends on a lot of different parameters, which are not all well understood, nor all innumerated.

could be

  1. loudness contour.
  2. brightness contour.
  3. pitch contour.
  4. degree of periodicity vs. inharmonicity.
  5. more parameters that might affect this nebulous thing we call "timbre" (like formant locations or even/odd harmonic ratio).

good luck.

  • $\begingroup$ wow. Sounds a bit like you don't trust me that I will find the answer to that problem. Unfortunately your answer won't help me. There are people out there who have done this very well and most of them seem to use MFCC for it. So I was looking to get an answer of an expert here. $\endgroup$ Aug 17 '16 at 19:36
  • 3
    $\begingroup$ my apologies that my expertise is lacking. hadn't been thinking about "trust" at all in writing. good luck with the MFCC. lotsa jargon flying about. dunno if jargon will solve your problem. $\endgroup$ Aug 17 '16 at 19:50
  • $\begingroup$ No problem. I think I got it wrong! I have edited my question so you might be able to get a better picture of what I am trying to achieve. Thanks! $\endgroup$ Aug 17 '16 at 21:07

The usual first step in correcting for different lengths is to apply the dynamic time warping algorithm.

I've never used the implementation in the mlpy Python library, but it seems to be implementing the right thing.

The neat thing about this algorithm is that it is distinct from the comparison algorithm you decide to use. It just tries to make up for the difference in lengths and timings of events in the two signals.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.