How to match a piece of very short audio based on key and pitch to find a piece in a large database that is most similar to it?

Question

For a large amount of musical audio tracks I am doing the following with every single track:

I am slicing the track into very small pieces whereas each piece represents a note onset event. So the piece begins at a note onset event and ends before the next event starts. The result is having hundreds of very small, typically sub-2-second slices for each track.

Now I want to take one of these slices - a totally random pick - and search through all the remaining slices to find the slice that is most similar to the picked one.

Most similar means that it "sounds" like the picked one. More specific this means that it is having the same key, pitch - not timbre and not rhythmically (since rhythm is cut off by slicing at note onsets)

The Problems

To find the slice that is most similar to / that most "sounds-like" a randomly chosen slice I have to determine its "sound". I have been reading about MFCCs but I am not sure if this will help determine the sound, which in this case will be key, pitch.

Next, when having a way to determine the "sound" in numerical way I need to find a way how to compare these results. There are things like euclidian distance or consine-similarity.

Last but not least the slices are of different length.

What I have done so far

Slicing based on note onset-events is done with the help of the onset-feature extraction methods of the librosa python library. This function returns a list of onset-events represented as time-stamps. Each time-stamp is used as a "cut-mark" to slice the audio tracks.

I have been playing with the fastDTW python library which does a dynamic time warping analyzation, compares the result with using an euclidian distance function and fed it with slices. But I am not sure if the result is the distance in MFCCs

The question

To summarize the above into a single question:

How to match a piece of very short audio based on key and pitch to find a piece in a large database that is most similar to it?

Answer 1

well, the "sound" of something depends on a lot of different parameters, which are not all well understood, nor all innumerated.

could be

1. loudness contour.
2. brightness contour.
3. pitch contour.
4. degree of periodicity vs. inharmonicity.
5. more parameters that might affect this nebulous thing we call "timbre" (like formant locations or even/odd harmonic ratio).

good luck.

Answer 2

The usual first step in correcting for different lengths is to apply the dynamic time warping algorithm.

I've never used the implementation in the mlpy Python library, but it seems to be implementing the right thing.

The neat thing about this algorithm is that it is distinct from the comparison algorithm you decide to use. It just tries to make up for the difference in lengths and timings of events in the two signals.