I am trying to solve the following problem: I have two audio tracks of music (modern pop-like music) which I cut into multiple slices whereas each slice is being cut at a note onset event of the track.

I am doing this by using the onset-feature extraction methods of the librosa python library. This function returns a list of onset-events represented as time-stamps. Each time-stamp is used as a "cut-mark" to slice the audio tracks.

Now I want to be able to pick a random slice and compare it to all the other slices in terms of how similar the other slices are to the picked one. I am thinking of using one of the other feature extraction methods of the librosa library.

After reading about musical similarity I think that using the MFCC-Module to get features which can be used to compare both tracks is a good start. But currently I am stuck and have the following problems.

1. When it comes to musical similarity I am not quite sure what kind of analyzation I should pick. What I am after is how similar the slice "sounds" to one of the others. The librosa library features different feature extraction methods just like the MFCC-Module as well as Chromatograms, etc.

2. What kind of "comparison-function" can be used to determine similarity? So far I have been reading about cosine-similarity, which is built in into the numpy python library.

3. Using cosine-similarity this would mean that both data source should be of the same length but to the nature of music and note onsets each slice has a different length!

For a large amount of musical audio tracks I am doing the following with every single track: 

I am slicing the track into very small pieces whereas each piece represents a note onset event. So the piece begins at a note onset event and ends before the next event starts. The result is having hundreds of very small, typically sub-2-second slices for each track.

Now I want to take one of these slices - a totally random pick - and search through all the remaining slices to find the slice that is most similar to the picked one.

Most similar means that it "sounds" like the picked one. More specific this means that it is having the same key, pitch - not timbre and not rhythmically (since rhythm is cut off by slicing at note onsets)

The Problems

To find the slice that is most similar to / that most "sounds-like" a randomly chosen slice I have to determine its "sound". I have been reading about MFCCs but I am not sure if this will help determine the sound, which in this case will be key, pitch.

Next, when having a way to determine the "sound" in numerical way I need to find a way how to compare these results. There are things like euclidian distance or consine-similarity.

Last but not least the slices are of different length.

What I have done so far

Slicing based on note onset-events is done with the help of the onset-feature extraction methods of the librosa python library. This function returns a list of onset-events represented as time-stamps. Each time-stamp is used as a "cut-mark" to slice the audio tracks.

I have been playing with the fastDTW python library which does a dynamic time warping analyzation, compares the result with using an euclidian distance function and fed it with slices. But I am not sure if the result is the distance in MFCCs

The question

To summarize the above into a single question:

How to match a piece of very short audio based on key and pitch to find a piece in a large database that is most similar to it?