Why can't music recognition services like Shazam or SoundHound recognize music if its speed is changed by more than 5 percent?

Music services like Shazam and SoundHound take an audio sample from your microphone and compare it to its database to recognize music. This algorithm works very well under high noise, unequal frequency bins, and even a simple hum or whistle to the music. However, if you change the song's pitch or speed by more than 5%, the algorithm cannot detect the song at all. A 5% change is very small and not even that noticeable, yet it completely fools the music detection algorithm. Why is this? Is it due to the different speed/pitch missing the targets in the audio fingerprint?

• "Very small and not even noticeable" on the contrary, I'd say it's very large and utterly noticeable. A 5% off-pitch song sounds wrong. And to answer your question: quite obviously, both temporal and frequency features of the song are used for detection, so you can't mess with these. Commented Jun 19, 2017 at 6:07
• Using an A tone of $440Hz$,a 5% difference in pitch means $\pm 22Hz$.Let's go with the positive sign,so 462Hz.The ratio between two successive tones(A,A#) is ~1.05946309.This means that if A=440Hz then A#=~466.16376151.Therefore,5% of difference in pitch is 462/466.16376151~99.1% of a semitone.A considerable difference. You can assume that pitch alteration will act uniformly and linearly, therefore resulting in the same pointcloud of features but slightly contracted (or dilated) to further report that it is song XYZ, altered by 5% in pitch.
– A_A
Commented Jun 19, 2017 at 10:36
• A 5% increase from A440 would be 0.84 semitones (84% of a semitone) since (462−440)÷(466−440)=0.84 Commented Jun 19, 2017 at 13:00
• Most people like to listen to music with a relative velocity of less than 35mph to the band
– user28715
Commented Jun 19, 2017 at 21:01
• These programs all use signal processing tools, and one of the easiest (and most important) aspects to isolate for most songs is probably the beat - by its nature it is one of the most detectable aspects of a song, and can be detected almost regardless of background noise (within reason). Further, most songs stay at one beat, and it is a big differentiator because there is such a wide variety of beats. And, finally, it is rare that anyone would want to identify a song that is at the wrong pace. Maybe a live band plays it faster for fun - but then the audience knows the song already. Commented Jul 31, 2023 at 19:42

this is a guess from someone who does audio, but has never ever done recognition nor watermarking algorithms for someone in the industry (but i have a little idea how they may be done).

my guess is that they extract some features from the STFT of the music. associated with each feature is an envelope that represents the quantity of that feature at the center window time of the STFT.

the collection of those envelopes can be thought of as a point in a metric space. if you have a metric space, you can define a difference function

$$\| \mathbf{X} - \mathbf{X}_n \|$$

between those of any pair.

then they compare the test envelope, $\mathbf{X}$, with each template, $\mathbf{X}_n$, of a set that come from the original targets (the copyrighted songs they're fishing for). there is a norm taken of the difference between these points in a metric space, between the collection of envelopes taken from the test sound and the template. if the norm is below a given threshold and if the norm is the minimum compared to all of the templates, then the song of that best matching template is identified as matching the input or test sound.

now if the pitch is wrong it won't match as good. but they could make their difference algorithm insensitive to a constant offset to pitch. it's like running the difference signal through a DC blocking filter before calculating the norm of it. but they would have to do that, otherwise a constant pitch difference between test tune and target will always be an error.

and they could match a tempo difference, but that is much harder. if memory and computational effort is not a problem (and they often are not for non-realtime applications running on a GHz-speed computer or server) then there could be a template for each of many speeds. and see which one matches best. but if these other tempo templates do not exist, and you submit a tune that does not exactly match a template of it at the original speed, then at some point in the tune or song, the pitches will not match at all. that will make a dumb algorithm think they're different tunes.

The actual classifiers used in these music services are proprietary intellectual property and anyone who knows the exact features is not likely to divulge that information.

Classifiers depend on features that are separable. The obvious answer to your question is the features that are used are sensitive to frequency shift.

I also suspect that there isn't a business case for Doppler shifted music, so this is probably not a problem that requires a solution.

I've used Sound hound and am very impressed with it.

I recall that back in the early 1990's that the music publishers was trying to automate song recognition for the purpose of calculating royalties of songs played on radio stations. This is an area that has some maturity.

• You mean SoundHound, not Sound Dog?
– jojeck
Commented Jun 22, 2017 at 19:56