I'm trying to synchronize (overlap) two audio recordings of the same song: a complete HQ version, and an incomplete, noisy version (phone recording for example). The noisy recording may have had its tempo increased/decreased.
Currently I have something working based on this 2001 paper: "The Beat Spectrum: a New Approach To Rhythm Analysis"
In short:
- Get a self-similarity vector for each song, which will contain peaks at every multiple of the song bar duration (every 4 or 8 beats for latin music for example)
- For each r (relative tempo ratio) from 0.7 to 1.3, scale the second vector and then do a dot product on the two vectors, using v1[i]·v2[i] = d1[i]*d2[i] (where d1[i] is the sign of v1's derivative; same for d2)
When the beats are clear in the recordings, I get a clear maximum at the correct ratio (usually 1.0). The problem is that when the beats are lost (due to EQ, bad mics, noise) the self-similarity chart stops showing the periodic bars and I get maximums in the wrong places in the dot product.
My question is, what else could I use instead of the dot-product-of-the-derivative-sign, and what other robust ways are there to determine the tempo change ratio in the presence of noise, besides doing BPM(song1) / BPM(song2).
In case someone can take this further: computing self-similarity by overlapping the sample twice (at [0, ofs, ofs+ofs] instead of just [0, ofs]) gives much more accurate maximums for each beat duration. But I haven't tested it yet on all the samples I have.
