I have two audio files that have mostly identical* content, most of which is just people talking. A typical example would be two files, each approximately an hour long, in which ~55 minutes of the total ~60 minutes are the same content (2-4 people talking), with the other 5 minutes of different content interspersed throughout.
I'm trying to pinpoint all the relevant offsets of the two files where the identical content lines up. So it should output something like this:
|File 1 Start||File 2 Start||Duration|
I started with effectively zero knowledge of how digital audio works and have spent the past week or more reading just about every DSP post, blog post, etc. I could find on this topic.
Now I know how to do a Fast Fourier Transform and wrote a Ruby script (after a bit of trial and error, I'm currently using audio sample rates of 11025 Hz and window sizes of 1024 samples each) to compare the two files' FFT output arrays (using the peak frequency from each bin as the array elements to compare) to try to find the offsets where the two audio files' contents match each other.
So far, the script works just well enough to indicate I'm on the right track, but nowhere near production-level performance. For example, it will find something like 30 seconds in common between the two one-hour files in total, even though in reality it should be more like 55 minutes. (It also will find 1-2 dozen other portions in common, but they're all about 1 second in duration or less, so I suspect it's just pauses between people speaking.) When I check the offsets for that 30 seconds, they do line up. The problem is just that it's not finding the vast majority of the content that's identical.
So I'm feeling a bit stuck. It feels like I must be missing something obvious, as my problem is significantly less complicated than, say, the Shazam use case. I can't figure out why my peak frequencies aren't lining up better.
- Should I be using overlaps between my sample windows?
- Do I need to use some kind of windowing function and, if so, which one?
- Is there some additional calculation I should be applying to the FFT output (other than / in addition to
sqrt(real**2 + imaginary**2)that's better optimized for human voices?
- Should I be capturing (and then comparing) the top, say, 2 or 3 peak frequencies per window rather than just the top 1?
Any pointers at all would be helpful. Thank you!
* I'm using "identical" in layman's terms here, as I don't know if the files necessarily match up perfectly, e.g. individual bytes of the files being exactly the same. The point is any differences in the audio are completely imperceptible to humans: there's no background noise in one vs. the other, etc. In other words, this is not a Shazam type use case where a fuzzy recording from a bar is being matched against a real music file.