I need to compare two audio tracks and compute the difference. The difference is how close they are in terms of pronunciation, rhythm etc. ignoring speaker voice characteristics.

The first track is the pattern, a word or a phrase spoken by a person. The second is a try to mimic the first track by another person.

I imagine this as:

  1. detect words in track #1 and #2 and pair them. I don't know how to do this properly.
  2. scale the second word in pairs to the duration of the first word. This one is simple.
  3. compare words in pairs somehow. Can two different voices be compared by simple subtraction in time domain? Should I use FFT first?

I think about using C# and NAudio library for audio coding and FFT. Any better alternatives?

And the most important question is: how difficult the task is? Can I expect any reasonable precision?

The question is quite broad, sorry. I've never worked with speech recognition or signal analysis before. Any push in the right direction is highly appreciated.

  • 2
    $\begingroup$ Have you read some of the many similar questions posted on this site? $\endgroup$ – MBaz Dec 13 '16 at 0:06
  • $\begingroup$ @MBaz maybe I searched by wrong keywords $\endgroup$ – Andriy Tylychko Dec 13 '16 at 0:15
  • $\begingroup$ This is a very difficult task. $\endgroup$ – CMDoolittle Dec 13 '16 at 0:29
  • 3
    $\begingroup$ On the "related" bar on the right I see four or five questions that could be relevant... you don't even have to search! :) $\endgroup$ – MBaz Dec 13 '16 at 1:39
  • $\begingroup$ @MBaz: I do think some of them are only indirectly related to what I'm asking. IMO the main difference is that the second person tries to imitate the first one, not just saying same words. Again, I'm not experienced so if you think it's a duplicate question please flag accordingly, even this would be helpful for me. $\endgroup$ – Andriy Tylychko Dec 13 '16 at 10:12

You should think about what level it makes sense to "compare".

You can look at your signals as time-series and compute statistics (e.g. cross-correlation, phase-correlation, etc). Alternatively, you can compute "features" such as MFCCs or other spectral features and compute differences between those features.

Finally, as you've kind of suggested, you can feed your signals through a speech-recognition pipeline (e.g. CMU Sphinx or Kaldi toolkits) to get back text transcripts. You can then compute differences between the texts transcripts to obtain word alignments and word-error-rates.

As the commentators have indicated this is a broad problem that can be looked at many levels of abstraction. You should narrow your scope a bit and read up on the "Similar Questions". Good luck!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.