I have two audio files in which a sentence is read (like singing a song) by two different people. So they have different lengths. They are just vocal, no instrument in it.

A1: Audio File 1
A2: Audio File 2
Sample sentence : "Lorem ipsum dolor sit amet, ..."

structure of sample audio files

I know the time (in decimal seconds) every word starts and ends in A1. And I need to find automatically that what time every word starts and ends in A2. So, how to find sound of a word (in A1) in another audio (A2) that has different duration (of word) and different voice?

  • $\begingroup$ In the example you provide, it is easy to identify the parts just by looking at them. Are all your comparisons this easy to do by eye? $\endgroup$ – Cedron Dawg Mar 25 '18 at 23:26
  • $\begingroup$ @CedronDawg There are thousands of them. I need, actually, some programmatic way, preferably Python or C#, or any algorithmic way that I can follow. $\endgroup$ – Kadir Şahbaz Mar 25 '18 at 23:49
  • $\begingroup$ What I was getting at is if all your samples are of the same phrase and you merely have to match the boundaries, or if you have to search for matches within much longer samples. As is, those two samples could be matched by their envelopes alone. $\endgroup$ – Cedron Dawg Mar 26 '18 at 0:05

You may consider using Dynamic Time Warping (DTW) for this. One approach which generally would work is as follows:

  1. Divide both the audios into smaller (analysis) windows with appropriate overlap as per the needed temporal resolution
  2. Transform the audio from time domain to cepstral. You may compute MFCCs.
  3. Copute DTW to analyse the warping

I believe that you don't really want a signal processing (mathematical) solution to your problem, instead the following programming approach could be a helpful too.

Assuming that the speakers in the A2 data set, do not alter the word ordering, then you know that each audio file in the second set has exactly the same number of words spoken by different people.

Then you should look for a decent speech-recognition (speech to text) library that has at least the capability of isolating spoken words from each other, and indicating the beginnig and edning timings of each isolated word. You don't need to know what is being said, all you need is their isolation from each other.

Then having the output of such an isolated set, you will map one by one every word in the reference audio to the corersponding isolated word in the test set.

Depending on the isolation algorithm sucess rate, you mey end in wrong results though. No algorithm can guarantee error free isolation anyways.

If you need a more robust approach, of course you can look for full recognition library with the spoken word as a text output. Then you will now better.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.