1
$\begingroup$

I have computed the MFCCs for two audio files and applied the DTW to both sets of coeficients for both files. Now each such pair the two files is described by one single value (computed by the DTW). Let's call this value Diff.

If the audio files differ more, then Diff is greater, which is great. (so Diff is proportional to the difference between the audio signals - great!).

My problem is that Diff seems to be also proportional to the length of the files (?). Therefore, for two similar long audio signals, Diff might be greater than for two different short signals.

First I thought that, as DTW computes the shortest path for two arrays, the final value should be proportional to the lenght of the two arrays (i.e. the number of frames for each of the signals). So I tried dividing it by the product and by the average of the two lenghts but the results was discouraging (too big for some files that did not differ so much, too small for others).

Then I thought that if it is proportional to the lenght of the files, then dividing it to the average number of frames (in which I divided the two signals for computing the MFCCs) would do the trick... but it didn't...

Finally, I thought that, as the MFCCs are computed from the result of the FFT, the final DTW result might be proportional to the sum of the FFT magnitudes, so I tried dividing it by this sum (of the spectrum elements, returned by the FFT algorithm). Again, the results were not what I hoped them to be...

Right now I'm out of ideas... What is the DTW result depending on, what is it proportional to?

EDIT: Here are some values, after pichenettes' suggestion to try the geometric average of the lengths of the files:

  1. same sound . avgLengths = 106000. Diff = 409. Diff / avgLengths = 38

  2. same sound . avgLengths = 191000. Diff = 609. Diff / avgLengths = 31

  3. diff sounds. avgLengths = 80000. Diff = 437. Diff / avgLengths = 55

  4. diff sounds. avgLengths = 193000. Diff = 692. Diff / avgLengths = 35

38 > 35 :(

(the sounds from 2. and 4. are almost similar, and so are the ones from 1. and 3. In 1. the trailing silence for one file is a little longer)

Improve this question
$\endgroup$
2
  • $\begingroup$ Have you tried the geometric average of the length of the files you're comparing? Length in seconds or # of frames are just the same up to a constant. $\endgroup$
    – pichenettes
    Mar 28 '13 at 15:18
  • $\begingroup$ @pichenettes : I did now - please see the edit. $\endgroup$
    – Ioanna
    Mar 29 '13 at 9:11
3
$\begingroup$

DTW is not proportional to anything, including frame length. It's not a good idea to compare DTW values between two different pairs like you are doing.

The issue is that DTW is not just the number but also an alignment. If alignment does not match, you can not relate numbers. You only can relate numbers if you align to the same thing.

Normalization is often suggested, but mathematically it doesn't have much sense.

In classical tasks like speech recognition you never compare the DTW values between pairs, instead, you compare a sample to several templates and find the best one. The fact that you try to match to same sample allows you to compare DTW values. In keyword spotting using DTW the threshold must depend on every template you are matching, you can not use single threshold for all the templates. Even more, for the two different templates of the same length DTW thresholds must be different.

Improve this answer
$\endgroup$
15
  • $\begingroup$ My two pairs for which I compared the DTW were [one template + a similar file] and [same template + a slightly different file] for pairs 1&3, 2&4 in my example. So according to what you are writing, I try to match the same sample => I should be allowed to compare DTW. However, it is true that I use the same threshold. That leads me to another dependency question: what does the threshold depend on? You are writing that it must depend on every template... but that for two templates of the same length it should be different. Does it depend on the magnitudes computed by the FFT? On their sum? $\endgroup$
    – Ioanna
    Apr 2 '13 at 6:16
  • $\begingroup$ Sorry, I'm not sure how can you write at the same time "same template" and what is the difference between 1 and 2. Threshold does't depend upon anything, it can be statisically estimated from a large set. For example you can collect some set of DTW distances for true matches and for mismatches and select threshold to split between them. If you describe the problem you are trying to solve with DTW better, one could suggest you a reasonable solution. $\endgroup$
    – Nikolay Shmyrev
    Apr 2 '13 at 14:28
  • $\begingroup$ I'm not writing at the same time the same template: what I'm doing more exactly is that I record a sound and use it as a reference file. Then I record exactly the same sound (same voice, approx. same background noise) again and compare the two. That is what I meant by "same sound" in the examples 1 and 2 from my question. Then I record a slightly different sound (but same voice, ~same background noise) and compare it to the same reference file - examples 3 and 4. What I meant in the comment was that examples 1 & 3 used the same reference file, and so did examples 2 & 4. $\endgroup$
    – Ioanna
    Apr 2 '13 at 14:47
  • $\begingroup$ Anyway, you are welcome to describe the problem you want to solve and what is your trouble actually. $\endgroup$
    – Nikolay Shmyrev
    Apr 2 '13 at 15:08
  • 1
    $\begingroup$ Ok, good we come to understanding. Say you have a word you want to verify. The point is that the threshold can't be calulated, it has to be estimated statistically. What you need to do is to have 1 reference sample, then 20 correct samles matching the reference, then 20 incorrect samples matching the reference. Then you need to collect DTW statisticis for the distance DTW(reference, sample). After you collected statistics you need to build a distribution of DTW differences. After that you can select optimal threshold to separate correct samples from incorrect ones. $\endgroup$
    – Nikolay Shmyrev
    Apr 3 '13 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.