0
$\begingroup$

I am trying to use Dynamic Time Wrapping (DTW) + KNN to detect word occurrences in human speech audio.

I am to implement the same approach suggested in this DSP StackExchange post, but I am seeing that the algorithm is giving me visibly lower cost when aligning one word against the same word spoken by a different speaker. But it is unable to distinguish between audio clips with and without the keyword:

INFO:__main__:random2             costs: 78 82 84
INFO:__main__:random4             costs: 72 80 82
INFO:__main__:random1             costs: 72 73 93
INFO:__main__:random7             costs: 75 82 89
INFO:__main__:light3              costs: 76 83 88
INFO:__main__:light4              costs: 71 74 76
INFO:__main__:light6              costs: 73 81 87
INFO:__main__:random3             costs: 82 84 86
INFO:__main__:random6             costs: 77 79 80
INFO:__main__:standalone_light1   costs: 37 38 49
INFO:__main__:light5              costs: 58 67 79
INFO:__main__:standalone_light2   costs: 31 35 41
INFO:__main__:light1              costs: 66 72 79
INFO:__main__:light2              costs: 56 62 69
INFO:__main__:random5             costs: 64 71 77

The keyword I am trying to spot is "light", and the three numbers in each line costs: x y z are the wrapping costs of the 3 nearest neighbors (all the keywords I am matching against are various recordings of the same word "light").

random[1-7] are audio clips without the word "light" being spoken. light[1-6] are audio clips with the word "light" in it. standalone_light[1-2] is a short clip with the word "light" alone.

All random[1-7] and light[1-6] are not very long, they are just short sentences. From the results its clear that DTW can identify two different recordings of the same word (see standalone_light[1-2]). But it's not able to distinguish between recordings with and without the keyword.

Am I missing something? Can somebody point me in the right direction?

Code:

keyword_data_set = load_audio_mfcc(keywords_directory)
target_audio_data_set = load_audio_mfcc(target_audio_dir, is_target_audio=True)

for target_audio_name, target_audio_mfcc in target_audio_data_set.items():

    costs = []
    for keyword_name, keyword_mfcc in keyword_data_set.items():
        cost = calculate_cost(keyword_mfcc, target_audio_mfcc)
        costs.append(cost)

    logger.info(target_audio_name + ' costs: %d %d %d' % tuple(sorted(costs)[:3]))

I am using github.com/pierre-rouanet/dtw to calculate the DTW cost:

def calculate_cost(v1, v2):
    dist, cost, _, path = accelerated_dtw(v1.transpose(), v2.transpose(), 'euclidean')
    return dist
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.