I am trying to use Dynamic Time Wrapping (DTW) + KNN to detect word occurrences in human speech audio.
I am to implement the same approach suggested in this DSP StackExchange post, but I am seeing that the algorithm is giving me visibly lower cost when aligning one word against the same word spoken by a different speaker. But it is unable to distinguish between audio clips with and without the keyword:
INFO:__main__:random2 costs: 78 82 84 INFO:__main__:random4 costs: 72 80 82 INFO:__main__:random1 costs: 72 73 93 INFO:__main__:random7 costs: 75 82 89 INFO:__main__:light3 costs: 76 83 88 INFO:__main__:light4 costs: 71 74 76 INFO:__main__:light6 costs: 73 81 87 INFO:__main__:random3 costs: 82 84 86 INFO:__main__:random6 costs: 77 79 80 INFO:__main__:standalone_light1 costs: 37 38 49 INFO:__main__:light5 costs: 58 67 79 INFO:__main__:standalone_light2 costs: 31 35 41 INFO:__main__:light1 costs: 66 72 79 INFO:__main__:light2 costs: 56 62 69 INFO:__main__:random5 costs: 64 71 77
The keyword I am trying to spot is "light", and the three numbers in each line
costs: x y z are the wrapping costs of the 3 nearest neighbors (all the keywords I am matching against are various recordings of the same word "light").
random[1-7] are audio clips without the word "light" being spoken.
light[1-6] are audio clips with the word "light" in it.
standalone_light[1-2] is a short clip with the word "light" alone.
light[1-6] are not very long, they are just short sentences. From the results its clear that DTW can identify two different recordings of the same word (see
standalone_light[1-2]). But it's not able to distinguish between recordings with and without the keyword.
Am I missing something? Can somebody point me in the right direction?
keyword_data_set = load_audio_mfcc(keywords_directory) target_audio_data_set = load_audio_mfcc(target_audio_dir, is_target_audio=True) for target_audio_name, target_audio_mfcc in target_audio_data_set.items(): costs =  for keyword_name, keyword_mfcc in keyword_data_set.items(): cost = calculate_cost(keyword_mfcc, target_audio_mfcc) costs.append(cost) logger.info(target_audio_name + ' costs: %d %d %d' % tuple(sorted(costs)[:3]))
I am using github.com/pierre-rouanet/dtw to calculate the DTW cost:
def calculate_cost(v1, v2): dist, cost, _, path = accelerated_dtw(v1.transpose(), v2.transpose(), 'euclidean') return dist