# Averaging Different Length Speech Signal

I have around 10 wav audio files all containing same word spoken i.e. 'Hello' but all having different length. So lets say 1st file when loaded consist of 20000 samples, 2nd of 15000, 3rd of 18000 and so on they vary. Now I want to create a average template made of averaging all these 10 files, so that I can have better speech recognition result. But the problem is all the files have different length.

How can you average 10 different length signals?

• Why not append zeros (silence) to the shorter signals? Or cycle the signal (append samples from the start to the end)? – MBaz Sep 30 '15 at 21:48
• @MBaz I was thinking of doing that but I read somewhere that there is more professional way of doing this by using template normalisation using DTW which shrinks the size. Didnt get the hang of it...so thought to ask. – Mohit Sep 30 '15 at 22:19
• Yes, DTW is a usual approach. – Peter K. Sep 30 '15 at 23:46
• @Peter K. Could you elaborate the process. I thought DTW is used to calculate distance bw 2 signals...how that can be used to normalise the signals? – Mohit Oct 1 '15 at 6:39

Dynamic Time Warping is pretty well explained on this site. I'll use some of the diagrams from the PPT on that site to explain.

The idea is to divide the signals into segments (frames) and then compare frames sequentially through each signal. As illustrated below, motion from a segment in one signal to the next segment depends on the similarity to the current segment in the other signal. For your example of audio files, you usually have features that you extract from them (LPCs or MFCCs for example). Extract these from each frame. Use these features as the comparison features used in the DTW algorithm.

1) Suppose I have 10-10 frames of my 2 signals and each frame has 13 MFCC coefficients so the 2 DTW signals will be all coefficients arranged sequentially?

Well, before DTW you'll have $MFCC_s^f$ for signal $s$ for each frame, $f$:

Signal 1: $MFCC_1^1$,$MFCC_1^2$, $\ldots MFCC_1^{N_1}$

Signal 2: $MFCC_2^1,MFCC_2^2, \ldots MFCC_2^{N_2}$

where $N_1 \not=N_2$ and $N_s$ is the number of frames for each signal, $s = 1,2$.

After DTW, you'll have the best matching path over $N_\mbox{min} = \min(N_1,N_2)$ frames:

Signal 1: $MFCC_1^1$,$MFCC_1^1$, $\ldots MFCC_1^{N_?}$

Signal 2: $MFCC_2^1,MFCC_2^2, \ldots MFCC_2^{N_?}$

Then the comparison will be done by running all $those$ MFCCs sequentially.

2) How this shortest path can be used to decrease or increase the size of one signal so that they are of same size?

See above explanation: After DTW you'll only have the same number of MFCCs for each and every signal. All other (unused) frames with MFCCs are discarded for comparison purposes.

• I have already implemented DTW and I came to an understanding that g(n,m) gives the distance bw the two signals. Following that I have 2 questions. 1) Suppose I have 10-10 frames of my 2 signals and each frame has 13 MFCC coefficients so the 2 DTW signals will be all coefficients arranged sequentially? 2) How this shortest path can be used to decrease or increase the size of one signal so that they are of same size? – Mohit Oct 1 '15 at 12:41
• @Mohit : See my edit. – Peter K. Oct 1 '15 at 13:58
• Nice explanation. So if I have to average 10 signals, I should perform DTW on 1st and 2nd then the 3rd with their result and so on? Is this method the most efficient way? I have read somewhere about using interpolation to make all the signals of same size and then averaging them. As my domain is speech recognition, I would like the process which provides best accuracy. – Mohit Oct 1 '15 at 15:54
• The trouble with interpolating signals to the same size is that you need to be very sure that the start and end points are the same, and that none of the phonemes in the speech signals are longer in some than in others... which isn't really practical. You need to decide which of the signals is your "gold standard" and do the DTW against that for all the others. – Peter K. Oct 1 '15 at 16:39