Audio signal comparison for automatic singing evaluation

Question

I am trying to compare audio waveform from input waveform (A) to preset waveform (B)from in the device.

I try to lookup everywhere. Most likely someone give such a source code as follow :

    function compireAudio(){  
    var audio1 =    "http://soundjax.com/reddo/86502%5Ealarm.mp3";  
    var audio2 =    "http://soundjax.com/reddo/44368%5EALARME.mp3"; 
    var i,j,d; 
    var matching = 0; 
    var t = 0;
    var i,j,d; 
    var matching = 0; 
    var t = 0; 
    var audio1Arr = Array(); 
    var audio1Len = audio1.length; 
    for (i = 1; i<=audio1Len; i++) {
        //reverse so its like a stack
        d = audio1.charCodeAt(audio1Len-i);
        for (j = 0; j < 8; j++) 
        {
            audio1Arr.push(d%2);
            d = Math.floor(d/2);
        } 
     }
     var audio2Len = audio2.length; 
     for (i = 1; i<=audio2Len; i++) {
       //reverse so its like a stack
       d = audio2.charCodeAt(audio2Len-i);
       for (j = 0; j < 8; j++) 
       {
         if(d%2 == audio1Arr[t])
         {
             matching++;
         }
         d = Math.floor(d/2);
         t++;
       }  
     }
   var avarage = Number(matching)/((Number(t)+Number(audio1Arr.length))/Number(2))*Number(100);
    alert('The Matching with the two audio is '+avarage+' %.');

And most people said this is not correct because it compare the stream instead of the audio content.

Anyway, I have think of compare every samples after normalization (ensure that amplitude is not a factor here). However, a phase is another problem here.

I have solution is cross-correlation between A and B and find the biggest convolution as output. In this case, they should be in same phase.

In short. WAS Question is What are the common methods for two waveform comparison ?

UPDATE Maybe I should ask What are the common characteristic of a audio waveform for comparing audio waveform ?

P.S. I read this Question and answer. The bottom method seems work, but the comment said No.

Answer 1

Please update your question to mention which kind of comparison you want to perform. You seem to want your comparison to be robust to amplitude changes and time delays, but what about:

• Audio coding. Should a signal and a MP3 version of itself be considered as similar or different?
• EQ and filtering. Should a signal and the same signal with the high frequencies shifted by 10° be considered similar or different?
• Dynamics change. Does applying a time-varying gain factor to one signal make it different from the original?
• Time and pitch shifting. Should a signal and a slightly accelerated version of itself be considered as similar or different?
• Noise and recording conditions. Should two versions of the same signal recorded on different equipment, microphone type, etc. be considered as similar or different?
• Timbre. Should two versions of the same melody played by two different instruments be considered as similar or different?

Depending on your application (assessing degradations through a signal processing chain, identifying copyrighted work in an audio stream, shazam-like song identification, plagiarism detection on music...), you might want your comparison to be robust to some of these transformations.

Your approach consisting in normalizing the signals is not very robust. If you have two identical signals $x$ and $y(t)$, and if you change $x(0) = 2 \max_t y(t)$, the normalized version of $x$ is twice smaller than the normalized version $y$ and the comparison will fail.

Using cross-correlation to look for a time delay is a sound approach.

Answer 2

The target application is automatic singing evaluation.

Do you want this to recognize that the lyrics are sung correctly? If not (you don't mind if the people is singing meaningless words, humming, whistling), the only thing that is worth being compared is the pitch contour - you can extract that and ditch all the other information from the waveform.

A simple system would at least do the following steps:

• Break down your signal into short frames (20 or 50 ms), use a pitch extractor such as YIN to extract the pitch of the reference signal (good singing), and of the recorded signal.

• Look for the optimal shift in pitch to account for the fact that people might sing one octave above or below the original, and shift the recorded pitch sequence by this. You don't want to penalize a female singer for singing one octave above the original song.

• Align the two sequences using DTW. How much the DTW path deviates from a straight line give you the rhythmic accuracy ; the total DTW distance after alignment gives you the overall pitch accuracy.

• You can also get an objective quality metric, which doesn't require an original recording, by measuring the average distance between the pitch contour and the nearest pitch on the tempered scale.

• You can also give "stylistic" bonus points by detecting vibrato (is there a steady sinusoidal modulation of the pitch contour on held notes)...

These techniques are well-known in the MIR community - and companies like BMAT or Voxler already have commercial SDKs for doing this.

For recognizing lyrics, the most basic solution would consist in extracting a sequence of MFCC from the original ; a sequence of MFCC from the recorded performance, then finding the MFCC transform matrix that minimizes the DTW distance between the two sequences (to account for voice timbre difference - aka Vocal Tract Length Normalization) ; and use this minimal DTW distance as a "lyrics similarity" score.