I need help figuring out a way to automatically synchronize two audio files that are both different recordings of the same source. More info below.


This example is from a concert, where there is only one audio source - the band.


  • In the first track of the image below, we have three minutes of audio recorded from the soundboard of the concert venue. This contains a crystal clear recording of the band's performance.

  • In the second track of the image below, we have roughly a minute and a quarter of audio recorded using an iPhone in the crowd. This audio still sounds ok, but is not great by any means. Still, it's very
    useable.

Since both of these recordings were taken at roughly the same time of the same performance, the audio in track two synchronizes somewhere with the audio in track 1.


Waveforms Out-Of-Sync


The image below shows where these two snippets of audio synchronize - about the 3 second mark.

Waveforms in Sync


I know that it's pretty easy to synchronize these two audio files up by hand and by ear, but I need to develop a script that synchronizes the two snippets automatically.


The Spectrogram and Spectrogram Log match up pretty well. I'm not sure if I can work with that or not.

Here is the spectrogram comparison:

Spectrogram

Along with the spectrogram log comparison:

Spectrogram Log


I've looked into Cross-Correlation of Waveforms using this video, FFT stuff, and other libraries. I'm lost and any help for my college project would be much appreciated!

  • You can use the Shazam algorithm. It works by examining the nearest neighbors of the spectral peaks. – Emre Nov 24 '14 at 4:34
  • Can you post 2 example audio files somewhere? – endolith Jan 29 '15 at 22:45

As good as AUDFPRINT is, I think it is overkill for your problem, as is solving a different problem (where the audio being compared can be arbitrary). You already know what that your shorter file and your longer file capture the same audio scene and you only need to calculate the "offset" between them.

This thread on cross-correlation has a lot of good info: Cross correlation vs FFT for finding phase between 2 signals

How do I implement cross-correlation to prove two audio files are similar?

If you go the cross-correlation route, I think that phase-correlation (variant) will give you much better results, than simply the basic cross-correlation.

The primary difference is that you normalize your fft coefficients to unit magnitude prior to your inverse-fft, so the phase-correlation is based only on phase information and is insensitive to changes in magnitude/intensity (which makes it more robust, able to synchronize quiet and loud recordings of the same scene)

You can use this offline matlab package by Dan Ellis. I guess you'll figure out how to use it for your problem on reading his descriptive documentation. Good luck.

The AUDFPRINT Matlab package referenced by @Yakku could be used to do what you need - if all you are after is the proper offset (and sampling rate correction.)

If you are looking to implement this yourself, then the Shazam algorithm referenced by @Emre would be one way to go.

AUDFPRINT seems to be an implementation of the Shazam algorithm.

The source for AUDFPRINT is available at the referenced site, so that it would be possible for you to look at an implementation of the Shazam algorithm to see how the algorithm can be translated to (Matlab) code. If you don't have access to Matlab, there are also tips on using AUDFPRINT with Gnu Octave, which is a free system that is (in large part) comaptible with Matlab.

If you would like to develop your own solution, then cross correlation would be an option.

Matlab and Octave both provide the xcorr method to compute the cross correlation. From the peaks in the result from xcorr you can find the best match and use that to synchronize your two files.

If you are interested in implementing your own cross correlation, this dsp.stackexchange.com question has a lot of good information on the algorithm and its implementation.

  • Just to add that there's also a python implementation of AUDFPRINT if you prefer to matlab/octave. However, I agree with other posters that AUDFPRINT is probably overkill for this problem. – tobassist Jul 29 '15 at 8:23

It might sound like overkill, but you can extract features (say, MFCC) from the spectrogram and apply Dynamic Time Warping (DTW) to locate the smaller clip. DTW is a popular technique in speech recognition, known for being relatively simple and fast. It is invariant to playing speed, and offers a lot of improvisations customized to speech waveforms.

DTW can provide a measure of the distance (similarity) between two strings. However, since the location of the smaller clip is necessary in your question, you might need to 'slide' it over the longer one and compute the distance measure at each location - sliding window DTW (check 2.2)

Take the amplitude spectrograms and then cross-correlate them along the time dimension and find the peak of the cross-correlation. Same as Shazam, but simpler and more robust because a lot of the complexity of Shazam is just data reduction for transmitting over the internet.

audfprint is the best way to do this. Don't mind the comments about being overkill. Correlations and such like in R will take you a long time to skill up on if you are not a statistician. If you get it working which is the hardest part (using pip install) of the necessary packages, you then just have to run 3 audfprint commands and a few linux commands to get the peaks of the original sample into a subdirectory, run peak generation over the sample to be aligned and then extract the answer at the end (the offset start time). The issue you might find is that your sample to be aligned is greater than 1 minute long and therefore you have to set a large max size for sampling for the clip to be found. To avoid using non-default settings for audfprint you can just extract the first 6 seconds or so from the clip (to be offset) and then run this into audfprint to ensure that you get 100% accuracy in its location. All you want is the offset after all. I've done this for 24 hour videos and it works really well.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.