0
$\begingroup$

I have 2 audio files recorded using laptop mic and one with external mic. Laptop mic recording starts after external mic. The time difference could be 2-60 seconds.

So I wrote this code which is pretty accurate, but when I bring both audio tracks (one from the video that uses laptop mic) and the newly adjusted audio, there is still like 50ms delay. Why could this be?

enter image description here

Sometimes it seems there is positive delay, and sometimes negative. enter image description here

from scipy.signal import correlate
from scipy.signal import fftconvolve
from scipy import signal
from pydub import AudioSegment
from pydub.utils import mediainfo

import numpy as np

def findOffset(audio1, audio2):
    correlation = signal.correlate(audio2, audio1, mode="full")
    lags = signal.correlation_lags(audio2.size, audio1.size, mode="full")
    lag = lags[np.argmax(correlation)]
    
    return lag


def adjustAudio(audio_segment, lag, frame_rate):
    # Convert lag from samples to milliseconds, rounding to nearest integer at the last step
    ms_lag = round((lag / frame_rate) * 1000)
    
    if lag > 0:
        # Audio needs to start later: pad audio at the beginning
        silence = AudioSegment.silent(duration=ms_lag, frame_rate=frame_rate)
        adjusted_audio = silence + audio_segment
    else:
        # Audio needs to start earlier: trim audio from the beginning
        adjusted_audio = audio_segment[abs(ms_lag):]  # Use abs to convert negative lag to positive

    return adjusted_audio


def alignAudioTrack(audioFile, newAudioFile, lag):
    audio_data, rate, audio_segment = loadAudio(audioFile, return_segment=True)

    # Adjust the AudioSegment based on lag, ensuring frame_rate is passed correctly
    adjusted_audio = adjustAudio(audio_segment, lag, rate)

    # Fetch original bitrate
    bitrate = mediainfo(audioFile)['bit_rate']

    # Save the adjusted audio preserving the original bitrate
    adjusted_audio.export(newAudioFile, format="mp3", bitrate=bitrate)


def loadAudio(file_path, return_segment=False):
    audio_segment = AudioSegment.from_file(file_path)
    data = np.array(audio_segment.get_array_of_samples())
    if audio_segment.channels > 1:
        data = data.reshape((-1, audio_segment.channels)).mean(axis=1)  # Convert to mono if stereo
    rate = audio_segment.frame_rate

    if return_segment:
        return data, rate, audio_segment
    else:
        return data, rate

audio1, rate1 = loadAudio(audioFile1)
audio2, rate2 = loadAudio(audioFile2)
lag = findOffset(audio1, audio2)
alignAudioTrack(origAudioFile, alignedAudioFile,  lag)
$\endgroup$
2
  • $\begingroup$ If they're recorded using different asynchronous hardware, they could have clock drift that changes the alignment over the recording (or just sample rate offset that causes one to be slightly shorter than the other, etc.) $\endgroup$
    – endolith
    Commented Apr 19 at 16:01
  • $\begingroup$ actually i found the issue it was about the audio extracted from the video had a delay. i am using moviepy VideoFileClip. So need to find a way to get the same audio without delay. $\endgroup$
    – Joan Venge
    Commented Apr 21 at 9:17

1 Answer 1

1
$\begingroup$

The mp3 de-/encoding messes up the alignment. Try exporting the audio to wav-format:

adjusted_audio.export(newAudioFile, format="wav", bitrate=bitrate)

Like this, the alignment should be perfect.

$\endgroup$
17
  • $\begingroup$ Thanks let me give this a try now. $\endgroup$
    – Joan Venge
    Commented Apr 19 at 6:32
  • $\begingroup$ Does it matter that I am comparing mp3 files though or do I need to convert those into wav also? $\endgroup$
    – Joan Venge
    Commented Apr 19 at 6:39
  • $\begingroup$ That does not matter. Once you have loaded them, they are no longer encoded. Just write the aligned ones as wav files. You probably have to adjust the bitrate or possibly other parameters. $\endgroup$
    – Max
    Commented Apr 19 at 6:43
  • $\begingroup$ Ok I just tried it and got the wav. I compare it with the mp3 but they look exactly the same so no change. I am using the same bitrate from the original so not sure what else I can do. I will check if other files have the same delay or not. $\endgroup$
    – Joan Venge
    Commented Apr 19 at 6:48
  • $\begingroup$ Ok I just checked it. Sometimes there is positive delay, sometimes negative. The editor I am using I can only zoom this much. But when I play both tracks, I hear double voice in any case. $\endgroup$
    – Joan Venge
    Commented Apr 19 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.