1
$\begingroup$

I am trying to do speech recognition on an audio file which has 1 channel (mono). My problem is that the channel is comprised of the signal of 3 microphones which have captured the audio at different distances of a person speaking and the signals have been mixed into mono, which I suspect has added reverberation and noise into the audio, making it impossible for the ASR (automatic speech recognition) to recognize speech.

I am unable to obtain the original signals from each microphone so I need to know in which way I can clean the audio or revert the effect of the downmix.

I have already tried applying the Weighted Prediction Error (Nakatani et al, 2012) filter under the hypothesis that 2 of the 3 microphones would account for reverberation in the audio as the signal has its phase shifted (due to reverberation time) but had no improvements.

I do not have the transcriptions of the audios.

What can I do? I have no more ideas on how to continue.

EDIT: This is an example utterance of the audio file (WAV file, it is in spanish and intelligible for a person)

Yoshioka, Takuya, and Tomohiro Nakatani. "Generalization of multi-channel linear prediction methods for blind MIMO impulse response shortening." IEEE Transactions on Audio, Speech, and Language Processing 20.10 (2012): 2707-2720.

Thank you.

$\endgroup$
  • $\begingroup$ You can probably share the audio sample to make your problem clear $\endgroup$ – Nikolay Shmyrev Jul 6 at 21:48
  • $\begingroup$ I have just added an audio sample. I didn't know I could share files here. Thank you. $\endgroup$ – Marcos Jul 7 at 2:07
  • $\begingroup$ Ok, pretty bad indeed, how much data do you have? If it is not a growning amount, it is probably easier to transcribe it manually, not worth a research time to decode it accurately. $\endgroup$ – Nikolay Shmyrev Jul 7 at 5:27
  • 1
    $\begingroup$ If you just need to get this done, you could try a commercial solution such as zynaptiq: UNFILTER with an advertised example scenario: "Remove comb-filtering caused by multiple "open" mics being mixed to one iso track, or by proximity to reflective surfaces." $\endgroup$ – Olli Niemitalo Jul 7 at 8:13
  • $\begingroup$ @NikolayShmyrev I have around 60 hours of audio (and more to come). Each track is a 1-2 hours long dialogue and it is really time-consuming to transcribe manually, for this reason, I need somehow make this work. Thank you. $\endgroup$ – Marcos Jul 7 at 19:02
0
$\begingroup$

Record the audio from the three microphones separately and simultaneously using three audio cards. Use low sensitivity microphone to avoid audio pilferage. You can use ffmpeg command line to record the audio.

$\endgroup$
  • $\begingroup$ Thank you for your answer. Unfortunately, I can only work with the provided audio file, I can't access the microphones. $\endgroup$ – Marcos Jul 7 at 19:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.