0
$\begingroup$

I have trained a machine learning model to recognize a specific audio event from mel spectrograms, and it works pretty well on a hold-out test dataset. I now want to run the model on a device and feed it with real audio but I have a problem that I don't now how to solve.

The problem is that when I'm testing my audio recognizer by playing audio clips it fails to recognize most of them, even if it can correctly recognize the audio in the original file. When I compare the spectrogram from the original audio file with the spectrogram computed by the recording of my device, I can see some differences:

enter image description here

enter image description here

Some of the differences has to do with the fact that the real time algorithm is doing the time-domain segmentation a little bit differently so that it "misses" the first few milliseconds of the event. But I have verified that this is not a big problem for the machine learning model. So the reason for why it fails seems to be because the loudspeaker and/or the microphone messes with the audio so that some frequencies are attenuated. The audio file has a 50kHz sample rate, which I think my high-end loudspeaker should be able to handle. My device samples the audio at 44.1kHz and the mel spectrogram uses frequencies up to 22.05kHz.

How can I make my feature extractor more robust? So that the spectrogram looks more similar irrespective of which microphone is used for recording or if I use a loudspeaker or if the audio event is actually happening in front of the microphone.

$\endgroup$
1
  • $\begingroup$ i think the answer is in part, train on data from a number of microphones. your mike data looks better at the low frequencies and has less at the high, so a gentle high pass filter might help $\endgroup$
    – user28715
    Commented Oct 7, 2019 at 16:43

1 Answer 1

1
$\begingroup$

Some ideas: You could train the model using audio with randomly chosen speaker--mic system impulse responses applied. To get you started, there are microphone impulse response (IR) packs available, intended for music production purposes. Or you could perhaps generate synthetic impulse responses that cover the variability in real-world impulse responses. Or measure your own using various speaker--mic placements. The room will have an effect too; possibly you could use room IR packs, for reverbs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.