How mic setting will affect audio classification model?

Question

I have audio samples with three classes :

100 audio samples : class 'A'
100 audio samples : class 'B'
100 audio samples : class 'C'

Class 'A' and Class 'B' audio samples are recorded from one phone mic with same setting ( distance, volume etc ) but class 'C' samples are from different phone mic, ( each audio sample is from different phone mic )

I am working on a ML classifier to classify all three audio classes. My question is if I downsample all the audio signals to one frequency (i.e 16 kHz)

• Will model still be biased because of different phone mic?
• What affects model will face because of different phone mic and what is the better solution for this problem other than downsampling to same frequency ?

Answer 1

We can't tell you what your classifier does, sorry.

But yes, you have a systematic bias in your data, and your classifier will cling to whatever is the strongest discriminator if it works as hoped. I will rename your classes to make this clearer:

• 100 audio samples : class 'Microphone 1 subclass 1'
• 100 audio samples : class 'Microphone 1 subclass 2'
• 100 audio samples : class 'Microphone 2'

If that strongest discriminator is the microphone used in the downsampled signals (and that is not an unreasonable assumption to make), then yes, your classifier performance will depend on the microphone used.

The "properest" way would be to go out and acquire some of class 'A' and 'B' with microphone 2; but assuming that is unreasonably much work: you could try playing back all sounds and record them through both microphones, and use random batches so that the you don't only see class C with the properties of microphone 2, but also classes A and B through 2, and C through 1. Maybe that is sufficient to reduce the role of the microphone characteristics for classification.

But really, you have a class that is really identical with being recorded via microphone 2, and your choice to call that class "C" instead of "recordings done with microphone 2" is your choice alone!

Answer 2

Two techniques are useful to increase robustness of your model: Normalization and Data Augmentation.

Normalizing the inputs to the models can be used to remove the difference in audio level between classes, samples, distance from-source, or device the data comes from. A common method with spectrogram as feature representation is to either mean/std normalize, or to max normalize. This can be done on the sample level, or analysis window level.

Data Augmentation. Synthetically creating modified examples to introduce robustness/invariance against the type of change. For robustness across audio devices, the most relevant data augmentation might be frequency response.