Dominant Speaker Selection in a Proximity Cluster of Smartphones with Arbitrary Positions (Not static)

Question

I am working on an algorithm for dominant speaker selection in a closed proximity cluster of smartphones (iOS and Android) with arbitrary positions. The smartphones act as clients, and one of them serves as the server. The server performs dominant speaker selection on the RMS of each packet and applies a moving average to smooth out instant surges and lows.
However, I am encountering an issue where even when a person speaks near one smartphone, another smartphone's microphone picks up the sound and incorrectly identifies it as the dominant speaker. This problem seems to be related to varying microphone sensitivities among different smartphone models, and manufacturers do not provide direct access to mic sensitivity information.
Considering the limitation of not knowing the microphone sensitivity of each smartphone, I am seeking advice on alternative factors or methods that could improve the dominant speaker selection process. I want to achieve accurate dominant speaker identification regardless of mic sensitivity.

Specifically, I would like to know:

1. Are there any alternative or indirect ways to estimate the microphone sensitivity of smartphones, or is there any public database that provides such information? What other factors, apart from microphone sensitivity, can significantly influence the dominant speaker selection process?
2. What additional signal processing techniques or algorithms can be applied to enhance the accuracy of dominant speaker identification?
3. Should I consider using multiple features from the audio signal, such as frequency characteristics, to improve the robustness of the algorithm?

I appreciate any insights, suggestions, or research papers that could help me address this challenge and achieve reliable dominant speaker selection in the described proximity cluster of smartphones. Thank you!

Answer 1

Tricky.

Unfortunately phones microphone pickup systems have a LOT of variability.

Are there any alternative or indirect ways to estimate the microphone sensitivity of smartphones, or is there any public database that provides such information?

There are no public data bases. In general iOS devices are reasonably well behaved and designed (flat frequency response, good manufacturing tolerance control, etc) and since the number of models is reasonably small, you could potentially calibrate this yourself. Note that you need to repeat the calibration after each iOS update since the operating system occasionally changes microphone behavior. The operating itself has also a good way of getting to the raw microphone signal bypassing all post-processing and modes.

For Android phones, it's the wild wild west. Too many models, too much variability, no standards and poor quality control. We found at some point that the frequency response of the exact same model can depend on the carrier. Google occasionally looks at this and tries to reel this in a bit, but so far nothing seems to have worked out.

What other factors, apart from microphone sensitivity, can significantly influence the dominant speaker selection process?

Unfortunately a lot: Form the outside in, we have:

1. placement of phone relative to talker
2. Room acoustics, especially node and anti-node locations of the room modes lining up with either the phone or the talker.
3. Acoustic directivity of the phone enclosure
4. Acoustic directivity of the capsules or capsule combination
5. Sensitivity of the capsules themselves
6. Pre-amplifier frequency response and gain
7. A/D converter
8. Low level driver and post-processing (filtering, noise shaping, automated gain control.
9. High level driver and application code: beam former, carrier flavoring, speech enhancement, echo cancellation, etc.

A lot happens to the signal before you can get your hands on it.

What additional signal processing techniques or algorithms can be applied to enhance the accuracy of dominant speaker identification?

Temporal cues are generally much more robust than dynamic or spectral ones. If you can manage to time sync the phones, you can cross correlate and determine relative time of arrival. Chances are, the phone that got it first, is the closest to that specific talker.

Another method would be to determine which phone signal has the best signal to noise ratio or highest speech intelligibility and just use that.

However, that all depends on how exactly you define "dominant" and what you want to do with this information.

Should I consider using multiple features from the audio signal, such as frequency characteristics, to improve the robustness of the algorithm?

Frequency cues could work for iOS but not for Android. As indicated above other features to look at are temporal properties, signal-to-noise estimates, speech intelligibility estimates. None of those is easy, though.