2
$\begingroup$

I am working on an algorithm for dominant speaker selection in a closed proximity cluster of smartphones (iOS and Android) with arbitrary positions. The smartphones act as clients, and one of them serves as the server. The server performs dominant speaker selection on the RMS of each packet and applies a moving average to smooth out instant surges and lows.
However, I am encountering an issue where even when a person speaks near one smartphone, another smartphone's microphone picks up the sound and incorrectly identifies it as the dominant speaker. This problem seems to be related to varying microphone sensitivities among different smartphone models, and manufacturers do not provide direct access to mic sensitivity information.
Considering the limitation of not knowing the microphone sensitivity of each smartphone, I am seeking advice on alternative factors or methods that could improve the dominant speaker selection process. I want to achieve accurate dominant speaker identification regardless of mic sensitivity.

Specifically, I would like to know:

  1. Are there any alternative or indirect ways to estimate the microphone sensitivity of smartphones, or is there any public database that provides such information? What other factors, apart from microphone sensitivity, can significantly influence the dominant speaker selection process?
  2. What additional signal processing techniques or algorithms can be applied to enhance the accuracy of dominant speaker identification?
  3. Should I consider using multiple features from the audio signal, such as frequency characteristics, to improve the robustness of the algorithm?

I appreciate any insights, suggestions, or research papers that could help me address this challenge and achieve reliable dominant speaker selection in the described proximity cluster of smartphones. Thank you!

$\endgroup$
2
  • $\begingroup$ This is a nice project. I am no expert on the field but I believe that the more information you include the better the algorithm (a tailored algorithm of course) will perform. That being said, when you include many features some may be redundant (or provide a tiny bit of new information) and choosing the best features is part of increasing the efficiency of your system. You could possibly try normalising the signals by their energy or some other metric and try working on them. I am not sure this will work at all but it may provide some information to move from there (or reject this approach). $\endgroup$
    – ZaellixA
    Jul 24, 2023 at 11:15
  • $\begingroup$ Hi @ZaellixA, Thank you for the response. I have already tried the Group Normalization technique, where I take the minimum and maximum values of each phone on the server, then set the global max and min, and perform normalization. However, this didn't help at all; it didn't reduce the unwanted switching. $\endgroup$ Jul 24, 2023 at 12:32

1 Answer 1

3
$\begingroup$

Tricky.

Unfortunately phones microphone pickup systems have a LOT of variability.

Are there any alternative or indirect ways to estimate the microphone sensitivity of smartphones, or is there any public database that provides such information?

There are no public data bases. In general iOS devices are reasonably well behaved and designed (flat frequency response, good manufacturing tolerance control, etc) and since the number of models is reasonably small, you could potentially calibrate this yourself. Note that you need to repeat the calibration after each iOS update since the operating system occasionally changes microphone behavior. The operating itself has also a good way of getting to the raw microphone signal bypassing all post-processing and modes.

For Android phones, it's the wild wild west. Too many models, too much variability, no standards and poor quality control. We found at some point that the frequency response of the exact same model can depend on the carrier. Google occasionally looks at this and tries to reel this in a bit, but so far nothing seems to have worked out.

What other factors, apart from microphone sensitivity, can significantly influence the dominant speaker selection process?

Unfortunately a lot: Form the outside in, we have:

  1. placement of phone relative to talker
  2. Room acoustics, especially node and anti-node locations of the room modes lining up with either the phone or the talker.
  3. Acoustic directivity of the phone enclosure
  4. Acoustic directivity of the capsules or capsule combination
  5. Sensitivity of the capsules themselves
  6. Pre-amplifier frequency response and gain
  7. A/D converter
  8. Low level driver and post-processing (filtering, noise shaping, automated gain control.
  9. High level driver and application code: beam former, carrier flavoring, speech enhancement, echo cancellation, etc.

A lot happens to the signal before you can get your hands on it.

What additional signal processing techniques or algorithms can be applied to enhance the accuracy of dominant speaker identification?

Temporal cues are generally much more robust than dynamic or spectral ones. If you can manage to time sync the phones, you can cross correlate and determine relative time of arrival. Chances are, the phone that got it first, is the closest to that specific talker.

Another method would be to determine which phone signal has the best signal to noise ratio or highest speech intelligibility and just use that.

However, that all depends on how exactly you define "dominant" and what you want to do with this information.

Should I consider using multiple features from the audio signal, such as frequency characteristics, to improve the robustness of the algorithm?

Frequency cues could work for iOS but not for Android. As indicated above other features to look at are temporal properties, signal-to-noise estimates, speech intelligibility estimates. None of those is easy, though.

$\endgroup$
4
  • $\begingroup$ Thank you for the response. I'll try these and will let you know. $\endgroup$ Jul 24, 2023 at 19:20
  • $\begingroup$ I just had a question: Is it possible to determine the proximity of the source solely through an analysis of the audio signal? I read somewhere that when we speak from a distance, the energy in higher frequencies tends to dissipate more swiftly than in low-frequency bands and will it be sensitivity independent? $\endgroup$ Jul 24, 2023 at 20:00
  • 1
    $\begingroup$ That's generally true outdoors but indoors it's hit or miss depending on the room acoustics. It also depends a lot which way the talker is facing and what the directivity of the microphone is. This effect would be indeed independent of sensitivity, but not of anything in the signal chain that's frequency dependent and unfortunately there is a lot of this. $\endgroup$
    – Hilmar
    Jul 25, 2023 at 15:24
  • $\begingroup$ Yup, thats what I observed. $\endgroup$ Jul 26, 2023 at 6:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.