How to detect acoustic and/or network degradations in speech as early as possible before automatic speech recognition?

Question

I'd like to learn about techniques and approaches to identify speech degradations before they enter automatic speech recognition system and cause false recognitions.

For example, I'd like to detect distorted speech in many different ways (reverb, clipping, pops, clicks, background noise, low SNR, to quite or too loud, ...).

I guess several checks or techniques should be applied to detect each type of anomaly, so I'd like to learn about them. Also, I'd prefer less resource intensive approaches as I have to detect them on more limited computing level.

I have found some techniques about general intelligibility level of speech, but most of them are intrusive and quite complicated. On the other hand, haven't found relevant practical resources on how to detect anomalies in more practical way, preferrably in time domain or simpler operations in frequency domain.

Thanks in advance for help, regards, Bob.

Answer 1

Sounds like what you want is Speech Intelligibility estimation and maybe also Speech Quality estimation, that can operate single-ended (no need for a reference recording). A standard for this used in telecommunications is ITU-T P.563. The areas is still under active research, and a recent review article is Nonintrusive objective measurement of speech intelligibility: A review of methodology.

Answer 2

This is by no means a trivial problem. And it is manyfold, so it has to be attacked in several ways.

Recording level issues

For issues of this category, you can just continually scan the signal for clipping (a certain number of samples at 0dBFs) and calculate an RMS level, which then can be compared to a minimum required level (something in the range of -35dBFs)

Transient distortion

Clicks, pops and transient parts of background noise are best detected in time domain. LPC error has proved to be a robust estimator for these types of degradation and it is quite lowcost algorithmwise, which is a bonus. A search for "click detection LPC" should produce enough examples to get you going.

Quasi-stationary distortion

Here we're talking about hum, random noise, spectral degradation, feedbacks and such. Most of these phenomena will be hard or even impossible to detect without any reference. As there can't be a direct reference, i suggest the following:

• Necessary assumptions: most of the time the microphone signal will be clean, the background noise low level, no severe degradations are present and the user is speaking at a "normal" level. Transient degradations can be ruled out by first running the LPC error detector.
• Under these assumptions, plug in a self learning voice recognition algorithm (not speech recognition) to identify which user is speaking. It might suffice to categorize (male/female).
• Once identified/categorized, compare the short time spectrum to a long term spectrum (which you create from the first $n$ seconds/minutes of speech from this user/category). Here, you should be able to detect hum and feedback directly. SNR degrading background noise can either also be detected directly (if it is quite loud) or indirectly, when you hit a speech pause.

All in all, a huge part of the work will be fiddling with parameters and thresholds. Try to experiment with a variety of equipment and speakers.