I know that usually ASR systems are evaluated using WER (word error rate), and I was wondering if it makes sense to use a more fine grained approach. E.g. transforming both the ASR hypothesis and the reference transcription to its phonetic representation (e.g. using an algorithm such as soundex or metaphone), and then taking the WER.
Some half-baked ideas for use cases:
perhaps this evaluation method can help to distinguish between errors that stem from a bad language model to those that stem from a bad acoustic model
when using third-party ASR APIs, where the there's no control over the language model, perhaps it can be used to pick the best hypothesis among the top results (especially when the speech contains non-standard names and other out-of-vocabulary) words