The Open ASR Leaderboard Adds Its First Global South Language

The Open ASR Leaderboard's new Hindi and Indian-English tests make it possible to ask whose speech a transcription model handles well. The Monsoon release pairs recordings with speaker information, including geography and age. Developers can break down errors by group instead of relying entirely on an overall average.

Hindi scoring also accepts multiple valid written forms of the same utterance, reducing penalties for spelling choices that still represent the speech correctly. Public and private test sets use different speakers. A separate earlier study in the same leaderboard series examined whether top models reproduce benchmark-specific reference text rather than what was said, a different problem from the speaker coverage and spelling variation addressed here.