Vibeleaderboard
← All Intel
Intel / article

Mai Transcribe 1 5 New Speech To Text Model Leading The Accuracy Speed Pareto Frontier

Source
artificialanalysis.ai
Author
Artificial Analysis articles
Date

Context

Microsoft's MAI-Transcribe-1.5 has a 2.4% word error rate on AA-WER, third on the Artificial Analysis speech to text leaderboard, behind Alibaba's Fun-Realtime-ASR-preview at 1.7% and ElevenLabs Scribe v2 at 2.2% (artificialanalysis.ai). What stands out is speed. It processes audio at about 276 times real time, the fastest of the ten most accurate models and more than double the speed of the next fastest among them. That combination puts it at the front of the accuracy-speed Pareto frontier, the set of models no other model beats on both measures at once.

Two months earlier, Artificial Analysis measured MAI-Transcribe-1 at 3.0%, fourth place, and about 69 times real time. The new version also expands language support from 25 to 43 and adds keyword biasing, which improves recognition of rarer vocabulary such as names and medical terms. The price through Microsoft Foundry is unchanged at $6 per 1,000 minutes of audio.

Artificial Analysis also breaks the result down by test set. MAI-Transcribe-1.5 ranks second on VoxPopuli at 1.6%, fourth on Earnings22 at 4.0% and fifth on AA-AgentTalk at 2.0%. These are the three sets defined in the AA-WER v2.0 announcement, which describes VoxPopuli as parliamentary speeches, Earnings22 as company earnings calls and AA-AgentTalk as a private set of speech directed at voice agents, and which gives AA-AgentTalk half the weight in the overall score and the other two a quarter each.

Key takeaways · AI-distilled
  • MAI-Transcribe-1.5 trails Alibaba's Fun-Realtime-ASR-preview (1.7% WER) and ElevenLabs Scribe v2 (2.2%), but is more than twice as fast as any other model in the top 10 for accuracy.
  • Per-dataset results: 2nd on VoxPopuli-Cleaned-AA (1.6% WER), 4th on Earnings22-Cleaned-AA (4.0%) and 5th on AA-AgentTalk (2.0%).
  • Keyword biasing improves recognition of rare vocabulary such as names and medical terms; the model is sold through Microsoft Foundry.
Read the source artificialanalysis.ai
More from Artificial Analysis articles
Recommended reads
Comments

Checking sign-in…

Loading comments…