Meta's Muse Voice Transcribe adapts latency word by word

Meta's Muse Voice Transcribe is a speech-to-text model for live audio: it writes the transcript while someone is still talking instead of waiting for a finished recording. Meta's launch post says the model, built by Meta Superintelligence Labs, also handles diarization, which labels who is speaking and works with more than 20 speakers, and endpointing, which detects when a speaker has finished. Developers can bias it toward an expected language, keywords or context; Meta's example is getting names such as Menlo Park right.

The hard decision in live transcription is when to commit a word. Meta's technical blog explains that audio arrives in 80-millisecond chunks, and after each chunk the model either keeps listening or writes text. Waiting gives it more context and a more accurate transcript, but adds delay. Instead of fixing one waiting time, Meta trained the model with reinforcement learning on a reward that combines word accuracy and delay, so it waits longer on difficult words and commits sooner on easy ones. Speaker changes and the start and end of speech come out as special markers in the same text stream, so one model produces all three outputs.

Meta says the model ranks first on the Artificial Analysis streaming speech-to-text leaderboard and on public diarization benchmarks. The first claim has outside support: Artificial Analysis reported a 3.1% word error rate on final transcripts, returned 0.16 seconds after speech ended, which put it ahead of Cartesia Ink-2 and ElevenLabs Scribe v2 Realtime on final accuracy. The diarization claim rests on Meta's own chart, which shows a 17.5% error rate averaged across the AMI and VoxConverse test sets against 21.1% or higher for five other systems that the blog does not name.