Transcript
This video dissects Voxtral Realtime and Voxtral TTS from Mistral as case studies for modern *realtime* voice models. Mistral Studio: https://mistr.al/audio-juliaturc-yt Voxtral Realtime: https://arxiv.org/abs/2602.11298 Voxtral TTS: https://arxiv.org/abs/2603.25551 We also cover important milestones in the history of speech models, including: • OpenAI's Whisper • Whisper Streaming • WaveNet And a few foundational concepts: • Audio samples vs frames • Audio quantization: →Vector Quantization / VQ →Residual Vector Quantization / RVQ →Finite Scalar Quantization / FSQ • Semantic vs acoustic audio tokens 00:00 Intro 01:03 Modular vs end-to-end speech models 03:30 Speech-to-Text 06:07 Delayed Streams Modeling (DSM) 09:41 Whisper Streaming 10:33 Voxtral Realtime 13:07 Voxtral Text-to-Speech 14:28 Throwback: WaveNet 15:24 Audio tokenization 20:39 The Voxtral Codec 21:49 Back to Voxtral TTS 25:30 Outro