← All IntelClip / EducationTwo never-ending streams: silence as a modeled token
From Full Duplex Models: Moshi and the New Voice AI Paradigm · ≈8:36
The concrete representational trick behind full duplex — model both participants as concurrent uninterrupted audio streams so there are no turns to hand over.
What’s in it
- The concrete representational trick behind full duplex — model both participants as concurrent uninterrupted audio streams so there are no turns to hand over.
Clip transcript
>> The core idea is to view this conversation as two separate concurrent audio streams, one for the user and one for the assistant. Crucially, neither stream is ever discontinued. When the user goes quiet, their stream simply carries silence, which is modeled like any regular piece of audio. So, there are no more turns to hand over. For instance, here, the agent is still answering the second question while the third one is already coming in. It's listening and speaking at the same time. A turn-based system would have to make an impossible choice here, either cut its answer short or miss the incoming question. Formally, this is known as a full duplex. The channel carries communication in both directions at the same time. This is exactly how OpenAI describes GPT live. The traditional turn-based systems are half duplex. The channel still works in both directions, but the agent is either listening or speaking, never both.
Comments
Sign in to comment.
Loading comments…