Qwen2-Audio: Chat with Your Voice!

Qwen's Qwen2-Audio accepts spoken instructions or audio paired with a written question and produces a text response. Its release article distinguishes voice interaction from audio analysis, which can cover speech, environmental sounds, and music.

The architecture joins an audio encoder with a language model and trains them to align audio information with language. This lets spoken instructions reach the model without a separate speech-transcription component. For an application builder, the supported interface is audio understanding followed by written output; generating a spoken reply would require an additional speech-generation component.

“Today, we release Qwen2-Audio, the next version of Qwen-Audio, which is capable of accepting audio and text inputs and generating text outputs.”