Vibeleaderboard
← All Intel
Intel / article

Fish Audio Open Sources S2

Source
fish.audio
Author
Fish Audio editorial sitemap
Date
Why it matters

S2 is an open-weights TTS model with free-form inline tags for word-level prosody control, plus code and an SGLang engine. Builders of voice agents can self-host it instead of depending on a closed API. Note the community license limits some uses.

Key takeaways · AI-distilled
  • Fish Audio reuses the same models for data curation and RL rewards: a speech-quality scorer filters training audio and later supplies the acoustic reward, and a transcription model that annotates data later scores intelligibility and -following.
  • S2 uses a Dual-AR design over a 10-codebook audio codec: a 4B-parameter slow model predicts the main semantic codebook along time while a 400M fast model fills the 9 residual codebooks per step, avoiding the 10x sequence growth of flattening all codebooks.
  • Fish Audio reports 0.54% Chinese and 0.99% English WER on Seed-TTS Eval, an 81.88% win rate against gpt-4o-mini-tts on EmergentTTS-Eval, and a 0.515 Audio Turing Test score versus 0.417 for Seed-TTS, all self-reported.
  • Because the architecture mirrors a standard autoregressive , S2 runs on SGLang with continuous batching and prefix caching. Reused cloning audio hits the prefix cache 86.4% of the time on average, and one H200 reaches a 0.195 real-time factor and about 100 ms to first audio.
Terms in this piece · Glossary
  • fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
  • streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.
  • Instruction — A statement in the context that tells the model how to behave or what to do; module 3's first layer.
  • Large language model — A model trained on large amounts of data to process and generate language.
More from Fish Audio editorial sitemap
Recommended reads
Comments

Checking sign-in…

Loading comments…