Introducing Arcee Supernova Medius A 14b Model That Rivals A 70b 2
Source
arcee.ai
Author
Arcee AI
Date
Context
Arcee AI built SuperNova-Medius by transferring training signals across models that initially used incompatible vocabularies. Its training account describes temporarily changing a Qwen model's vocabulary to match Llama's, allowing it to learn from previously stored predictions of likely next tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition →.
The team then restored the Qwen vocabulary and merged that trained model with another distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → model and an internal fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition →. The mechanism explains how the 14-billion-parameter release combined learning from different larger models. The article's claim of approaching a 70-billion-parameter model is a company characterization; its detailed contribution is the vocabulary conversion and merging procedure.
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
