Trinity Large uses a highly sparse MoE architecture:
We’re releasing three variants from the same training run:
- Trinity-Large-Preview: lightly post-trained, chat-ready
- Trinity-Large-Base: full 17T token pretraining checkpoint
- Trinity-Large-TrueBase: 10T token checkpoint with no instruct data or LR anneals
Sparsity was a core design choice.
By keeping the routing fraction low (4-of-256), Trinity Large achieves significantly faster inference than dense models in its weight class, while maintaining strong performance.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
Why it matters
Three checkpoints from a single 17T-tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → run, including a rare base model with no instruct data or LR anneals, plus a published sparsity and stability recipe for training at this scale.
