Kolibri: A Sovereign Open-Weight Model
- Source
- aleph-alpha.com
- Author
- bastitx
- Date
Kolibri is Aleph Alpha's bid for teams that need a capable model they can host themselves under European data-residency rules. The company's announcement describes a 78B mixture-of-experts model with about 3B parameters active per token and context of up to 1M tokens, built with a training pipeline it says is reusable for further sovereign models. The Apache 2.0 license matters as much as the size: it allows commercial use and modification without the custom terms attached to many open-weight releases. An independent walkthrough by Tejas Kumar lists slightly different figures, 3.5B active parameters and 262k context, so confirm the model card before sizing hardware or planning long-context work. That walkthrough also covers tokenizer behavior and where the model falls short, which is the more useful read if you are deciding whether Kolibri fits a German-language workload or whether a larger general model still wins.
Context
Kolibri gives organizations an English-German language model they can download and run on their own infrastructure. Aleph Alpha's release article says the full weights can be downloaded and used under the Apache 2.0 license, and that the model's size lets customers run it on-premise without sending internal data to third-party services.
German capability is also part of the training design. The company says German accounts for 21.3% of tokens and that it retuned data filters that would otherwise remove the register public administration writes in. This connects the model's deployment options to its intended audience: organizations working with German-language material and wanting control over where it is processed.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
- pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
- mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…