Vibeleaderboard
← All Intel
Intel / article

Thinking Machines introduces Inkling, an open-weight 975B MoE model

Source
thinkingmachines.ai
Author
Thinking Machines
Date

Context

Thinking Machines released Inkling as an model that developers can customize through further training. Its launch article describes a design with 975 billion parameters in total and 41 billion active per token: each processing step uses a selected portion of the model's capacity.

Inkling handles text, images and audio, and the company's Tinker service provides a hosted route to it for specialized tasks. Although the model supports a of up to one million tokens, the launch offered 64,000- and 256,000-token options on Tinker. Model capacity and the limits of a particular hosted service therefore need to be read separately.

Terms in this piece · Glossary
  • open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
  • mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
  • fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Key quotes

“Our mission is to build AI that extends human will and judgment.”

“Inkling is not the strongest overall model available today, open or closed.”

“Inkling is just the start: our first release in a model family we will continue to build on.”

“I need to handle the full loop: create a training dataset and objective, fine-tune the current model on Tinker, evaluate it against the base model, and switch to the improved weights.”

Inkling
Read the source thinkingmachines.ai
More from Thinking Machines
Recommended reads
Comments

Checking sign-in…

Loading comments…