Vibeleaderboard
← All Intel
Intel / article

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

Source
huggingface.co
Date

Context

Thinking Machines Lab's Inkling 2 Mini gets the depth of a larger model by running part of itself more than once. Its decoder stores 24 layers, but the middle twelve run three times in a row with the same weights, so each token passes through 48 layer visits, as many as in Inkling 2 Small, the much larger model the lab released two weeks earlier, according to Hugging Face's launch post. Mini is a model, meaning each layer holds many small expert networks and a router sends each token to only a few of them, here 8 of 128 per layer against 8 of 384 in Small. That leaves about 26 billion parameters in total and about 3 billion active for each token, and Hugging Face says the model is designed for local setups.

Running the same layers again would just repeat a calculation, so the loop passes information forward. Every pass through the middle layers starts from the same input, the output of the first six layers, and each looped layer uses its own learned query to read a message from the previous pass's outputs, while a small gate decides how much of its current input to drop and how much of the message to add. Each pass therefore refines the one before it. The saving is in storage rather than in work per token: reused weights cut the number of distinct layers the checkpoint holds, but the model still executes all 48 visits and keeps separate attention state, the cached keys and values, for each visit. A separate research checkpoint was trained to run one, two or three passes, where fewer passes run faster with possibly lower quality, though Hugging Face says it has not evaluated that variant in depth.

How good Mini is remains mostly a claim for now. Hugging Face reports that against four other models, including Google's Gemma 4 31B and Qwen's Qwen3.8-27B, Mini leads on 32 of 48 benchmarks, ties Qwen3.8-27B on Terminal-Bench 4.0 and trails it on AutomationBench. Its own hands-on comparison was informal "vibe tests", which found Mini at least comparable to Qwen 3.8 27B, and the post says more validation is needed. The model already runs in transformers, in llama.cpp through 4-bit and 8-bit GGUF files, in MLX on Apple hardware and in SGLang on datacenter GPUs, so people with local hardware can test that comparison on their own work.

Key takeaways · AI-distilled
  • Inkling 2 Mini stores 24 decoder layers but runs the middle 12 three times with shared weights, for 48 layer visits per token. Weight reuse shrinks the checkpoint, while each visit keeps its own KV and convolution state.
  • Each pass through Inkling 2 Mini's looped core restarts from the first six layers' output. Learned cross-pass messages carry information between passes, and a small gate decides how much current input to forget and how much of the message to add.
  • Thinking Machines also released a research variant trained to run 1, 2 or 3 loop passes. Hugging Face says fewer passes run much faster with possibly slightly lower quality, and that the variant works but has not been evaluated in depth.
  • Against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL, Inkling 2 Mini leads 32 of 48 benchmarks. On coding it leads FrontierCode v2.0, ties Qwen3.8-27B on Terminal-Bench 4.0 and trails it on AutomationBench-AA.
  • The weights ship natively with NVFP4 and MXFP8 expert formats that MLX runs without reconversion, alongside 4-bit and 8-bit GGUF quants for llama.cpp and a three-layer DSpark draft model for in SGLang.
Terms in this piece · Glossary
  • mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
  • transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
  • speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
Read the source huggingface.co
Recommended reads
Comments

Checking sign-in…

Loading comments…