unsloth v0.1.922-beta — Inkling Mini + Bug fixes
- Source
- github.com
- Author
- danielhanchen
- Date

Context
Unsloth's headline for this release is that it can run Inkling 2 Mini, which it calls "the strongest 26.4B model that runs on 16GB RAM" (release notes). Inkling 2 Mini is an open-weights model from Thinking Machines, released under Apache 2.0 with a 1M-token . It has 26.4 billion parameters in a design, in which only some of the model's weights work on each token; Unsloth's guide puts the active count at 5.4 billion. Thinking Machines' own announcement explains the unusual part: some layers run several times for each token instead of once (our explanation of that announcement).
The 16GB figure depends on which file you download. Unsloth publishes the model , meaning each weight is stored with fewer bits so the files shrink at some cost in accuracy. Its Inkling 2 Mini guide lists the memory each version needs, counting RAM plus graphics memory or a Mac's unified memory: 13 to 15 GB for 3-bit versions, 17 to 18 GB for 4-bit and 24 GB for the 8-bit file, which Unsloth labels full precision. The version it recommends is a 4-bit file of 17.6 GB, so a 16GB machine is limited to the 3-bit or smaller versions. The guide adds that longer contexts, image inputs and other running apps can use more memory.
Unsloth measures each smaller file against its own 8-bit file, which keeps the original expert weights, rather than against a 16-bit version. On that measure the recommended 4-bit file picks the same most likely next token as the 8-bit file 98.0% of the time. Unsloth's main files store the repeated layers in its own layout and need its own build of llama.cpp, the open-source engine many local tools build on. Standard llama.cpp reads a different layout, so its users need the separate files in the guide's Mainline_Support folder.
- Unsloth reports MiniMax-H3-Image now renders a new prompt in 36.7 seconds on an 8GB GPU, down from 82.7 seconds, and FLUX.1 steps take 16 to 28% less time when its weights stream from system RAM.
- Training also got faster in Unsloth's numbers: AsyncGRPO steps run 1.46x to 2.47x faster on A100 with TRL 1.17 or newer, and SDXL training takes 28 to 31% less time per step on T4 and L4 GPUs.
- The MCP token savings come from keeping each chat's tool list fixed, so a message only processes what is new. The Tool output length setting now also caps MCP results on llama.cpp, vLLM, Ollama and Custom connections.
- Video restyle on Apple Silicon Macs needs about 42 GiB of GPU memory at bf16, or about 31 GiB with q8_0, according to the release notes.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
- quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
- LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
Checking sign-in…
Loading comments…






