Intel / repo

unsloth v0.1.910-beta — MOSS-Image, Flint & NVIDIA Bolt E4B

Source: github.com
Author: danielhanchen
Date: Published Oct 23, 2026

Context

The most consequential change in this Unsloth release lets a Mac run models that are larger than its memory, by leaving part of the model on the SSD (release notes). Unsloth is an open-source tool for running and fine-tuning AI models on your own computer. A mixture-of-experts model is built from many smaller sub-networks, called experts, and uses only a few of them for each piece of text, so most of its weights sit idle at any moment. The release notes say this lets Qwen3.8-Flash-Next run on a 32 GB Mac, and that DeepSeek-V4-Flash and GLM-5.3-Flash need far less memory than their size.

The trade-off is speed. In the pull request that added the feature, the author warns that generation becomes "disk-bound, several times slower" than with the whole model in memory. By default the feature switches on only when a model would not otherwise fit, and it moves the smallest share of the model to disk that makes it fit. The author's test on a 128 GB M3 Max put all of GLM-5.3-Flash's experts on disk and cut its in-memory footprint to 21 GiB.

The release also adds three models. Recursive's Flint is a coding-agent model with 40 billion parameters, 2.5 billion of them active at a time, which the notes say fits in 24 GB at 3-bit precision for short contexts. OpenMOSS's MOSS-Image is a 5.1-billion-parameter image generator that can render English or Chinese text. NVIDIA's Nemotron 3.5 Bolt E4B is a fine-tune of Google's Gemma 4 E4B for , scored at 63.2 on the Tau2-Bench tool-use benchmark against the original's 38.7. Those scores are as reported in the release notes, not independent tests.

Key takeaways · AI-distilled

Unsloth now implements OpenAI's fine-tuning API (/v1/files and /v1/fine_tuning/jobs), so the OpenAI SDK can upload datasets and run supervised fine-tuning of local models.
Requests made with an Unsloth API key can no longer run Python or terminal tools unsandboxed; only the Unsloth UI can allow that. Installs also reject eMudhra's compromised emSign certificates, though the optional vLLM and SGLang engines still trust them.
Per Unsloth, Recursive's Flint has 2.5B active of 40B parameters and 256K , scoring 89.7 on Pro v2 vs Qwen 3.6's 73.8. NVIDIA's Nemotron 3.5 Bolt E4B beats Gemma 4 E4B on 16 of 21 NVIDIA tests, most in tool use and multi-turn chat.
REAP pruning trades some quality for size, and Unsloth says its Python reap_prune runs up to 54x faster than llm-compressor's. MLX models that keep experts on SSD reply more slowly than fully in-memory ones.
Training fixes: DPO and other padded training runs up to 3.8x faster with less memory on NVIDIA Ampere or newer, and a BF16 gpt-oss checkpoint loaded with load_in_4bit = True no longer gets random expert and router weights.

Terms in this piece · Glossary

mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.

Read the source github.com