Local fine-tuning and inference tooling now handles a 40B coding-agent mixture-of-expertsA model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.Full definition → in 24 GB and runs larger MoE models on a 32 GB Mac by keeping experts on SSD. REAP pruning exports smaller MoE models.
Key takeaways · AI-distilled
Unsloth now implements OpenAI's fine-tuning API (/v1/files and /v1/fine_tuning/jobs), so the OpenAI SDK can upload datasets and run supervised fine-tuning of local models.
Requests made with an Unsloth API key can no longer run Python or terminal tools unsandboxed; only the Unsloth UI can allow that. Installs also reject eMudhra's compromised emSign certificates, though the optional vLLM and SGLang engines still trust them.
Per Unsloth, Recursive's Flint has 2.5B active of 40B parameters and 256K context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →, scoring 89.7 on SWE-benchThe standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.Full definition → Pro v2 vs Qwen 3.6's 73.8. NVIDIA's Nemotron 3.5 Bolt E4B beats Gemma 4 E4B on 16 of 21 NVIDIA tests, most in tool useA model's ability to call external functions — run code, search the web, edit files — instead of only generating text.Full definition → and multi-turn chat.
REAP pruning trades some quality for size, and Unsloth says its Python reap_prune runs up to 54x faster than llm-compressor's. MLX models that keep experts on SSD reply more slowly than fully in-memory ones.
Training fixes: DPO and other padded training runs up to 3.8x faster with less memory on NVIDIA Ampere or newer, and a BF16 gpt-oss checkpoint loaded with load_in_4bit = True no longer gets random expert and router weights.
Terms in this piece · Glossary
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
