Intel / repo

unsloth v0.1.914-beta — Glass Theme, Personalize & Faster Generation

Source: github.com
Author: shimmyshimmer
Date: Published Nov 3, 2026

Context

The most consequential change in this Unsloth release is a large speedup in chat generation, but it applies to a narrow set of setups. Unsloth is an open-source toolkit for fine-tuning and running language models on your own hardware, with an app called Studio for chatting with and training models. The release notes say supported single-GPU Llama and Qwen3 models now generate faster in Studio's Transformers backend on NVIDIA GPUs under Linux or WSL, citing an A100 test in which a 4-bit Qwen3-8B went from 25.7 to 99.1 tokens per second.

The pull request behind that figure shows where it comes from: Studio now turns on CUDA graphs by default when generating text. A CUDA graph records the sequence of GPU operations for one generation step and replays it, rather than having the CPU issue each operation again for every token, a technique that typically helps most when that per-token overhead dominates, as with small batches. The PR reports speedups of roughly 2 to 3.9 times for most of the small and 8-billion-parameter Llama and Qwen3 models it tested on T4, L4 and A100 GPUs, with identical output tokens and unchanged time to the first token. The exception was a 1-billion-parameter Llama at 16-bit precision, which gained about 1.65 times on an L4 and a noisy 1.0 to 1.4 times on a T4. It also lists what stays on the old path: other model families such as Gemma, Mistral and models, multi-GPU or offloaded setups, AMD, Intel and Apple hardware, CPUs, and native Windows unless forced on. H100 and B200 GPUs were not measured, and neither the release notes nor the PR says anything about GGUF models, which Studio runs through llama.cpp rather than the Transformers backend.

The rest of the release is broad but incremental. Text models can now be fine-tuned on a CPU from Studio's Train page when no GPU is available, with no speed figures given, multi-GPU training gains more parallelism options, and the app adds Glass visual themes and other personalization. The notes also claim Unsloth's DeepSeek-V4.1 GGUF files rank strongest on several agent benchmarks, which is the project's own analysis and was not checked here.

Key takeaways · AI-distilled

Unsloth's Transformers backend now generates faster for supported single-GPU Llama and Qwen3 models on NVIDIA GPUs under Linux or WSL; in Unsloth's A100 test, 4-bit Qwen3-8B went from 25.7 to 99.1 tokens/s, a 3.9x speedup.
SDXL now renders under a 4 GiB VRAM budget, tested on T4 and L4 GPUs, and audio up to four hours long can be transcribed with audio.cpp, subject to upload and storage limits.
Supported non- models larger than one GPU can now load by placing only each GPU's assigned pipeline layers, and UNSLOTH_PARALLEL configures supported combinations of data, tensor, pipeline and parallelism.
unsloth.tile_mlp(model) reduces MLP activation memory during training, and audio vision models such as Gemma 3n and Gemma 4 can now be fine-tuned on datasets that mix audio and images.
On Mac, the experimental Fast Verify option speeds generation for supported MLX models, with the stated caveat that replies may differ slightly.

Terms in this piece · Glossary

mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.

Read the source github.com