Intel / repo

unsloth v0.1.912-beta — Trick or Treat + Optimizations

Source: github.com
Author: danielhanchen
Date: Published Oct 28, 2026

Context

The Halloween themes are a seasonal skin; the substantive change in this Unsloth release is multi- batching. Unsloth makes open-source software for AI models, and its app keeps a queue of fine-tuning jobs. The jobs this change affects use LoRA, a method that leaves the original model frozen and trains a small add-on set of weights, called an adapter, for each task. Until now each queued job ran on its own. When several compatible jobs are waiting, the app now trains up to four together, each with its own adapter on one shared copy of the base model.

The saving comes from not repeating identical work. The release notes say new GPU kernels run the frozen base layer once for the whole batch, while each adapter learns only from its own job's data with its own loss and learning rate. The pull request defines compatible narrowly: jobs must use the same base model, precision, sequence length, optimizer, chat template and several other training settings, though their datasets, learning rates and epochs may differ. Some jobs always run alone, including multi-GPU runs, models and jobs that resume from a checkpoint. The developers' GPU tests compared each batched job's training loss with the same job run alone and found them close.

The only speed figure is narrow. With four adapters, one 4,096 by 4,096 layer's forward and backward pass ran 1.52 times faster than training the adapters one at a time, on an A100 GPU in bf16 precision at rank 16. That is a single-layer benchmark, not a measured end-to-end job time, so the real gain depends on how much of a job is spent in that work. The release also fixes a failure with practical consequences: local models now skip tool calls that hit the output limit, after cut-off rewrites had shrunk a 175-line test file to as few as six lines.

Key takeaways · AI-distilled

Multi-LoRA batching uses new Triton kernels that run the shared base layer once per batch while each adapter trains only on its own , with its own loss and learning rate. Unsloth measured 1.52x faster forward and backward with 4 adapters on an A100.
Local models now drop tool calls cut off by the output limit, so a half-written file write cannot overwrite a file. Unsloth says truncated rewrites had previously shrunk a 175-line test file to as few as 6 lines on an RTX 3090.
Jobs submitted while the GPU is busy now queue in order and can be cancelled one at a time, and a finished job's fine_tuned_model name loads its adapter for chat. /v1/videos requests also queue during a render, and matching image requests are batched.
Unsloth reports that preparing a 1,000-row chat dataset for Gemma 3 4B took 1.75 s instead of 6.6 s in its test, and that with 497 cached models the model list uses about a third of the CPU time it did.
New chat training setups turn on sequence packing and assistant-only completions where the model and data support them. Gemma 4 E2B and E4B now keep a dataset's thinking text in training, and local JSON or JSONL datasets load even when later rows add keys.

Terms in this piece · Glossary

LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Read the source github.com