Intel / repo

unsloth v0.1.50-beta — Introducing AMD support

Source: github.com
Author: Etherll
Date: Published Jul 20, 2026

Context

Unsloth, software for training and running language models locally, added AMD GPU support in v0.1.50-beta. The release covers Windows, Windows Subsystem for Linux, and Linux, with changes to GPU detection, installation, and compatibility with AMD's ROCm software.

The release also adds placement controls for fitting models onto available hardware. Users can choose which GPUs and model layers to use, or move the expert layers of a model into system memory. That provides another option when graphics memory is tight. The release does not establish how fast each such configuration will run.

Key takeaways · AI-distilled

Dynamic NVFP4 keeps accuracy-sensitive layers in FP8 or BF16 and runs everything else at 4-bit weights and activations. That mix is how it claims up to 2.5x faster inference on Blackwell without the usual quality drop.
FP8 KV caches roughly double usable at the same memory budget, because the cache, not the weights, is what grows with sequence length.
Agent safety here is a permission ladder rather than a toggle: tool calls can be set to Ask, Approve for me, Off, or Full access, and agents run in an isolated workspace so an unintended call stays contained.

Terms in this piece · Glossary

mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Key quotes

“Starting today, our AMD collaboration, custom Triton kernels, and math algorithms enables you to train and run 500+ models across AMD's Radeon, Instinct, Ryzen and data center GPUs, up to 2× faster with 70% less VRAM and no accuracy loss.”

“Unsloth Dynamic NVFP4 keeps accuracy-sensitive layers in FP8 or BF16 while running the rest in W4A4.”

Read the source github.com