vllm v0.33.0rc3 — v0.33.0

vLLM is open-source software for serving language models: it runs a model on GPUs and answers requests through an OpenAI-compatible API. The v0.33.0rc3 release is the third release candidate for version 0.33.0, a test build ahead of the final release, and its notes describe 927 commits from 389 contributors. The notes still contain a to-do reminding the maintainers to verify download URLs and tags after publishing, so they describe the planned contents of 0.33.0 rather than a final record.

The change most likely to affect an upgrade is the build platform. Release wheels and Docker images are now built only for CUDA 13.0, NVIDIA's GPU programming toolkit, and the CUDA 12.9 wheels and images are no longer published; the notes tell users to switch to the default CUDA 13.0 builds. vLLM also moves to PyTorch 2.14.0 and Triton 3.8.0. On AMD hardware the default ROCm image and wheels move to ROCm 10.0, with ROCm 7.2 builds still published under a separate tag. A deployment tied to CUDA 12.9 has to move to CUDA 13.0 to use the published builds. Every image is also now published under the shorter vllm/vllm name.

Several removals can break existing setups. Medusa and the MLP speculator, two older speculative-decoding methods, are gone, as is the legacy Ray compiled-graph executor, leaving RayExecutorV2 as the only Ray executor. Requests sent while generation is paused now receive an HTTP 503 error, the NIXL connector for transferring cached attention data between servers requires NIXL 1.5 or newer, and the Rust frontend now rejects unsupported serve flags and request fields instead of ignoring them. The LM Format Enforcer structured-output backend is deprecated, becoming opt-in in 0.34.0 and due for removal in 0.35.0.

The speedups are large in places but narrow, because each figure comes from the pull request that made the change, measured on particular models and GPUs. Speculative decoding has a small draft model propose several tokens that the main model then checks together, and it only pays off when enough guesses are accepted. In this release, the DFlash and DSpark drafters no longer reserve draft slots in the token budget, which took DeepSeek's V4.1-Flash on NVIDIA GB300 from 1,373 to 3,390 output tokens per second in that pull request's benchmark. FLy, a new opt-in verification policy that the notes call lossy, so output can differ from what the main model would produce alone, raised decode throughput 1.04 to 1.31 times on AMD MI355X and NVIDIA B300. Kimi-K3 decodes 1.5 to 2 times faster on GB300, but only in an opt-in latency mode. Whether any of this speeds up a given server depends on running the same model, hardware and settings.