Speed Up Image Generation with FLUX 3 Image Klein on NVIDIA Blackwell GPUs
Source
developer.nvidia.com
Author
Olivia Stoner
Date
Why it matters
Quantifies which optimizations speed up image generation on B200: NVFP4 (up to 1.62x), Cache-DiT (up to 1.94x with LPIPS 0.033), sparse attention, and 8-GPU parallelism. Helps you pick speedups against fidelity cost.
Key takeaways · AI-distilled
NVIDIA reports that combining torch.compile, NVFP4 with PrimsTS GEMMs, Sol-attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → and Cache-DiT in TensorRT LLM VisualGen made FLUX 3 Image Klein Base 3.46x faster for text-to-image and 4.45x for editing at 2048x2048 on one B200, versus uncompiled BF16.
Denoising dominates: at 2048x2048 on a B200 it was 96.8% of Base text-to-image latency, and dense BF16 attention took about 60% of DiT kernel time. That is why NVIDIA's gains come from quantizationShrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.Full definition → and sparse attention and step caching, not the text encoder or decoder.
Cache-DiT, which reuses transformerThe neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.Full definition → outputs across denoising steps without retraining, nearly halved Base latency at a 0.24 residual threshold (1.97x at 1024x1024 with LPIPS 0.038), but gave only 1.20x to 1.26x on the 16-step distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → model.
Quality costs are measured, not assumed: VC-Attention FP8 gave 1.18x for Base with LPIPS 0.086, beating cuDNN FP8's 1.08x. Sol-Attention at tau 1.0 gave 1.24x with 16 initial dense steps or 1.30x with 12, and NVIDIA found higher tau cut quality with little extra speed.
On eight B200 GPUs with CFG and Ulysses parallelism, Base text-to-image dropped from 14.90 to 3.00 seconds, but the distilled model only went from 2.73 to 1.08 seconds because about half of its eight-GPU time is outside denoising, which parallelism does not speed up.
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
