Experiment with Qwen3.8-Flash-Next 176B Model on NVIDIA GB300 NVL72 for Agentic Coding
Source
developer.nvidia.com
Author
Michelle Horton
Date
Context
NVIDIA's report on serving Qwen3.8-Flash-Next gives two different measures of speed: more than 16,000 generated text tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → per second per GPU in aggregate, and more than 200 per second per user. The deployment article places those figures on GB300 NVL72, a rack-scale system containing 72 GPUs.
Aggregate throughput describes serving capacity across users; the per-user figure is closer to the pace an individual experiences. Keeping them separate helps interpret the hardware claim without mistaking combined output for one exceptionally fast response. These are NVIDIA's reported results for that deployment.
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
