StepFun Step 5 Preview: 600B open-weight agent model with NVFP4 recipes
Source
huggingface.co
Author
NVIDIA AI
Date
Why it matters
A 600B-class agentic open-weight model with a 1M context can be served on four B300 GPUs via vLLM or SGLang today, with an NVFP4 checkpoint to cut memory. Custom license terms and unseen benchmark figures need checking before adoption.
Key takeaways · AI-distilled
Step 5 Preview is a sparse mixture-of-expertsA model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.Full definition → with 600B total and 27B active parameters and up to 1M tokens of context and output, which StepFun pitches at repository-level coding, long-horizon tool useA model's ability to call external functions — run code, search the web, edit files — instead of only generating text.Full definition → and finance work.
StepFun's example BF16 layouts use eight B300 GPUs or two 8-GPU H200 nodes versus four B300s for FP8, and it warns they are a starting point for loading the weights that does not guarantee memory for a full 1M-token request.
On B300, the expert-parallel vLLM modes use the MegaMoE backend, which requires expert parallelism, while H200 uses the default MoE backend; the vLLM commands enable MTP speculative decodingA speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.Full definition → with 3 tokens and SGLang uses EAGLE.
Setting --dtype bfloat16 on the FP8 checkpoint does not convert its quantizationShrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.Full definition → weights, so BF16 needs the separate checkpoint, and StepFun says its BF16 Transformers shard workflow is not yet validated end to end.
Terms in this piece · Glossary
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
