NVIDIA Nemotron 3.5 Super VL 120B A12B Base BF16 released

This Base checkpoint is NVIDIA's Nemotron 3.5 Super VL captured partway through training. It has the same design as the post-trained release: 120 billion parameters with about 12 billion active per token, a vision encoder for images and video, and up to 1 million tokens of context. What it lacks is the later work. NVIDIA's training description for the finished model lists supervised fine-tuning, reinforcement learning and distillation from specialist teachers, and a final round of training for its multi-token prediction head, which drafts tokens for speculative decoding; the Base card lists only the earlier stages.

Those earlier stages are still substantial. The language backbone starts from Nemotron 3 Super Base and is distilled from Nemotron 3 Ultra Base, then trained on 100 billion tokens of Indic-language text, extended to 1 million tokens of context, and given 67 billion tokens of software-engineering data such as repositories, commit diffs and CI logs, before a vision encoder is attached. NVIDIA aims it at developers who want to run their own continued training, fine-tuning, reinforcement learning or distillation, and lists training frameworks (PyTorch, Megatron-LM, NeMo) rather than the serving recipes on the other cards.

Its benchmark table needs care: the card says the scores, such as 84.14 on MMLU and 91.74 on GSM8K, measure the text-only backbone before vision-language training, so they describe neither this exact checkpoint's visual abilities nor the finished assistant model. People who want a ready-to-use model are directed to the BF16 and NVFP4 releases.