Intel / article

NVIDIA releases Nemotron 3.5 Bolt, a 4GB open model for local agents

Source: research.nvidia.com
Author: NVIDIA AI
Date: Published Oct 23, 2026

Context

Nemotron 3.5 Bolt E4B is a small open model from NVIDIA meant to run a talking, tool-using assistant on local hardware in about 4GB of memory. It is not trained from scratch: according to NVIDIA's announcement, it starts from Google's Gemma 4 E4B and adds NVIDIA's own post-training for conversation, instruction following and multi-turn tool calls. The intended uses are voice and device assistants, game characters and robot interfaces running on GeForce RTX cards, Jetson boards and NVIDIA's Spark machines. NVIDIA argues these agents need a different mix of skills from coding agents: holding a persona, deciding when to call a tool instead of talking, obeying behavioral constraints and staying in the app or world state they are given.

NVIDIA's headline claim is roughly 40% higher average scores than its previous small model, Nemotron 3 Nano 4B, across the benchmarks it chose, and roughly 10% higher than the other peers it tested. The full tables show a narrower picture against those peers. Bolt leads on several conversation and instruction tests, such as IHEval, which checks whether a model follows the higher-priority instruction when instructions conflict (84.8 against 60.6 for Qwen 3.5 4B and 67.0 for Gemma 4 E4B). But Qwen 3.5 4B scores higher on Tau2-Bench, a multi-turn customer-task test (75.1 against 55.1), and on the long-context tests LongBench v2 and RULER, while Gemma 4 E4B edges it on EQ-Bench v3. Two of the benchmarks NVIDIA used, Brevity Bench and When2Call-MT, are its own and have not yet been released.

The other selling point is restraint. NVIDIA says Bolt produces up to about 80% fewer output tokens than peers in reasoning-enabled comparisons, which reduces decoding work and can shorten responses, and that it added length penalties during training because answers grew longer as post-training went on. Its measured speeds show where that matters: on an RTX 5070 the model used 3.5GB and returned its first token in 548 milliseconds on a 3,000-token prompt, while on an 8GB Jetson Orin Nano the same test took about 5.8 seconds to the first token. All of these figures are NVIDIA's own measurements. The accompanying technical report PDF and the demo video in NVIDIA's post on X could not be read for this explanation, so it rests on the announcement page.

Key takeaways · AI-distilled

Nemotron 3.5 Bolt E4B is Gemma 4 E4B with Nemotron post-training, aimed at local conversational agents. NVIDIA reports about 40% higher average scores than its Nemotron 3 Nano 4B and about 10% above peers on its chosen benchmarks.
The gains are uneven: Bolt leads its comparison set on IHEval instruction-hierarchy (84.8 vs 67.0 for Gemma 4 E4B) and RoleBench, but Qwen 3.5 4B beats it on Tau2-Bench (75.1 vs 55.1), LongBench v2 and RULER 64K.
Training mixed RLVR and , ran every sample in both reasoning and non-reasoning modes with a 2:1 non-reasoning ratio per batch, and raised length penalties when traces grew longer. The model was then quantized to Q4_K_M with imatrix on about 1M rollout tokens.
NVIDIA measured about 3.5GB VRAM and 548ms time to first token on an RTX 5070 at 3K input tokens, versus nearly 6 seconds on Jetson Orin Nano. A bundled DFlash parallel drafter gives up to 21% more throughput than Gemma 4 E4B's MTP drafter.

Terms in this piece · Glossary

grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
RLHF — Reinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.

Read the source research.nvidia.com