Vibeleaderboard
← All Intel
Intel / article

Post-Training NVIDIA Nemotron to Increase Accuracy in Your Scientific Domain

Source
developer.nvidia.com
Author
Gautham Dharuman
Date

Context

NVIDIA has published a recipe for teaching one of its open models a specific piece of laboratory science, working from a spreadsheet of past experiments. The model it starts from is Nemotron 3.5 Super VL, the 120-billion-parameter model NVIDIA released in October (earlier Intel on that launch), and the worked example is drug metabolism: given a starting compound and the enzyme that acts on it, predict the molecule the body turns it into. Training a model to get better at a task needs something to judge its answers. The recipe's central move is to let the recorded experimental result do the judging.

That works because the answers are molecules, and molecules can be scored by a computation rather than by a person's opinion. Each row of experimental records becomes a question, with the measured product as the reference answer. The model reasons through the biotransformation and submits a structure written as a SMILES string, a text notation for a molecule's atoms and bonds. A scoring function then converts both the submitted and the recorded molecule into fingerprints, numerical summaries of their structural features, and measures the overlap between them as a Tanimoto similarity, a score from 0 to 1 where higher means more alike. That number becomes the reward the model is trained on, an arrangement NVIDIA calls reinforcement learning with verifiable rewards. No human preference labels are involved, and NVIDIA notes the same slot can hold a richer judge, such as a molecular simulator or a live autonomous lab, where one is available.

NVIDIA reports that on this task, at a 16,384- for the model's output, supervised improved the accuracy of the model, which the post calls Nemotron for Science, by 12% and reinforcement learning then lifted it a further 17%. The post does not say whether those are relative improvements or percentage-point changes, and it gives no starting accuracy, so the absolute gain cannot be read from it. It also does not report results on a second domain, so the claim that the pipeline generalizes to any task that can return a numeric score is a property of the software design rather than something measured here.

The post also describes an optional preliminary step for when a model struggles before reinforcement learning begins. A stronger model is run against the same environment, its step-by-step reasoning is scored by the same similarity measure, and only the attempts whose answer scored close enough to the recorded result are kept as training examples. In NVIDIA's run of that step, Anthropic's Claude Sonnet 5.5 produced 1,123 usable reasoning traces from 1,934 training prompts at a similarity threshold of 0.7 or higher. NVIDIA presents this warm start as something that typically speeds up convergence, not as a measured result; which supervised data produced the 12% figure above is not stated. For a research group with a table of experimental outcomes, the useful part of the post is the shape of the pipeline and its working code, with the caution NVIDIA itself ends on: start from a narrow task and validate the scoring method before expanding.

Key takeaways · AI-distilled
  • NVIDIA's recipe turns a CSV of experimental records into a reinforcement-learning environment: each row becomes a prompt and the measured result acts as the verifier, so the reward comes from the data rather than from human preference labels.
  • In the drug-metabolism example, the model predicts a metabolite's SMILES and is scored by Tanimoto similarity between Morgan fingerprints of the prediction and the reference, a 0 to 1 overlap score. NVIDIA notes richer verifiers such as simulators can replace it.
  • NVIDIA reports that on this task at a 16,384-token output budget, supervised fine-tuning raised Nemotron's accuracy by 12% and RL post-training added a further 17%.
  • For an optional warm start, Claude Sonnet 5.5 generated reasoning traces on 1,934 training prompts, and only the 1,123 traces whose answers reached Tanimoto 0.7 or higher were kept as SFT data before RL.
  • NVIDIA recommends enabling importance-sampling correction to reduce mismatch between the vLLM rollout engine and the Megatron training engine, which it calls particularly important for models.
Terms in this piece · Glossary
  • multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
  • token budget — A cap on how many tokens a task, session, or agent run may consume — the practical control on both cost and how long an agent will grind.
  • fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
  • mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
Read the source developer.nvidia.com
Recommended reads
Comments

Checking sign-in…

Loading comments…