Intel / article

How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo

Source: developer.nvidia.com
Author: Tanya Lenz
Date: Published Jul 14, 2026

Context

An automated experiment loop can handle model-training work while leaving the research objective and judgment with a person. In NVIDIA's worked example, OpenAI's Codex built a synthetic task requiring a Qwen vision- to count colored stars, measured its baseline, and ran supervised training on labeled examples. Reported exact accuracy rose from 25% to 96.875% on a fixed 64-example held-out . The successful method trained small adapter weights; an earlier reinforcement-learning attempt had not improved this task. The result demonstrates automated experimentation on this counting problem, with humans retaining responsibility for selecting and interpreting the work.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Key quotes

“With this approach, autonomous AI agents translate high-level goals into hypotheses, edit and test a real codebase, keep metric-improving ideas, and hand results back to the human researcher.”
Tanya Lenz

“In the examples provided in this post, Codex first brings up a full NeMo RL and NeMo Gym stack for a vision language model (VLM) RL training smoke test. It then conceptualizes and creates a novel NeMo Gym visual counting environment from scratch, and trains the Qwen3-VL-2B-Instruct model, increasing its accuracy from 25.0% to 96.9% on the task.”
Tanya Lenz

“The goal of autoresearch is not to remove the researcher from the loop, but instead to hand off the repetitive setup and iteration work to the agent. Researchers are still responsible for setting goals, reviewing milestones, steering strategy, and making the final decision.”
Tanya Lenz

“Context drift: Long sessions can pass through multiple context compaction cycles. Without durable memory, the agent may forget the objective, loaded skills, or stop rules.”
Tanya Lenz

“Set explicit budget constraints before starting a long-running autoresearch session, such as a time limit, GPU-hour budget, or maximum number of experiments. This keeps the agent focused on the highest-value hypotheses and prevents overly broad experiment sweeps.”
Tanya Lenz

Read the source developer.nvidia.com