Auditable Human-Agent Teams for Autonomous X-ray Experimentation

A team of AI coding agents ran X-ray experiments at a national laboratory for more than 50 hours under one scientist's direction, and because every message was logged, the researchers could trace who proposed, decided and corrected what. The paper, led by researchers at Brookhaven National Laboratory, describes VISION-CC, a system built on Anthropic's Claude Code and used at the Complex Materials Scattering beamline of the National Synchrotron Light Source II, a particle accelerator that produces intense X-ray beams. A lead agent turns the scientist's goals into tasks for specialist agents: one controls the instrument through Bluesky, the beamline's existing control software, one processes the scattering images with established analysis code, and one runs gpCAM, a tool that fits a statistical model to the measurements so far and chooses where on the sample to measure next. The agents coordinate this validated scientific software rather than doing the physics themselves.

Over three beamtime sessions weeks apart, the team mapped how rod-shaped nanoparticles packed and lined up in self-assembled nanocrystal superlattices, while the scientist changed the objectives along the way. Setting up the agent team, including the closed measurement loop, took 52 minutes in the first experiment, 35 in the second and 16 in the third. The authors put a conventional basic setup at roughly six hours, which is the accessibility problem the work is meant to address.

The audit is the paper's main contribution. Language-model raters labeled every message in the human-agent and agent-agent transcripts and traced each decision back to the ideas and findings it relied on and forward to the code it changed. Across the three experiments, 61 to 67% of messages involved interpretation or judgment, and methodological and scientific contributions, as opposed to operational ones, rose from 46% in the setup-focused first experiment to 56 to 61% in the later two. Between 59 and 68% of error corrections involved more than one participant, and about half of all decisions drew on another participant's input. The scientist wrote under 1% of the text but most of the messages proposing ideas, while the lead agent sent most messages and both introduced and caught a large share of the errors. Some agent errors surfaced only when the human looked, such as measurement positions that stayed on a 0.05 mm grid despite a configured 0.01 mm step.

These figures come from one beamline, one scientist and one agent platform, and the AI raters agreed with human annotators on the purpose of each message only about as well as the human annotators agreed with each other. The authors present the results as something that may guide the design of multi-agent systems for science. In these sessions the scientist could disconnect the agents from the beamline at any time, and the paper names motor limits as the main safeguard against unsafe movement.