Verifiable pXRD: Hill-climbing pXRD structure determination with LLM agents
- Source
- arxiv.org
- Author
- Samuel Chen, Jiaru Bai, Austin Henry Cheng, Varinia Bernales, Al\'an Aspuru-Guzik
- Date

Context
A general-purpose coding can work out many crystal structures from simulated powder X-ray diffraction patterns, and the best setup in this workshop paper from researchers at Stanford, the University of Toronto and collaborators recovered the exact structure in 69% of 200 test problems, up from 27% for the same model answering without tools. Powder X-ray diffraction, or pXRD, fires X-rays at a powdered sample and records how strongly they scatter at each angle. The pattern depends on how the atoms are arranged, and determining the structure means working backwards from the pattern to that arrangement, a step the authors call a bottleneck for self-driving materials laboratories. The task suits AI agents because an answer can be checked. Any proposed structure can be used to simulate the pattern it would produce, and the mismatch with the measured pattern, a residual called Rwp, shows how well it fits. A low residual means a candidate is consistent with the data, though it cannot rule out another structure that fits equally well.
The authors built pXRD-Bench, 200 simulated single-phase problems: half drawn from a standard set of known structures that existing AI structure generators were trained on, half constructed to be new to them. Answers count only if they match the hidden reference structure. OpenAI's GPT-5.6 Sol given only a prompt scored 27.0%, close to the best existing methods at 30.5% to 32.0% on a single try, two of which were given extra information about the cell. Running it as a coding agent with a scientific Python environment and crystal databases raised the score to 52.0%, adding standard crystallography programs (GSAS-II and DICVOL14) to 57.0%, and adding a compact package learned on 50 separate practice problems, a 139-word instruction plus a helper for one of the programs, to 69.0%. The gains cost compute: the coding agent used about 6.09 million tokens and 25.3 minutes per problem, against 161,293 tokens and 6.94 minutes for the plain answer. On a 20-problem subset, eight independent runs produced an exact answer for 18 problems, and simply picking each problem's candidate with the lowest residual put the exact answer first in all 18, so the pattern itself could identify the right answer without access to the reference.
Real samples are harder, because a synthesis product can contain several phases and nobody has the true answer. The authors tested 15 measured patterns from reactions of tungsten oxide with various metal compounds, 13 of which the published expert interpretations describe as mixtures. To learn without answers, a second agent with the same model studied solver failures on simulated mixtures whose contents it knew, and wrote guidance and a tool for building candidate structures from chemically similar known ones. With that package, the mean residual on the real patterns fell from 7.90% to 6.71%, better on 10 of 15, while the average run time rose from 27.0 to 50.2 minutes. On one strontium sample, the package let the agent build a Sr2WO5 phase that was missing from its reference library by adapting a Ba2ReO5 structure; across nine attempts it did so three times, while the unaided agent never did. The comparison that matters most cuts the other way. Simply giving the agent read-only access to the training setup itself, including its description of the sample family and its simulation and scoring code, produced the lowest mean residual, 5.98%, without finding the missing phase, and adding the learned package to that access found the phase in five of nine attempts but worsened the average fit to 6.49%.
The authors draw the conclusion that scientific helps an agent use what it already knows, while practice on checkable synthetic problems can produce specific new procedures, and that the second does not reliably improve average fitting once the first is in place. The real-sample evidence measures how well each fitted model reproduces the pattern, not whether its structures are correct. The expert interpretations are a comparison, not ground truth, and a lower residual can come from adding phases or flexible background terms. The simulated benchmark uses idealized patterns with background removed, half of its problems were seen by existing generators in training, and the real-sample panel is 15 patterns with three attempts each. The paper also cites Periodic Labs, which is building AI for materials discovery, as reporting better interpretation of experimental multiphase patterns by further training its model and spending more computation per problem. This paper leaves the model's weights untouched and changes only the agent's tools, instructions and number of attempts. Periodic Labs' plan to train models on feedback from physical experiments, including AI for analyzing lab measurements, is discussed in this Latent Space conversation.
- The authors frame powder X-ray diffraction structure determination as verifiable search: candidates are scored during by forward-simulated profile agreement and during evaluation by matching to a known reference structure.
- Their pXRD-Bench has 200 simulated single-phase problems covering in-distribution cases, hard and polymorphic structures, and controlled distribution shifts. Agent adaptation raised selected structural recovery from 27% to 69%.
- For real multiphase patterns, an evolver agent diagnoses solver failures on self-generated synthetic problems with known structures and builds persistent guidance and tools, so no collection of solved experimental measurements is needed.
- On 15 measured patterns from solid-state synthesis, the evolved package improved average profile agreement and recovered a phase whose structure was absent from the supplied reference library, per the abstract.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
articleVerification as an Architectural Layer for LLM Agents: A V-Model Design, and a Pilot Study of Its Deterministic CoreAli Afoud, Jie JW Wu
articleAuditable Human-Agent Teams for Autonomous X-ray ExperimentationGunik Luthra, Ronald J. Pandolfi, Benjamin T. Diroll, Ricky Ronquillo, Ruipeng Li, Christopher B. Murray, Marcus M. Noack, Kevin G. Yager, Esther H. R. Tsai
articleLabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI AgentsLei Liu, Yikun Zhang, Jialin Chen, Wanjia Zhao, Rex Ying, Wengong Jin, Hua Xu, James Zou, Tianyu Liu, Hongyu Zhao
Checking sign-in…
Loading comments…