Vibeleaderboard
← All Intel
Intel / article

Claude-shaped science: a harness and field notes for agent-driven research

Source
anthropic.com
Author
Anthropic
Date

Context

Harvard physicist Matthew Schwartz's earlier experiment, written up as Vibe Physics, had Claude work like a graduate student, and he says getting a good paper out of it meant correcting every sentence. In this guest post for Anthropic he takes the opposite approach: stop asking the model to be a scientist and look for problems that fit what it already does well, namely broad knowledge, coding, mathematics and fast reading of papers and data. He calls these Claude-shaped problems, meaning questions in one field that a technique from mathematics, physics or computer science would settle if anyone there knew it existed.

The method grew out of his own field. He had Claude port scattered particle-physics code into one framework for a bootstrap technique, which narrows an answer through constraints and then pins it down with very high-precision numbers, so anyone can check the result by running scripts. The accumulated tools became BootLoops, an open-source that Schwartz owns, not Anthropic. He reports 30 integrals computed end to end within weeks, 15 of them never computed before. Because the same equations recur across science, Claude then spotted matching problems in ecology, population genetics, economics, linguistics and other fields.

The pattern Schwartz describes is that Claude was almost always technically correct, but the results were not very interesting until a domain expert redirected them. Solving a 20-year-old equation from neutral biodiversity theory showed that tree species on Panama's Barro Colorado Island turn over 4.5 times faster than the theory allows, which plant biologist James O'Dwyer expected many ecologists to shrug at, so he suggested studying what remains after removing the neutral prediction. Geneticist Michael Desai similarly pointed a mutation result toward pairs of nearby mutations, which led to evidence for gene conversion across 5.7 billion pairs. Other projects drifted beyond calculation, including a workflow that ran 4,452 economics papers' replication packages, described in an NBER working paper.

The operational half of the post covers running 36 manuscripts at once: one Claude Code session per project, a coordinating session and an adversarial referee session, plus recurring problems such as Claude declaring victory early, misjudging time and grinding through multiday calculations instead of building a faster tool. Schwartz notes that his list of additional highlights is still undergoing exploration and verification, and that he was a visiting researcher at Anthropic during the work.

Terms in this piece · Glossary
  • agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
  • LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Read the source www.anthropic.com
More from Anthropic
Recommended reads
Comments

Checking sign-in…

Loading comments…