Vibeleaderboard
← All Intel
Intel / article

Early work on monitorability evaluations

Source
metr.org
Date
Why it matters

Gives an early method for measuring how well monitors catch covert behavior, which you need if you rely on monitors for agent oversight.

Terms in this piece · Glossary
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
  • LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Recommended reads
Comments

Checking sign-in…

Loading comments…