A New Initiative For Developing Third Party Model Evaluations

Anthropic's July 2024 evaluation initiative sought to fund outside organizations to build better tests of AI capabilities and risks. The announcement says good evaluations are hard to make: a test can end up measuring memorization because its data was in the model's training set, and even strong results on a safety evaluation often turn out not to be enough to worry experts.

Its proposed remedies include tasks built with subject-matter experts, expert baselines for comparison, and uplift trials, which compare task performance between groups with and without model access. Those approaches ask whether AI changes what people can accomplish. The initiative was a call for proposals to improve the evidence used in safety decisions; the announcement does not report any resulting evaluations.