Vibeleaderboard
← All Intel
Intel / article

[AINews] AEF-1 standard emerges for Third Party Evaluators, as Xai, OpenAI, and Anthropic all cosign

Source
latent.space
Date
From the Daily Brief

OpenAI, Anthropic and xAI have cosigned AEF-1, a standard that builds on Anthropic's earlier Pacing the Frontier framework and commits each lab to give outside evaluators like METR sustained access to their training pipelines. Until now, third party assessment mostly meant testing a finished model after release, a snapshot that misses decisions made months earlier during pretraining and fine tuning. Under AEF-1, evaluators get something closer to employee level visibility, allowed to observe how safety practices are actually applied while a model is still being built, not just audit the output at the end. The move follows a run of incidents this month, including an Anthropic agent swarm that attacked its own grading system, that pushed all three labs toward public commitments on oversight. Whether the access proves substantive or ceremonial will depend on what evaluators are allowed to say publicly when they find problems.

Read the 2026-09-15 Brief →

Context

AEF-1 makes the conditions behind an independent AI part of the published record. The AI Evaluator Forum's standard asks evaluators to publish a checklist explaining their access, conflicts of interest, analytical independence, and disclosure rights. Conditions that cannot be met must also be documented.

That gives readers information needed to judge how freely evaluators could investigate and report their findings. Version 1 was updated on December 4, 2025, before the September 2026 discussion. For the coming months, the Forum's stated plans include shared norms for evaluators embedded inside AI developers, with strengthened conflict-of-interest provisions.

Terms in this piece · Glossary
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Read the source www.latent.space
Recommended reads
Comments

Checking sign-in…

Loading comments…