OpenAI's Astra for Law Beats General Web-Search Agent on Legal Bench
- Source
- x.com
- Date
OpenAI launched Astra for Law today, and used the Vals Legal Research Bench to validate it. On OpenAI’s self-reported runs, Astra for Law beats GPT-6 Astra with web search at every price point: 90.0% vs 84.1% weighted rubric and 53.7% vs 39.6% all-pass.
Context
OpenAI launched Astra for Law, a legal research , and validated it using the Vals Legal Research Bench, a Vals AI built to grade legal research answers against a rubric and against a stricter all-or-nothing pass standard. Vals AI's post reports that on OpenAI's self-reported runs, Astra for Law beat GPT-6 Astra using general web search at every price point tested: 90.0% versus 84.1% on the weighted rubric, and 53.7% versus 39.6% on the stricter all-pass measure. These are OpenAI's own runs relayed by Vals AI, not an independent Vals AI , so the gap describes what OpenAI reported rather than a third-party result.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Checking sign-in…
Loading comments…