• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Three labs hitting the same failure through the same third-party testing vendor makes this a systemic problem with eval infrastructure rather than an isolated mistake.

• articleThird-party cyber evaluations involving OpenAI modelssimonwillison.net
• articleThird-party cyber evaluations involving OpenAI modelsSimon Willison
• article[AINews] AI Cybersecurity becomes top of mindLatent Space

Sign in to comment.

Loading comments…