Gibberish wins: a game exposes the limits of prosocial-behavior scoring

• Steering Arena's scorer came from showing Olmo 3 pairs of more and less prosocial answers to the same prompts and finding the internal pattern that separated them, then using that pattern to rate how strongly new prompts pushed the model.
• After about 600 submissions, all of the top 36 entries were strings no person would normally write, such as "Undert! AH :-) Rog Appl)", according to Ai2.
• Padia worked with Olmo 3-32B through the NSF-supported National Deep Inference Fabric instead of hosting it himself. Ai2 says releasing more than weights let him investigate why nonsense scored well and share the measurements behind the scores.

• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

A probe-based behavior score can reward strings no human would write. Treat activation-derived eval metrics as gameable and check them against human-readable outputs.

• postTraining an LLM by showing it answers people prefer – and ones they don’t – can…
• postAi2 uses psychometrics to audit what LLM safety benchmarks measure
• postMolmoMotion forecasts object motion in 3D from a few frames and an instruction
• postA substitution test shows LLMs answer drug questions from name patterns

• articleWhat a crowdsourced game revealed about steering Olmo 3allenai.org
• articleHow a Georgia Tech team used the open Olmo stack to trace social reasoningallenai.org
• articleGames as Model Eval: 1-Click Deploy AI Town on Fly.iofly.io

Checking sign-in…

Loading comments…