What a crowdsourced game revealed about steering Olmo 3
Source
allenai.org
Date
Key takeaways · AI-distilled
After roughly 600 public submissions, the top 36 scoring entries in Steering Arena were all unreadable tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → strings (e.g. "Undert! AH :-) Rog Appl)") — the best plain-English entry ranked 37th, scoring about 2.7x lower than the top one.
The scorer measures how strongly text shifts Olmo 3's internal activity toward a prosocial pattern, regardless of whether the text reads as prosocial to a human — so players could optimize for what the model responds to internally rather than for human-legible wording.
One participant used an automated optimization method to search directly for higher-scoring token strings, with successive submissions sometimes differing by only a single token as the search converged.
The underlying evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → is built from 135 pairs of contrasting responses across 15 qualities (empathy, fairness, safety, privacy, respect, etc.), each pair holding the prompt constant while contrasting a more vs. less prosocial reply.
Because Olmo 3 documents its pretrainingThe first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.Full definition → and post-training data, researcher Soham Padia could trace whether a discovered prosocial direction originated in pretraining or a later fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → — a distinction he says is unanswerable on most closed models.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
Shows how full model openness enables crowdsourced evaluations that expose steering weaknesses a small internal research team would likely miss, a reusable method for other open-model teams.
