Tracing Olmo's reasoning back to the text that produced it
- Source
- x.com
- Date
What kinds of training data shape different model capabilities? A @GeorgiaTech team used our fully open model flow to trace Olmo’s performance on social/general reasoning and social-science/STEM knowledge tests back to the types of text it trained on. 🧵 https://t.co/qrPoTrdj2B
@GeorgiaTech LLMs train on huge mixtures of text. When a model gets better at figuring out what a user is thinking or trying to do, it’s often hard to know whether that ability came from literature, technical writing, or some other part of its training data.
@GeorgiaTech The Georgia Tech team set out to test whether they could trace different model abilities back to the kinds of text that contributed to them. Olmo 3 made that possible because its full pretraining corpus, Dolma 3, is public.
@GeorgiaTech The team sampled millions of files from Dolma, then used influence functions – which estimate how much each file influenced a model’s answer – to see which types of files mattered most across tests of social/general reasoning, plus social-science & STEM knowledge.
Context
A Georgia Tech team used Ai2's openly released Olmo model and training data to investigate which kinds of writing supported its social reasoning. They estimated individual documents' influence on answers, then grouped those estimates by topic and format. Narrative and interpersonal writing stood out for questions about social situations. Ai2's account explains how access to the actual training corpus made the investigation possible.
The researchers also trained the model to forget selected influential literature documents. Its social-reasoning score fell more than when they targeted random documents from the same category. This provides partial experimental support for the attribution method and a way to test proposed data changes, without establishing that adding more literature will automatically improve reasoning. Research paper.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Checking sign-in…
Loading comments…