OptimismBench: Forecasting Bias and the Alignment Effect in Language Model Judgment
Source
Seonglae Cho, Adriano Koshiyama
Author
Seonglae Cho, Adriano Koshiyama
Date
Key takeaways · AI-distilled
The trick is inverted pairs: ask one model for P(success) and P(failure) on the same scenario. They should sum to 1, and the gap is a signed bias score needing no ground truth — which is exactly what calibrationHow well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.Full definition → metrics miss, since they average unsigned error.
Across 16 models from 8 providers, 14 tilt optimistic. Pessimism showed up only in Anthropic's frontier tier.
Post-training, not pretrainingThe first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.Full definition →, sets the direction of the tilt. Eleven matched base-versus-chat pairs across four model families shifted sign after alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition →, and different families shifted opposite ways.
The tilt belongs to the model, not the wording. It survived changes to prompt, temperature, perspective, and explicit self-debiasing, and across six languages model identity mattered 4.7x more than the language used.
Terms in this piece · Glossary
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
Why it matters
If LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → judges tilt optimistic in a measurable direction, every eval and scoring pipeline built on them inherits that tilt.
Key quotes
“When an LLM rates a startup's success at 70% but its failure at 15%, the missing 15 points expose a distortion no aggregate score flags.”
“Across 16 models from 8 providers, fourteen are optimistic; pessimism appears only in Anthropic's frontier tier.”
“Eleven matched base-versus-chat pairs across four families show post-training sets the sign of the bias, with opposite shifts in different families.”
“When alignment makes a model more helpful, it also tilts its probabilities; downstream pipelines inherit the tilt by default.”
articleEvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language ModelsXinning Li, Kemunto Ochwang'i, Aryasomayajula Ram Bharadwaj, Alexandra Souly, Robert Kirk
articleMisalignment Has a Personality: A Big Five Account of Emergent MisalignmentHasibur Rahman, Smit Desai
articleBias Audits Detect Bias but Disagree on Ranking: Evidence from Ten Instruments and Ten Frontier ModelsWilliam Guey, Pierrick Bougault, Wei Zhang, Vitor D. de Moura, Jos\'e O. Gomes