Misalignment Has a Personality: A Big Five Account of Emergent Misalignment
Source
Hasibur Rahman, Smit Desai
Author
Hasibur Rahman, Smit Desai
Date
Terms in this piece · Glossary
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
Corpora that induce broad misalignment share a measurable Big Five signature — lower agreeableness and conscientiousness, higher extraversion and neuroticism — detectable in the training data before fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → imprints it on the model.
articleOptimismBench: Forecasting Bias and the Alignment Effect in Language Model JudgmentSeonglae Cho, Adriano Koshiyama
articleConstitutional Midtraining: Content Presence Drives Alignment GainsDesiree Cho, Cameron Tice, Bernie Hogan, Hunar Batra, Puria Radmard, Jun Zhao, Nigel Shadbolt