Agreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical Judgments
Source
arxiv.org
Author
Octavian M. Machidon, Alina L. Machidon, Vojko Strahovnik, Mateja Centa Strahovnik, Jonas Miklav\v{c}i\v{c}, Marko Robnik \v{S}ikonja
Date
Why it matters
High label agreement with human annotators can hide divergent underlying reasoning — if your evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → only scores final answers, it is measuring less alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition → than you think.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
articleAgreement Overstates Evidence: Error Dependence in LLM Judge ConsensusElias Hossain, Niloofar Yousefi, Ser-Nam Lim
articlePosition: The Alignment Community is Unintentionally Building a Censor's ToolkitSarah Ball, Phil Hackemann
articleDiagnostic Foundation for Evaluating LLMs' Research Integrity as Co-ScientistsYash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi, Shivank Garg, Lin Li