The setup adapts a three-stage deliberation paradigm from human studies to models from three LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → families; in humans, averaging small groups' consensus estimates beats the classic wisdom of independent crowds.
Across all four domains, including sports forecasting against a real prediction market, deliberation reduced collective error beyond passive aggregation, and individual post-deliberation answers kept that gain.
The benefit required model diversity: groups made of clones of a single model did not gain from deliberating, which the authors read as diversity being an active ingredient.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
Shows structured multi-agentUsing several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.Full definition → deliberation, not just independent voting, reduces error across domains including detecting a hidden malicious AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →, a usable design pattern for multi-agent evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → and safety pipelines.
articleDeepInstructor: An Agentic AI Instructor for Experience-Driven Idea EvaluationRongcan Pei, Fang Guo, Qinglin Qi, Qi Zhu, Yun Luo, Jianhao Yan, Minjun Zhu, Qiujie Xie, Dehong Zheng, Yue Zhang
articleAgreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical JudgmentsOctavian M. Machidon, Alina L. Machidon, Vojko Strahovnik, Mateja Centa Strahovnik, Jonas Miklav\v{c}i\v{c}, Marko Robnik \v{S}ikonja
articleWhy Better Models Can Create Riskier Systems: Evidence from LLM Agents in Financial MarketsJillian Ross, Eric So, Zoe De Simone, Charles Pozniak, Andrew W. Lo