Discussion between different AI models improved collective judgments in the experiments reported by The Wisdom of Artificial Deliberative Crowds. The researchers compared deliberation with simply combining independent answers, using models from three families across four domains, from visual numerical estimation to sports forecasting. They report lower collective error after deliberation, with individual judgments retaining the improvement afterward. Groups made from copies of one model did not receive the same benefit. Within these experiments, the useful ingredient was discussion among different models; adding more copies of the same model did not reproduce it.
Key takeaways · AI-distilled
The setup adapts a three-stage deliberation paradigm from human studies to models from three LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → families; in humans, averaging small groups' consensus estimates beats the classic wisdom of independent crowds.
Across all four domains, including sports forecasting against a real prediction market, deliberation reduced collective error beyond passive aggregation, and individual post-deliberation answers kept that gain.
The benefit required model diversity: groups made of clones of a single model did not gain from deliberating, which the authors read as diversity being an active ingredient.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleBeyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language ModelsLeonardo Ferreira, Gardenia Liu, Kaden Zheng
articleAgentic Data Engineering for LLMs: System Design and a Controlled Empirical StudyRuomeng Ding, Qianli Shen, Zhaoyang Han, Meixin Chen, Daoyuan Chen, Yaliang Li
articleDeepInstructor: An Agentic AI Instructor for Experience-Driven Idea EvaluationRongcan Pei, Fang Guo, Qinglin Qi, Qi Zhu, Yun Luo, Jianhao Yan, Minjun Zhu, Qiujie Xie, Dehong Zheng, Yue Zhang