Asking additional AI models to review an answer can introduce errors as well as repair them. COMED addresses that trade-off with a controller that decides whether an initial answer needs more collaboration. It considers consistency in the first model's answers, how decisively the model-selection system chose it, and a lightweight check from another model.
The authors report improvements across all 16 evaluated open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → settings while invoking fewer models and generating less text than an approach that collaborates on every query. The useful finding is that collaboration can be selective: retaining a sufficiently confident answer avoids extra work and the opportunity for other models to turn it into a wrong one.
Key takeaways · AI-distilled
The authors find collaboration is non-monotonic: peer models can rescue failures no single model solves, but can also corrupt answers that were initially correct.
COMED combines anchor self-consistency, router margin and a lightweight peer probe to accept confident answers, verify ambiguous ones, and escalate only when collaboration looks beneficial.
Across 16 open-weight settings it improved both fixed and routed anchors, gaining up to 10.7 points on MedQA while invoking fewer models and decoding fewer tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → than dense collaboration.
With frontier models on HLE, COMED lifted GPT-5.5 from 23.1% to 28.1%, beating dense collaboration.
Terms in this piece · Glossary
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
articleWhy Better Models Can Create Riskier Systems: Evidence from LLM Agents in Financial MarketsJillian Ross, Eric So, Zoe De Simone, Charles Pozniak, Andrew W. Lo
articleCaught in the Story: Narrative Captivity in Multi-turn LLMs ConversationYuhe Wu, Guangyu Wang, Yujie Chen, Jiatong Zhang, Yuran Chen, Yutong Zhang, Xiyin Cheng, Wenpeng Cao, Zhuang Liu, Guang Zhang
articleAgentic Data Engineering for LLMs: System Design and a Controlled Empirical StudyRuomeng Ding, Qianli Shen, Zhaoyang Han, Meixin Chen, Daoyuan Chen, Yaliang Li