Large Language Models Threaten Double-blind Review
Source
Bulambo Mwendelwa Gloire, Prasenjit Mitra
Author
Bulambo Mwendelwa Gloire, Prasenjit Mitra
Published
Why it matters
If author identity leaks from problem framing alone, anonymized review as currently practiced is already compromised — directly relevant to anyone submitting to or reviewing for ML venues.
Transcript
Double blind peer review serves as the scientific community primary defense against status and affiliation bias. Its effectiveness rests on the assumption that anonymized manuscripts convey scientific merit without revealing their authors. While authorship can often be recovered using citation networks or stylistic markers, we show that this assumption is increasingly fragile in the presence of large language models (LLMs). Using only titles and abstracts from papers published after model training, we find that LLMs collapse anonymity more efficiently than humans, with belief concentrating onto a small subset of plausible authors drawn from pools of five domain expert candidates. This vulnerability persists even when stylistic and bibliographic cues are excluded, indicating that stable patterns in problem framing and research focus function as latent conceptual signatures of authorship. Together, these findings indicate that double blind review is vulnerable to automated semantic inference, necessitating a revaluation of how anonymity and fairness are maintained in an AI augmented research ecosystem.