Do large language models scrutinise what they review? A multimodal audit of scoring calibration, error detection, and author-identity effects

Explicit checking instructions improved two AI reviewers' error detection, but most planted errors still escaped notice. The study tested Qwen2.5-VL-72B and Pixtral-Large-124B, including 145 detectable errors inserted into 55 manuscripts. A verification instruction raised detection from 12.1% to 22.2%, leaving roughly four-fifths undetected. Within this experiment, asking for more careful review helped without making the reviews dependable checks of manuscript consistency. Methods and results.