The Knowing-Saying Gap: When Probes See Errors that Confidence Misses

Detecting that an AI encountered corrupted information does not necessarily predict whether its final answer will be wrong. In multi-step arithmetic experiments, researchers used probes, simple classifiers that inspect a model's internal activity, to detect corrupted context with near-perfect accuracy. Those signals nevertheless failed to reliably distinguish correct final answers from incorrect ones. Asking models to express confidence in a structured format also failed to separate the outcomes. Attempts to intervene sometimes repaired errors but sometimes damaged correct reasoning. The finding separates two monitoring tasks: spotting suspicious information inside a computation and deciding whether the resulting answer needs correction.