The Knowing-Saying Gap: When Probes See Errors that Confidence Misses
Source
Jyotin Goel, Ipshita Bandyopadhyay, Justin Shenk
Author
Jyotin Goel, Ipshita Bandyopadhyay, Justin Shenk
Date
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Anyone building runtime monitoring for LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → or AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → pipelines learns that probe signals and verbalized confidence are not interchangeable, and that the deployable choice is error-type-aware routing rather than one universal intervention.
articleOn the Robustness of LLMs' Internal Representation of Code CorrectnessFrancisco Ribeiro, Sohaila Abdulsattar, Renata Gonzalez, Mahmoud Kassem, Sarah Nadi
articleConfidence Estimation for Financial Vision-Language Models in Chart and Document UnderstandingReza Khanmohammadi, Simerjot Kaur, Charese H. Smiley, Ivan Brugere, Mohammad M. Ghassemi