Extrinsic Hallucinations in LLMs
- Source
- lilianweng.github.io
- Author
- Lilian Weng
- Date
Context
An AI answer can be checked and revised against retrieved evidence after it has been written. Lilian Weng's survey of fabricated factual claims explains RARR, a research-and-revision method that turns statements in an answer into search queries and retrieves supporting material.
A comparison step looks for disagreement between the draft and the retrieved evidence. An editing step then changes unsupported content while trying to preserve the original answer's intent, and the output includes an attribution report. This makes verification a distinct operation in the writing process. Its must consider both whether sources support the revised claims and whether the revision still preserves what the answer was trying to say.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
- hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
- grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
“To avoid hallucination, LLMs need to be (1) factual and (2) acknowledge not knowing the answer when applicable.”
“Equally importantly, when the model does not know about a fact, it should say so.”
articleOpen challenges in LLM researchChip Huyen
articleUnified Hallucination Fuzzing for Multimodal Large Language ModelsPengfei Zhou, Jiajun Song, Zhiwei Tang, Yixing Ma, Xiaopeng Peng, Donghui Si, Yuhang Xu, Huiqi Song, Yiyuan Miao, Yichen Qian, Weihua Chen, Wangbo Zhao, Bohan Zhuang, Jiasheng Tang, Yang You
articleClosed-World Resolution Against Tool Hallucination in LLM AgentsLaxmipriya Ganesh Iyer
Checking sign-in…
Loading comments…