LongNovel: A Multi-Scale Benchmark for Hallucination Detection in Long-Context Novel Summarization
Source
Ruizhi Zhang, Jinwei Chen, Xiangju Lu, He Yan, Mo Yu, Junmin Zhu, Wei Zhang
Author
Ruizhi Zhang, Jinwei Chen, Xiangju Lu, He Yan, Mo Yu, Junmin Zhu, Wei Zhang
Date
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
Long context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → windows did not solve summarization hallucinationWhen a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.Full definition →. This benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → isolates eight hallucination types across inputs from 16k to 100k tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition →, letting you measure how your summarizer degrades as context grows rather than assuming it does not.
articleEvaluation & Hallucination Detection for Abstractive SummariesEugene Yan
articleOpen challenges in LLM researchChip Huyen
articleUnified Hallucination Fuzzing for Multimodal Large Language ModelsPengfei Zhou, Jiajun Song, Zhiwei Tang, Yixing Ma, Xiaopeng Peng, Donghui Si, Yuhang Xu, Huiqi Song, Yiyuan Miao, Yichen Qian, Weihua Chen, Wangbo Zhao, Bohan Zhuang, Jiasheng Tang, Yang You