Universal Pathologies, Conditional Consequences: A Triple-Robustness Analysis of RAG for Multi-Hop Traceability
Source
Meftun Akarsu, Burak Ozdemir
Author
Meftun Akarsu, Burak Ozdemir
Date
Terms in this piece · Glossary
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
Why it matters
Anyone choosing between GraphRAG and vector RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition → is usually generalizing from one corpus; this shows which failure travels and which does not, with the experimental scale to back it.
Key quotes
“GraphRAG underperforms vector RAG on citation precision in many reports, but where and why have remained corpus-bound.”
“Over-citation is architecturally universal: GraphRAG emits 11-15 IDs per answer at citation precision 0.12-0.23 and retrieval recall 0.68-0.87 across all three settings.”
“Its faithfulness consequence is corpus-conditional: in typed-edge DO-178C, GraphRAG faithfulness collapses 74%->40% across hops; on Wikipedia chains the same pipeline rises 42%->58% because over-cited paragraphs remain topically supporting.”
“We argue triple-robustness is the minimum bar for trustworthy RAG architecture claims.”
articleVITAL-RAG: Invariance Race for Context Allocation in Coding AgentsZijian Lu, Yonghua Lu, Mingcai Chen, Yiping Zuo, Xin He, Weijun Wang, Weibei Fan
articleWhen Knowledge Changes: Metamorphic Testing of RAG Systems with MutationsJinhan Kim, Samuele Pasini, Paolo Tonella
articleBM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation ParadigmsPengyu Wang, Benfeng Xu, Shaohan Wang, Xin Zeng, Huarui Wu, Lei Zhang, Licheng Zhang