Index — Latest Intelligence

Intel

Chunkless RAG: the document outline is the retrieval index — Intel — VibeLeaderboard

Clip / AI Tools

Chunkless RAG: the document outline is the retrieval index

From Structuring the Unstructured - Cedric Clyburn, Red Hat · ≈14:06

“we're doing RAG but without having to use a chunker or embedding model or vector database, etc., etc. So the index ends up being the markdown outline of the document.”
AI Engineer

“we can allow an LLM to be able to pick the most relevant part of the document that is related to the user's question and pull that full text from the DocQuery document itself to try to answer that question for the user”
AI Engineer

“the final answer in one iteration was pulled from that source material without having to go in the vector database but instead search that docling document structure for the specific text”
AI Engineer

“the entire retrieval index typically would be thousands of vectors in a database where we would do semantic similarity”
AI Engineer

What’s in it

A genuinely different retrieval architecture — the agent reads a markdown section outline, picks the relevant section, and pulls its full text, removing the chunker, embeddings and vector store entirely.

Watch this moment ↗

Clip transcript

questioning and answering over our source data. And for this example, I want to show you what's known as chunkless RAG or agentic RAG using DocQuery. And now by starting off with our document outline, so processing a PDF with DocQuery like we just did, we can allow an LLM to be able to pick the most relevant part of the document that is related to the user's question and pull that full text from the DocQuery document itself to try to answer that question for the user. And this can run in a agentic loop. And what's really important here is that we're doing RAG but without having to use a chunker or embedding model or vector database, etc., etc. So the index ends up being the markdown outline of the document. Now when the user asks a question, the entire retrieval index typically would be thousands of vectors in a database where we would do semantic similarity to make sure, "Hey, these sections are similar to the user's question." But for us, what we're going to do is be able to see a markdown outline of the document with each section summary outline. So if the LLM is looking for something about getting started with DocQuery, well it can just pull from this reference of text to say that DocQuery can be installed from PyPI. And that's the entire retrieval index. So let's say we have a query, "What are the main AI models used in DocQuery?" Well we're setting up a rag agent here to be able to iterate on that specific question about five times. So, we see that there are 20 sections available when the user is asked the question, and we're going to search for that specific part of text that talks about the AI models and be able to determine, "Hey, is this relevant to answering the question?" And you can see here the final answer in one iteration was pulled from that source material without having to go in the vector database but instead search that docling document structure for the specific text. So, it's a quite interesting way to be able to answer users' questions through this chunklist retrieval augmented generation pattern.