← All IntelClip / AI ToolsDeterministic graph loading vs. entity-extraction GraphRAG
From AI on Your Lakehouse: Context Comes in Shapes, Not Queries — Zach Blumenfeld, Neo4j · ≈47:55
“So it's all deterministic with the containment tree for one and then the ordering of the sections as well as the links.”
“the benefits of having a deterministic load like this is number one, it's going to be item potent.”
What’s in it
- Shows a deterministic alternative to LLM-driven GraphRAG graph building
- Explains how to encode document structure as containment trees plus ordering links
- Argues this lexical graph approach is faster and idempotent than entity extraction
Clip transcript
documents in a sense. Um and the data model that we're going to use in this case we're actually going to import the data into graph. Um I have a I have a quick question for everyone. So, everyone here is knows Graphrag. When they think of Graphrag, and maybe someone can raise your hand and I'll I'll pick on someone for a second. I want you to tell me what you think like the pro the the building the graph process is for Graph Frag. So, I don't know if anyone wants to wants to volunteer if they're familiar. You're going to raise your hand. >> Yes. Yeah. And they build they build the they build the links from from the So there's different ways, right? And and you said I think in in your when you talk about creating um a graph, you build the links from the data that's inside of the documents, right? And so graph rag oftentimes we have an entity extraction piece to it as well. So we can use an LLM to kind of say hey like extract the different parts the different vehicles from the from the graph and have them be separate entities and then have those links to like the original document chunks and all all of this stuff. And that's great and that works really well. Um here what I'm going to show you is something that's much more lightweight and I think maybe to the point that you were making um we're actually going to use a deterministic loading. So if you see here the way that this graph structure is going to work is you're going to have your library and then you're going to have this containment tree that reflects what we saw. And all this is really doing is it's breaking down, you see the folders, the document, and then these documents have sections and they can have multiple sections that will also nest underneath each other at different section levels. Um, and then you'll have um next section links. So you get a concept of ordering as well as linked to which uses in this case we're using um these named links uh sort of like you might see inside of an Obsidian vault a little bit um but you can also do it with hyperlinking and other things depending on depending on what your data looks like. So, it's all deterministic with the containment tree for one and then the ordering of the sections as well as the links. Um, and the benefits of having a deterministic load like this is number one, it's going to be item potent. Um, so like you're not relying on an LLM in the beginning. Um, it's often going to be a little bit faster. Um and so if you already have documents which have a lot of inherent structure to them and a lot of interlinking um sometimes just to get something up and running quickly it can be very beneficial to approach a graph like this where I'd say it's more of a lexical or document structure graph rather than like a full like entity uh you know extraction type of pipeline to create a graph.
Comments
Checking sign-in…
Loading comments…