LLMs are no longer created w/ human data alone. They rely on other models to generate & filter data, evaluate outputs, & guide dev work.
So what is a modern LLM built on? Olmo 3 → 89 model + 183 dataset dependencies; Nemotron 3 → 273 + 560
We made ModSleuth to trace this. 🧵
Modern LLM dependencies are scattered, recursive, & hard to see. So how do we even find them all?
ModSleuth helps by reading papers, model & dataset cards, code configs, & upstream artifacts, then reconstructing a model's “family tree.”
Some dependency chains go 8 hops deep—a web of models & data that contributed to an LLM’s core.
Turns out AI supply chains may be more tangled than we thought.
A model's lineage is broader than its training data, & every step can affect what – and how – the final model learns.
Without provenance, it's harder to know where model dependencies came from, whether benchmark scores are accurate, & which upstream licenses/terms may apply.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
Model provenance runs many hops deep through judges, filters, and data generators. ModSleuth maps that graph, surfacing inherited license terms and train/evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → coupling that manual review of a model card will not catch.
Read the full thread on X ↗
