ModSleuth maps the hidden model dependencies inside modern LLMs

Modern language models are partly built by other models. One model may write synthetic training examples, another may decide which documents are kept, and a third may judge answers during training or evaluation. ModSleuth, from researchers at Ai2 and UC Berkeley, is an AI agent built on Claude Code that traces these relationships. It reads a release's official technical reports, model and dataset cards and code repositories, records each upstream model or dataset together with the passage that documents it, and then repeats the process on those upstream artifacts, because their own dependencies are often described only in separate releases. The paper separates direct dependencies, which shape a model's weights or training data, from indirect ones such as models used only for evaluation.

The figures in Ai2's announcement are the paper's verified direct dependencies: 89 links to upstream models and 183 to datasets for Ai2's Olmo 3, and 273 and 560 for NVIDIA's Nemotron 3 Super. Each relationship was kept only after a second automated check, in which Claude Sonnet 4.6 searched the web and marked it verified. The tracing surfaced concrete problems. Olmo 3 trains on prompts derived from IFEval, an instruction-following benchmark it is also scored on, which makes that score harder to interpret. Some dependencies run several steps deep: the paper traces training data for DR Tulu, another Ai2 model, back to Claude Sonnet 3.7 through an intermediate system. In another chain, math data used for Hugging Face's SmolLM3 was selected by a classifier trained on labels from Llama-3-70B-Instruct, and the paper calls whether that brings the data under Llama 3's license an open interpretive question.

These counts are a floor. ModSleuth can only find what public documents declare, so private or undocumented dependencies are missing, and with no complete reference graph the authors cannot measure how much it misses. The agent and its verifier also both rely on Claude models and could share blind spots. Across four releases it recovered 1,060 verified relationships, more than three times the best fully automated alternative tested, and the authors are releasing the tool and the resulting graphs.