[Paper Club] DocETL: Agentic Query Rewriting + Eval for Complex Document Processing w Shreya Shankar
Source
youtube.com
Author
Latent Space
Date
Context
DocETL, a research system for running language-model operations over large collections of documents, treats a difficult document-processing request as a workflow that can be reorganized. In the paper discussion, its author explains that asking a model to extract many fields from a long document in one operation may fail because the request is too demanding.
A rewrite changes the processing pipeline: it can split the material, perform narrower extraction steps and combine their results. The document itself is not what gets rewritten. This gives developers a structured way to change how an extraction task is carried out, rather than relying entirely on revisions to one large prompt.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.