Scaling Document Processing Across the Harvey Platform
Source
x.com
Date
Why it matters
Anyone running retrieval over customer documents hits the same wall Harvey did: the naive single-pipeline design breaks unevenly across extraction, chunking, embedding and indexing.
This post shows how one of the largest production legal-AI deployments split those stages to hold latency and reliability through a 26x volume increase.
Author: Tom McCormick, Jin Zhang, Shunrang Cao, Jinfeng Zhuang, Anna Zhang, Adam Shen, Gary Lam
At Harvey, document processing sits underneath Vault, Assistant, and every workflow that interacts with a customer's own data.
Unlike many other AI products, almost every query from a lawyer involves documents.
These queries often range from a single file to hundreds or thousands of documents at once.
Contracts, closing binders, deposition transcripts, email archives, scanned PDFs, and entire data-room exports all pass through the same path: fetch the file, extract the content, split it into useful pieces, and make it searchable.
A year ago, that platform handled just under one million documents in a busy week.
Users rarely think about document processing when it is working.
Over the last 12 months, our work has been focused on keeping it that way while weekly volume moved from hundreds of thousands of documents to tens of millions.
The main lesson was simple: At this scale, document processing stops being one pipeline and starts behaving like several different systems.
Extraction, chunking, embedding, indexing, storage, and retrieval all break in different ways.
We had to split the work along those lines.
Below, we share a look into some of the work we did to stay reliable and low latency while scaling incredibly fast, plus a glimpse into the future and some of what we have planned for growth.
Measured by completed-processing time across customer workspaces, the scale curve looked roughly like this:
At current scale, full weeks are regularly in the tens of millions of documents.
The latest complete week in the measurement window averaged about 3.5 million documents per day.
Compared with a year ago, that is 26x more documents and 39x more data.
Caption: Rounded quarterly snapshots.
Raw weekly data has normal operational and holiday volatility; this view emphasizes the underlying scale curve.
