Serving 2 Million Models Without Melting: Scaling the Hugging Face Hub — Arek Borucki, Hugging Face

Hugging Face keeps its model catalog searchable by preparing information for search before anyone types a query. The company hosts AI models and datasets, but its search database stores their descriptions and other metadata separately from the model files. In its infrastructure talk, the team describes a separate copy of that metadata optimized for browsing and listings.

When a model is added, its name is split into searchable pieces. An autocomplete index searches those prepared pieces, replacing the earlier pattern-matching queries that became slow as the catalog grew. The reported improvement comes from changing how information is organized and retrieved, while keeping large model files in separate storage.

“At 20,000 models, any query is fast. Even without an index, trust me, no one would notice. At 3 million, same approach breaks.”

Arek Borucki

“P99 is much more important than P50 and we are paying a lots of attention to P99”

Arek Borucki

“MongoDB does not store the models themselves. It stores everything about the models.”

Arek Borucki

“We tokenize model names on insert time, not at query time.”

Arek Borucki

“The pattern is simple. Primary should focus on what only primary can do. Anything else can be pushed to different machines.”

Arek Borucki