Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash

DoorDash search lead Raghav Saboo describes a pattern for using a large language model without running it on every search: do the expensive reasoning once, offline, and distill it into small, fast models that serve live queries. In the talk's description, a large model first produces relevance labels, judgments of whether a product matches what a shopper asked for, and those labels train the smaller retrieval and ranking models.

The motivating problem is that engagement and relevance can diverge. In the speaker's example, a ranker trained on what sells may show regular spaghetti for a gluten-free pasta search because spaghetti sells well. Training on explicit relevance labels gives the system a signal about that mismatch instead of relying only on orders.