Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash

• LLM-generated relevance labels counter rankers trained on engagement, which can show regular spaghetti for a gluten-free pasta search because it sells well; DoorDash says the labels improved retrieval NDCG by 2.3%.
• Semantic IDs, a learned taxonomy of DoorDash's catalog, raised ranking MRR by 4-5% according to the talk, and the same IDs power query reformulation.
• DoorDash keeps consumer memory at three timescales stored as text, vectors and a graph, and its steerable LLM-generated personalized collections lifted order rate by nearly 1% in the pets category.

• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

DoorDash's search lead details distilling offline LLM reasoning into small serving models, using LLM-generated relevance labels and semantic IDs to fix engagement-biased ranking, like 'gluten-free pasta' surfacing regular pasta.

• videoTeaching LLMs to Speak Spotify — Yves Raimond & Jacqueline Wood, Spotify
• videoWhy LLM Recommenders Will Be AI's Biggest Consumer App — Devansh Tandon, Meta
• videoWorld Models Need Causality, Not Pretty Pixels — Christopher Manning, Moonlake AI
• videoRobotics Has Been Stuck for 70 Years — Deepak Pathak, Skild AI

• videoWhy LLM Recommenders Will Be AI's Biggest Consumer App — Devansh Tandon, MetaAI Engineer
• articleUsing LLMs to amplify human labeling and improve Dash search relevanceIlya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo Agriel,Ishan Mishra,Eric Wang,Dmitriy Meyerzon,Dmitriy Meyerzon
• podcastReiner Pope – The math behind how LLMs are trained and servedDwarkesh Patel

Checking sign-in…

Loading comments…