Improving Recommendation Systems & Search in the Age of LLMs

A useful role for language models in recommendations is preparing information that a smaller live system can use. Eugene Yan's review of search and recommendation research describes this pattern through generated descriptions and training labels, rather than assuming a large model must answer every recommendation request.

One example in his account uses a language model to label poor job matches, then trains a lightweight filter on those judgments. This moves the expensive assessment into data preparation while leaving a smaller model to screen live recommendations. The design question becomes which part of the service benefits from language understanding, and which part must remain fast enough to run for every user.