Engineering AI Discover
- Source
- netflixtechblog.com
- Author
- Netflix Technology Blog
- Date
Context
Netflix is testing AI Discover, a tab in its TV app where a small group of members pick a theme, such as Quick Watches, and get a row of matching titles, a short note on why they fit and narrower themes to try next. Netflix's engineering post describes the system behind the tab, and its central decision is that a language model may propose what to do while ordinary code decides whether it happens. The reason is the setting. Every response ends on someone's television, and at Netflix's scale even rare failures (a slow call, an unavailable title, an off-brand line) happen many times a day.
Each request becomes a plan: a graph of approved steps, each a named capability such as retrieval, ranking or text generation that declares its inputs, its outputs and what happens if it fails. A plan either comes from a set Netflix has already tested or is drafted on the spot by a language model. Before anything runs, a controller checks that every step is approved, every input has a source, nothing loops and the plan ends in exactly one page. If the planner is slow or its plan fails a check, a tested plan runs instead. Most capabilities wrap systems Netflix already uses, such as search and large-scale ranking, so even a freshly drafted plan stays in Netflix's catalog. Language models get narrow jobs, such as writing row titles or suggesting next themes, and Netflix treats what they return as untrusted input.
Rules that must always hold, such as maturity standards and titles a member has asked not to see, are enforced in code, because a rule written into a prompt is followed only probabilistically. The planner writes placeholders where values like a profile's maturity rating belong, and the controller fills them in after validation, which Netflix says also protects against . Generation can take several seconds while a typical TV request returns in moments, so the app acknowledges each button press at once, shows a thinking state and fetches the page when the server says it is ready. In this version members cannot type requests or chat; they choose themes and tuning options such as don't recommend.
The post describes design decisions and reports no measured results for engagement, latency or cost; Netflix lists how cost and latency hold up as traffic grows among its open questions. The design answers a problem Netflix described in its earlier GenRec work, which noted that off-the-shelf language models tend to recommend titles outside the catalog and ignore business constraints.
- Netflix's AI Discover uses a plan-and-execute agent rather than a reason-act : an LLM may draft a plan, but code validates the graph (every step approved, every input produced, no loops, exactly one page) and falls back to a tested plan if the planner is slow or fails.
- Safety rules are enforced in code rather than prompts. The planner writes placeholders instead of values such as a profile's maturity rating, and the controller fills them from the request after validation, which Netflix says also protects against prompt injection.
- Generation requests are asynchronous: a remote press returns an acknowledgement immediately, the TV shows a thinking state, and the server notifies the TV when the plan finishes, so slow generation does not hold threads open across services.
- All LLMs sit behind one interface and each task's model is a per-task setting Netflix can override live. Candidates are scored on the same tasks with automated checks and LLM judges before a swap, and prompts live server-side so changes ship without a TV app release.
- MemberLLM, Netflix's in-house model, writes member taste summaries ahead of time to keep that work off the request path. This version has no free-text input or chat, only themes and tuning options, which Netflix says keeps the LLM's job narrow by design.
- grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
- prompt injection — An attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- agentic loop — The cycle an agent runs in: decide, call a tool, read the result, decide again — repeating until the goal is met or a stop condition fires.
Checking sign-in…
Loading comments…

