Intel / article

Bootstrapping Conversational Recommendation Agents At Spotify: Synthetic Data Generation and Self-Improvement Loops

Source: arxiv.org
Author: Enrico Palumbo, Alexandre Tamborrino, Victor Ode, Ben Lacker, Adri\`a Casas Escoda, Jeremy Hopple, Marcus Better, James Leoni, Hugo Galv\~ao, Hugues Bouchard, Mounia Lalmas, Jos\'e Luis Redondo Garc\'ia, Abenezer Abebe, Ann Clifton, Anton Blomberg, Henrik Lindstr\"om, Dani Doro, Christine Doig Cardet
Date: Published Sep 28, 2026

Context

A conversational recommendation service needs to handle follow-up requests before it has real conversations to learn from. Researchers describing an at Spotify address that starting problem by expanding single prompts into synthetic multi-turn conversations for before launch.

Their improvement loop then identifies planning and tool-use mistakes and uses a coding agent to help repair them. Planning here means choosing which tools to call and in what order to satisfy a request. The authors report using this pipeline in production development. Its supported contribution is a way to exercise and refine conversational behavior before live interaction data becomes available; the paper's separate online comparison evaluates the resulting experience against an earlier product.

Key takeaways · AI-distilled

To tune a conversational recommender before launch, Spotify turned single-turn prompts into realistic multi-turn synthetic conversations, giving the team an evaluation set without real user traffic.
Its self-improvement loop pairs variance-based contrastive optimization with a coding agent that iteratively finds and fixes planning and tool-use errors, adding 8% quality on top of an already heavily tuned manual prompt.
In online A/B tests against a prior experience that only supported session refinement, the shipped agent lifted listening by 14% and weekly active users by 5% while cutting skip rate by 5%.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Read the source arxiv.org