Vibeleaderboard
← All Intel
Intel / article

QD-Rec: Evolving Diverse Reasons for Decisions with Quality-Diversity

Source
pub.sakana.ai
Author
Sakana AI
Date

Context

Open-ended search, where a model keeps adapting and recombining earlier attempts to find new ones, has mostly been used to generate things: algorithms, research ideas, agents. Sakana AI asks whether it can help with choosing among things that already exist, such as which company to acquire or which customer to target. Its answer in the QD-Rec write-up is that the list of options may be fixed, but the reasons for picking one are not. A company might be worth buying for its technology or for a new market. So QD-Rec searches over pairs of an item and a written hypothesis explaining why it fits a particular decision maker, and tries to collect many good, different pairs rather than one best answer, which is the quality-diversity idea in the name.

Each step has three parts. A contextual bandit, a model that learns from past scores which items are worth examining next, picks a candidate and gets a bonus for items unlike those already collected. A proposer is shown three earlier recommendations from the archive and writes a new hypothesis by adapting one, combining several, or trying something the examples do not cover. An LLM judge then decides whether the result offers a genuinely new reason, in which case it is kept even with a low score, or whether it should compete blind against its most similar entry and replace it only if it wins. Keeping low scorers is deliberate. In one example, a proposal for JPMorgan to buy Southwest Airlines scored 1/10, but its idea of folding banking and rewards into travel purchases became the parent of a 5/10 case for buying Liberty TripAdvisor, which a from-scratch baseline had dismissed at 1/10.

The tests cover acquisition targets among about 6,300 US companies, customers for two products among 10,000 synthetic personas, and paper recommendations from about 17,500 accepted ML papers, each run with 1,000 evaluations and Claude Sonnet 5 or Haiku 4.5 as proposer. Against versions without the bandit or without evolving hypotheses, QD-Rec had the highest mean quality in most settings and usually the largest archive, though not everywhere: with Haiku on both paper-recommendation profiles, the variant without the bandit scored slightly higher. The bigger caveat is that in these tests quality means a score from a single LLM judge, gpt-5.6-terra, chosen from a different provider to limit self-preference, and the same scores guide the search. The write-up says the full paper also studies book recommendation using recorded human ratings; that study is not covered here. Sakana's announcement calls the experiments a starting point and says real use will need fresher data and expert human feedback to check whether the results hold beyond an LLM judge. The code is released, with the acquisition data left out because it comes from a paid LSEG subscription.

Key takeaways · AI-distilled
  • Sakana's QD-Rec treats picking from a fixed pool as open-ended search over reasons: each unit is an item plus a natural-language hypothesis for why it fits a decision maker's profile, and the goal is a diverse, high-quality set of such pairs.
  • QD-Rec chooses the next item with a contextual bandit over item and past scores, plus a bonus for candidates unlike those already in its archive, so a limited budget goes where coverage is thin.
  • A new recommendation joins the archive if an LLM judge finds its reason novel against its three nearest neighbors; otherwise it must beat its closest neighbor in a blind pairwise comparison, so distinct reasons coexist like ecological niches.
  • Weak ideas still served as stepping stones: a 1/10 proposal for JPMorgan to buy Southwest Airlines seeded a 5/10 case for buying Liberty TripAdvisor, versus 1/10 for a from-scratch proposal about the same company.
  • Across M&A, marketing and paper recommendation, with Claude Sonnet 5 and Haiku 4.5 proposers and a GPT-5.6 Terra judge, QD-Rec had the highest judge-rated quality in most settings and usually the largest archive for the same 1,000 evaluations.
Terms in this piece · Glossary
  • LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
  • embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Read the source pub.sakana.ai
More from Sakana AI
Recommended reads
Comments

Checking sign-in…

Loading comments…