Synthetic Personas for Market Research: Where LLM Agents Break

About

Ishan Anand analyzes a study where LLM agents replayed a ~1,000-person market research survey and matched human averages but with far less variance, revealing that synthetic respondents smooth over the real messiness of human populations. He shows how small prompt changes shift purchase probability due to invented confounders, argues that aggregate accuracy can mask distorted subgroups and flattened distributions, and proposes calibrating synthetic-vs-human agreement against a human-vs-human noise floor before using personas to drive decisions.