The Miranda Hypothesis: How Hamilton Poisoned Persona Evals - Jacob E. Thomas, Results Gen

A historical chatbot's convincing voice does not establish that its answers fit the period it represents. The Miranda Hypothesis talk proposes an experiment to test that distinction, using Abraham Lincoln at different points in his life. The speaker explicitly presents a preregistered design, without results from a large-scale run.

The experiment would ask the same diagnostic questions under three conditions: period-specific primary writings, a modern biography, or a date without supporting documents. A historian-designed rubric would assess later ideas appearing too early, consistency with the supplied documents and plausibility within the period. This makes historical fidelity measurable separately from how fluent or recognizable the persona sounds.

“If a dominant failure mode is anachronistic compositing, and your evals measure fluency and personality consistency, then your evals cannot detect the dominant failure.”

Jacob E. Thomas

“the composite Hamilton knows he will be the subject of a Broadway musical. The composite Lincoln has already read the Gettysburg Address, even if he was summoned before he wrote it.”

Jacob E. Thomas

“Compositing is not a bug that you patch in post training. Post training reinforces it.”

Jacob E. Thomas

“You don't train a persona. You compose an encounter. And you keep the receipts.”

Jacob E. Thomas

“You do not want a model that is your mother. You want a model that can speak with your mother's documents in the room.”

Jacob E. Thomas