Predictive Human Preference: From Model Ranking to Model Routing

A model's overall ranking can miss which response a person prefers for a particular question. In a 2024 experiment, Chip Huyen trained a predictor using the prompt and the identities of two competing models. On 2,367 held-out comparisons without ties, it predicted the preferred response with 76.2% accuracy, compared with 75% when using only the model identities and 74.1% for choosing the higher-ranked model. Read the experiment.

The potential use is routing each request to a suitable model before generating its answer. The result is preliminary: Huyen did not reserve a separate validation set for tuning, and the comparisons were noisy. The experiment did not measure production cost savings.

“I found that for all non-tie matches in my test set, the model with the higher Bradley-Terry score is preferred 74.1% of the time.”

“I found that for all non-tie matches, my preference predictor can predict the match outcome accurately 75% of the time if not using prompts, and 76.2% of the time if using prompts.”

“Friends who have deployed this in production told me that most users don’t read both options and just randomly vote for one.”

“My primitive experiment suggests that predictive human preference is feasible using a surprisingly small amount of data.”