← All IntelClip / EducationFine-tuning improves alignment even for unseen groups
From Synthetic Personas for Market Research: Where LLM Agents Break · ≈11:37
“Alignment improved even for the unseen groups.”
“That seems almost magical.”
“So, a lesson you can kind of take away is that your persona that you're looking for is in there.”
“We just need to figure out the way to summon it or elicit it.”
What’s in it
- Breaks down the Subpop paper's method for aligning LLM outputs with real survey data
- Reveals a surprising transfer effect: gains generalize to unseen demographic groups
- Argues that LLMs already contain latent personas that fine-tuning helps surface
Clip transcript
answer is you're going to have to test it and validate it against ground truth. The other natural thing you might expect is well, hey, we can fine-tune it, especially if it's missing data that isn't there, especially for example, if it's behaviors or something that wouldn't be in the training text. And this is a great paper to be inspired by for this. This is the Subpop paper. Basically, they construct a prompt template, which is the demographic information, then the survey question they want to ask, and then they compare the known human data distribution to the distribution that comes out of the model, and they do fine-tuning until the model and the human data align. Now, here's the interesting thing. When they did this, as you'd expect, the results that were from the populations they gave to the model, that's the ones in blue, improved. But very interestingly, the ones in white also improved by almost the same degree. Alignment improved even for the unseen groups. That seems almost magical. And some subsequent research has hinted that what might be really happening here is that the model itself has a latent understanding of these groups. It just didn't know how to express it in the format of surveys. And if you think about it, LLMs aren't used to doing surveys as a task. And so they aren't going to be as good as fitting it, especially to a prompt format they may not have seen on the first go-around. But fine-tuning actually is helping it learn the task or how to express itself. So, a lesson you can kind of take away is that your persona that you're looking for is in there. We just need to figure out the way to summon it or elicit it.
Comments
Checking sign-in…
Loading comments…