Llm See Llm Do Guiding Data Generation To Target Non Differentiable Objectives 2024 07 05

Training a language model on another model's generated text can pass along characteristics beyond the requested subject matter. Cohere's research abstract reports sensitivity to properties of synthetic training data even when the prompts used to create it appear neutral.

The authors also report deliberately shaping that data to steer the trained model toward properties such as more varied vocabulary or lower toxicity. Their finding makes the data-generation process part of controlling model behavior: the examples supplied for training can influence how the resulting model writes.