Mix Data Or Merge Models Optimizing For Diverse Multi Task Learning 2024 10 15

Training separate language specialists and then combining their learned parameters (model merging) can outperform pooling the same training examples into one mixture. In this study from Cohere, researchers fine-tuned copies of the Aya 23 8B model on six individual languages, then merged them into one model. The comparison model learned from a multilingual mixture of the same available data. Study methods.

The six-language merge beat that mixed-data baseline on both general performance and harmful-response measures; the abstract reports gains of about 4% and 7%. GPT-4 judged the responses, and the paper reports that gains depend strongly on which languages are merged: a three-language merge did better than the six-language one. The result supports model merging as an option to evaluate when one multilingual model must balance usefulness and safety, not as a general guarantee. Results and evaluation.