Cohere's 2024 Aya model addresses instruction-following beyond the small set of languages with abundant training resources. Its publisher abstract reports support for 101 languages, more than half considered lower-resourced, and public release of the model and instruction data. The associated Aya dataset work explains part of the training foundation: fluent speakers supplied instruction-and-answer examples, supplemented by a broader collection assembled through translation and templates. Those data resources give researchers material for developing assistants across more languages; the coverage count alone does not establish equal quality in each.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
postCohere releases open-weights translation model North Small Translate
articleThe Data Provenance Initiative A Large Scale Audit Of Dataset Licensing And Attribution In Ai 2023 10 25
