Vibeleaderboard
← All Intel
Intel / article

There is No Theoretical Curse of Multilinguality For Embedding Space Structure

Source
arxiv.org
Author
Niyati Bafna, Neha Verma, Vil\'em Zouhar, Philipp Koehn, David Yarowsky
Date

Context

Adding languages need not require a proportionate expansion of the numerical space used to represent meanings. This paper constructs multilingual representations that preserve similarity relationships within each language and keep translations closer together than different concepts, while retaining distinguishable representations.

Under its assumptions, the extra dimensions needed above a fixed concept space grow logarithmically with the number of languages. The result challenges the idea that representation space must inevitably become prohibitively crowded. It leaves open how reliably training can learn such a structure: the theorem concerns geometry, not total model size, training cost or guaranteed translation quality.

Terms in this piece · Glossary
  • embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
Recommended reads
Comments

Checking sign-in…

Loading comments…