Children, but not language models, show accelerating returns in word learning
Source
arxiv.org
Author
Michael C. Frank
Date
Why it matters
It puts a sharp empirical edge on the data-efficiency gap between human learning and scaling, which is the argument underneath most of the pretrainingThe first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.Full definition → debate.
Terms in this piece · Glossary
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
articleRecursive Language Models Generalize Out of DomainChenxiao Yang, Zhiyuan Li, David McAllester, Nathan Srebro
articleModular Cognitive Architecture Emerges in Large Language ModelsPengrui Han, Jacob Andreas, Evelina Fedorenko, Andrea Gregor de Varda
articleAccelerating LLM Inference via Vector Index Based Output EmbeddingsMartin Loretz, Sepp Hochreiter