Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narratives
Source
arxiv.org
Author
Karolina Dro\.zd\.z, Micha Heilbron
Date
Why it matters
Small models may already carry discourse tracking ability that teams assume requires scale.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
articleRecursive Language Models Generalize Out of DomainChenxiao Yang, Zhiyuan Li, David McAllester, Nathan Srebro
articleModular Cognitive Architecture Emerges in Large Language ModelsPengrui Han, Jacob Andreas, Evelina Fedorenko, Andrea Gregor de Varda
articleDo Agents Know When They Succeed? Calibrating Agent Confidence from Internal RepresentationsPriyanka Mary Mammen, Emil Joswin, Srujananjali Medicherla