One Model to Loop Them All: Elastic Latent Recurrence for Language Models
Source
arxiv.org
Author
Faisal Ladhak, Shantanu Acharya, Santiago Akle Serano, Simeng Sun, Boris Ginsburg, Krishna C Puvvada
Date
Context
Elastic Latent Recurrence lets one LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → spend more computation by repeating part of its internal processing. The paper trains a shared block of layers to work at several loop counts, preserving useful performance when it runs only once. Additional passes reuse the same learned parameters.
In the four-billion-parameter experiment, the one-loop model averaged 65.1% across ten tasks, close to the matched non-recurrent baseline's 65.7%. Four loops raised that average to 72.4%. This offers a computation-versus-accuracy choice within one trained model: extra passes cost processing time without requiring a larger set of model weights.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
articleParcae: Doing more with fewer parameters using stable looped modelsTogether AI
articleMeasure, Don't Optimize: Forecasting Recovery in LLM UnlearningZirui Song, Huaxing Liu, Xiang Wang, Shuai Li, Xinye Li, Lang Gao, Jinghui Zhang, Zheng Lu, Fengxian Ji, Xiaojun Chang, Xiuying Chen
articleMemorization Diagnostics for Code LLMs Should be Scale-AwarePrateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djir\'e, Xunzhu Tang, Yewei Song, Iyiola Emmanuel Olatunji, El Hacen Diallo, Jacques Klein, Tegawend\'e F. Bissyand\'e