Memorization Diagnostics for Code LLMs Should be Scale-Aware
Source
Prateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djir\'e, Xunzhu Tang, Yewei Song, Iyiola Emmanuel Olatunji, El Hacen Diallo, Jacques Klein, Tegawend\'e F. Bissyand\'e
Author
Prateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djir\'e, Xunzhu Tang, Yewei Song, Iyiola Emmanuel Olatunji, El Hacen Diallo, Jacques Klein, Tegawend\'e F. Bissyand\'e
Date
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Why it matters
If you use contamination probes to judge whether a code model memorized a benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition →, those probes likely fail on exactly the large models you ship — the methodology, not the model, is what breaks.
articleMeasure, Don't Optimize: Forecasting Recovery in LLM UnlearningZirui Song, Huaxing Liu, Xiang Wang, Shuai Li, Xinye Li, Lang Gao, Jinghui Zhang, Zheng Lu, Fengxian Ji, Xiaojun Chang, Xiuying Chen
articleBeyond FLOPs: Energy-Aware Knowledge Distillation for Sustainable LLMs on Code-Related TaskEnrique Barba Roque, Lu\'is Cruz, Annibale Panichella
articleBeyond the Traceback: Using LLMs for Adaptive Explanations of Programming ErrorsAlexandru-Radu Moraru, Shreyan Biswas, Ujwal Gadiraju