Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories
Source
arxiv.org
Author
Shuyan Huang, Kai Du, Andrew Lan
Date
Context
Reusable guidance pooled across developers helped coding agents more consistently than personalized guidance in this study. Researchers extracted written skills from interaction histories and evaluated them using 206 sessions from 13 developers, with a LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → simulating the developer during replay. Research abstract.
Personalized skills produced small, uneven gains over using no skills, while generic skills delivered the strongest and most consistent improvements. Personalization worked better when histories repeatedly expressed preferences relevant to later tasks. The result favors transferable procedures in this replay setting, with more selective value from individual preferences.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
videoSkills are the New SDKs - Elvin Aghammadzada, DataRobotAI Engineer
articleAn Empirical Study of Agent Skills' Downstream UtilityYu Cheng, Dehai Zhao, Zhongxin Liu, Qing Huang, Zhenchang Xing, Xiaoxue Ren
articleRethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple RoundsYuxuan Liu, Zhaochen Su, Yuhao Zhang, Jiahe Guo, Zhongwei Xie, Huihao Jing, Lingyun Xie, Qing Zong, Yauwai Yim, Zhixiong Zhang, Haoran Li, Yangqiu Song