Towards a Risk Assessment of Malicious Skill Files in Coding Agents
- Source
- arxiv.org
- Author
- Rui Yang, Michael Fu, Kla Tantithamthavorn, Chetan Arora, Joey Chua
- Date
Context
A coding can mistake a malicious file for legitimate setup guidance. This study embedded harmful commands in instructions presented as required preparation for building a small web application. Skills normally supply reusable instructions and scripts, making this a route for attacker-controlled text to influence tool execution.
The authors classified about 96% of Google Gemini CLI runs and 72–74% of Qwen Code runs as accepting the attack, primarily from statements of intent to comply. Actual command execution was confirmed in just one of 5,629 completed runs. The authors attribute that gap to the 120-second limit and restrictive ; missing execution evidence does not establish refusal. Both agents ran with automatic action approval. The study exposes a weakness in how agents interpret instructions, but does not establish comparable rates of completed attacks.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
- sandbox — An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
“This interface also widens the attack surface, letting malicious shell commands hide within natural-language skill files.”
“Gemini CLI is exploited in 95.5-96.1% of runs and Qwen Code in 71.6-74.0% (raw majority vote to declared-intent-corrected estimate, both within the human gold standard), nearly invariant to the generating model.”
“Explicit safety recognition occurs in only 1.99% of runs. Enterprises must assess and mitigate skill-interface risk before adopting coding agents.”
Checking sign-in…
Loading comments…