Agents that pass standard injection tests can still be hijacked through answers to their own clarifying questions. Treat ask-user responses as untrusted input and test that path.
Key takeaways · AI-distilled
Scale Labs' ASPI runs 728 AgentDojo-based scenarios twice, once with a complete instruction and once with a detail removed so the AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → must ask. Injections arriving through the clarification answer succeeded 10x more often for several models.
Per Scale, Claude Opus 4.7 was the only model robust in both settings, GPT-5.5 and GPT-5.4 showed smaller gaps, and open models started weaker: DeepSeek V3.2 had 65% attack success before clarification was involved.
A prompt guard and a tool filter each cut attack success (Gemini 3 Flash fell from 35.7% to 27.0% and 23.9%) without closing the gap, because the answer the agent needs and the injected instruction often arrive in the same message.
The worst failure mode is an agent that completes the user's task correctly while also executing the attacker's instruction, so evaluations that only check task completion can pass a compromised agent.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
prompt injection — An attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
