← All IntelClip / EntertainmentBuilding trust in auto mode via red teaming
From Reflecting on a Year of Claude Code (Boris Cherny) · ≈9:08
“And so what we did was we collected thousands of transcripts of like an entire agent trajectory and a permission prompt and had auto mode classify whether or not it was safe.”
“So it's not only just protecting you against the vulnerabilities that are out there in the wild today, but, the most intelligent attacks that we can construct.”
What’s in it
- How Claude Code's auto mode classifies safe vs. unsafe agent actions
- Red-teaming and prompt-injection testing behind the permission system
- Why it aims to block sophisticated attacks, not just known ones
Clip transcript
And so what we did was we collected thousands of transcripts of like an entire agent trajectory and a permission prompt and had auto mode classify whether or not it was safe. And it was extremely good at this. So then we got red teamers, and we asked them to try to prompt inject, and try to hack the code base. And we use this to create evals and make sure that all of these were denied. And then we had our own internal teams try to prompt inject and hack Claude Code’s auto mode. And then we improved auto mode to make sure that we caught all of these. So it's not only just protecting you against the vulnerabilities that are out there in the wild today, but, the most intelligent attacks that we can construct.
Comments
Sign in to comment.
Loading comments…