The coming AI security crisis (and what to do about it) | Sander Schulhoff
Source
Lenny Rachitsky
Author
Lenny Rachitsky
Date
Terms in this piece · Glossary
prompt injection — An attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.
guardrails — The checks around a model that block bad inputs and outputs — filters, validators, and permission rules the model itself can't override.
Why it matters
If you're building or deploying AI agents — especially browser agents that read untrusted web content — this breaks down why off-the-shelf guardrailsThe checks around a model that block bad inputs and outputs — filters, validators, and permission rules the model itself can't override.Full definition → don't stop prompt injectionAn attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.Full definition → and what defense approaches actually hold up, from someone who ran the first large-scale prompt injection competition.
Key quotes
“You can patch a bug, but you can't patch a brain.”
Sander Schulhoff
“If someone is determined enough to trick GPT-5,
they're gonna deal with that guardrail, no problem.”
Sander Schulhoff
“And we found that, first of all, humans break everything.
100% of the defenses in maybe like 10 to 30 attempts.”
Sander Schulhoff
“any data that AI has access to,
the user can make it leak it.
Any actions that it can possibly take,
the user can make it take 'em.”
Sander Schulhoff
“the only reason
there hasn't been a massive attack yet
is how early the adoption is, not because it's secure.”
Chapters
00:00Introduction to Sander Schulhoff and AI security
05:14Understanding AI vulnerabilities
11:42Real-world examples of AI security breaches
17:55The impact of intelligent agents
19:44The rise of AI security solutions
21:09Red teaming and guardrails
23:44Adversarial robustness
27:52Why guardrails fail
38:22The lack of resources addressing this problem
44:44Practical advice for addressing AI security
55:49Why you shouldn't spend your time on guardrails
59:06Prompt injection and agentic systems
01:09:15Education and awareness in AI security
01:11:47Challenges and future directions in AI security
podcastThe 100-person AI lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI)
podcastWhy humans are AI’s biggest bottleneck (and what’s coming in 2026) | Alexander Embiricos (OpenAI Codex Product Lead)
podcastWhy most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon
podcastThe non-technical PM’s guide to building with Cursor | Zevi Arnovitz (Meta)
podcastAn AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines | Simon WillisonLenny Rachitsky
podcast“Engineers are becoming sorcerers” | The future of software development with OpenAI’s Sherwin WuLenny Rachitsky