Vibeleaderboard
← All Intel
Intel / video

Defending against AI jailbreaks

Source
youtube.com
Author
Anthropic
Date

Context

Researchers at Anthropic explain in a roundtable why they target universal jailbreaks: a single prompting strategy that works across many harmful questions and gets detailed answers. Their concern is that a non-expert could reuse such a template instead of cracking each question separately. A narrow trick, such as a role-play that works for only one topic, is far less useful.

The defense is layered. An input classifier reads the whole conversation, Claude can itself refuse, and an output classifier watches the response as it streams and can stop it. The classifiers are trained from a plain-language constitution listing harmful and harmless categories, with Claude generating synthetic examples, so changing the rules means editing the text and retraining.

They report that a prototype blocked over 40% of benign claude.ai requests while the demo system blocked 0.38%, and that in a public red-team demo participants spent an estimated 3,750 hours before anyone passed all eight questions, and roughly 600 more before a universal one. They say grading was imperfect and false positives remained.

Terms in this piece · Glossary
  • jailbreak — A prompt crafted to make a model ignore its own guidelines — usually through roleplay, hypotheticals, or encoding rather than a direct request.
Read the source www.youtube.com
More from Anthropic
Recommended reads
Comments

Checking sign-in…

Loading comments…