Anthropic details Constitutional Classifiers, a protective LLM layer designed to stop AI model jailbreaking by monitoring inputs and outputs for harmful content
inputs designed to bypass its safety training and force it to produce outputs that might be harmful. Our new technique is a step towards robust jailbreak defenses. Read the blog post: https://anthropic.com/... [image] Simon Willison / @simonw : I find that details of this evaluation disappointing: I would value a system that protects against ALL jailbreaks a whole lot more than one that can protect against some but not all Pietro Schirano / @skirano : Excited to announce a new research preview at @AnthropicAI today. A demo of our new Constitutional Classifiers. Can you break the system and find a universal jailbreak that lets the model answer all 8 questions we've defined? [video] @anthropicai : We challenged jailbreakers to try to break a prototype version of the system to test its robustness. After thousands of hours of red teaming, not one participant found a reliable jailbreak that extracted detailed information across a set of 10 harmful questions. @anthropicai : New Anthropic research: Constitutional Classifiers to defend against universal jailbreaks. We're releasing a paper along with a demo where we challenge you to jailbreak the system. [image] Forums: r/Anthropic : Anthropic dares you to jailbreak its new AI model | Week-long public test follows 3,000+ hours of unsuccessful bug bounty claim attempts. Msmash / Slashdot : Anthropic Makes ‘Jailbreak’ Advance To Stop AI Models Producing Harmful Results
Context & Ripple Effects
Anthropic’s safety work has repeatedly exposed how prompt-based defenses can fail: many-shot prompts evaded guardrails, and later Best-of-N Jailbreaking showed a black-box route to bypass frontier-model safeguards. That makes a separate screening layer a meaningful change in where the company places its defenses.
The approach extends Anthropic’s earlier constitutional AI framework from shaping model behavior to checking traffic around the model. Its public challenge and red-team result provide evidence against the tested attack set, while criticism of the evaluation underscores that this is not proof against every jailbreak.
First-order effects
- Anthropic can apply an additional input-and-output filter to the prototype it tested, aiming to block harmful requests and responses even when a prompt bypasses the base model’s training safeguards.
- Developers and users of systems using the layer may face more safety screening at the request boundary; jailbreak researchers gain a public target for testing its stated robustness claim.
Second-order effects
- Model providers facing demonstrated prompt attacks have added incentive to pair training-time alignment with runtime classifiers, rather than treating either as sufficient alone.
- Evaluation quality becomes more consequential: Anthropic’s challenge result is useful for comparison, but outside scrutiny will focus on the coverage of harmful questions and on false positives as well as bypasses.
Third-order effects
- If layered defenses prove durable, frontier-model safety may increasingly be implemented as a stack of model training, traffic monitoring, and adversarial evaluation rather than a property claimed by the base model alone.
- That shift could make model access and the surrounding enforcement layer more central security boundaries, while leaving an unresolved trade-off between broad blocking and useful model access.
The trend: Frontier AI labs are moving from alignment-only safety claims toward defense-in-depth systems tested against adaptive jailbreak techniques.