Jailbreaks and safety bypasses
Inputs that get a model to produce what its policy forbids. A jailbreak defeats the model's own refusals, which is why refusals alone are not a security boundary.
- LLM01:2026 Prompt Injection
- AML.T0054 LLM Jailbreak
- EU AI Act 15(5) Adversarial examples or model evasion
How it works
- 1Models are trained to refuse some requests, but refusal is a learned behaviour, not an enforced rule.
- 2Framing, role-play, encoding and long multi-turn conversations can shift the model away from refusing.
- 3In an agent, a jailbroken model is also a model whose tool use is no longer constrained by its training.
What to watch for
- Repeated reframing of a refused request
- Outputs that violate the deployment's policy
- Encoded or obfuscated inputs
How to defend
- Moderate inputs and outputs with a separate layer, not only the model.
- Enforce what the agent may do in code, outside the model.
- Test each model release; resistance changes between versions.