UTC

Jailbreaks and safety bypasses

Inputs that get a model to produce what its policy forbids. A jailbreak defeats the model's own refusals, which is why refusals alone are not a security boundary.

How it works
  1. 1Models are trained to refuse some requests, but refusal is a learned behaviour, not an enforced rule.
  2. 2Framing, role-play, encoding and long multi-turn conversations can shift the model away from refusing.
  3. 3In an agent, a jailbroken model is also a model whose tool use is no longer constrained by its training.
What to watch for
  • Repeated reframing of a refused request
  • Outputs that violate the deployment's policy
  • Encoded or obfuscated inputs
How to defend
  • Moderate inputs and outputs with a separate layer, not only the model.
  • Enforce what the agent may do in code, outside the model.
  • Test each model release; resistance changes between versions.