Jailbreak

Techniques include role-play framing, obfuscated encodings, multi-turn escalation, and long chains of examples. Jailbreaks target the model’s own restrictions, while prompt injection targets the application built around it, though the methods overlap.

The distinction matters for who gets hurt. A jailbreak is usually the user working against the model they are talking to, so the damage is mostly to the provider: policy violations, reputational harm, misuse of a service. Prompt injection is a third party working against a user, hiding instructions in content the system will later process, so the damage lands on someone who did nothing wrong. If you run an AI feature, jailbreaks are a content and brand problem, and injection is a security problem.

The uncomfortable structural point is that refusals are trained behavior rather than a boundary. Safety training shifts the probability that a model declines a request; it does not create an enforcement mechanism, and no amount of it produces a guarantee. Treating a refusal as a control is the same category error as treating client-side validation as authorization. It raises the cost of misuse without ever making misuse impossible.

That is why the durable controls sit outside the model. What the system is permitted to do, what data it can reach, what its output is allowed to touch, and what gets logged are all enforceable in ways that a model’s willingness to say no is not. Testing belongs outside the model too: AI red teaming exists precisely because the failure modes are empirical rather than specified, and the only way to know how your deployment behaves is to attack it.