Guardrails aren't prompts

There's a comfortable lie in a lot of early AI work: that you can make a system safe by asking it nicely. Put "never do X" in the system prompt and call it a guardrail. It feels like a control. It is not one.

A prompt is a suggestion the model is very good at following until the one time it isn't. Under an unusual input, a long context, a cleverly worded request, or just the tail of the distribution. If the only thing standing between your agent and a harmful action is a sentence in the prompt, you don't have a guardrail. You have a wish.

The rule I build to now

A real guardrail lives outside the model, in code the model can't talk its way past. The model proposes; a deterministic layer disposes. It sounds obvious written down, but it inverts how most people start: they try to make the model itself trustworthy, when the durable move is to make the model's environment unable to permit the thing you're afraid of.

Concretely, that means the model never takes the consequential action directly. It emits an intent. Structured, typed, inspectable. And a separate layer validates that intent against rules that don't depend on the model's mood: is this action in the allowed set, are the parameters within bounds, does this need a human signature, has this been rate-limited. The model's job ends at "here's what I want to do." Whether it happens is not the model's call.

Defense in depth, borrowed from security

None of this was new thinking, I just imported it from security engineering, where we long ago gave up on any single control. You assume each layer will occasionally fail and you stack them so a failure has to line up across all of them:

The prompt sets intent and reduces how often the model even wants to do the wrong thing. Structured output constrains the shape of what it can ask for. A validation layer checks every request against hard rules. Least-privilege credentials mean even an approved-but-wrong action can only reach so far. And an audit trail means when something does slip, you see it fast and learn from it. No layer is trusted alone. The safety is in the stack.

The prompt still matters. It's the cheapest, highest-leverage way to shape default behavior. But it's the first layer, not the only one, and never the one you'd bet a production system on. Treat it as steering, not as brakes. The brakes go in code.