Prompt, not guardrail
The failure
Section titled “The failure”A constraint that has to be certain gets put in a prompt rather than in something that actually intercepts the action. A prompt shifts the odds; it does not guarantee. Once the instruction is trusted like a guarantee, the run proceeds as if the constraint were enforced, and the one time the model doesn’t follow it, nothing is there to catch the miss. The gap between “asked for” and “enforced” is invisible until it matters.
TODO: map to a shipped example.