There is a simple appeal to guardrails. Something can cause harm, so we put a boundary around the behavior we don't want. We do this everywhere. A permission system prevents someone from reading data they shouldn't see. A rate limit prevents a service from accepting more traffic than it can safely handle. A railing keeps someone from walking off the edge of a platform. A language model refuses a request its designers have classified as unsafe.

Sometimes that's exactly the right solution. If we already understand the failure well enough to prevent it cheaply, there is little virtue in allowing it to happen again. The interesting problem begins when the boundary we can enforce isn't quite the same as the distinction we care about.

Suppose I want an AI system to avoid helping someone cause harm. That's easy to say, but now I have to turn that principle into something the system can actually distinguish. Which information is dangerous? Under what circumstances? Does the identity of the person asking matter? Does their stated purpose matter, and can we trust it? What if the same information is necessary for education, journalism, medicine, security research or ordinary maintenance? The principle may be relatively stable while its application becomes complicated very quickly.

So we build a guardrail using whatever information is available at the time of the decision. It may examine the request, the surrounding conversation, the tools available to the system, perhaps something about the user or application, and then decide whether an action should be allowed. That decision creates a boundary that can be useful without being identical to the principle that motivated it.

I think that's the guardrail problem.

A guardrail is a representation of a distinction, and representations leave things out. This doesn't make them bad. We couldn't operate complicated systems without permissions, schemas, interfaces, policies and tests that reduce messy reality into distinctions we can actually act upon. The problem begins when we forget that the representation is smaller than the thing it represents.

Consider a physical railing. Its job is unusually easy to specify: people on this side should remain separated from the drop on the other side. Even there, context exists. Someone may need access beyond the railing to perform maintenance, emergency personnel may need to cross it, or the railing itself might block an evacuation route if placed badly. Still, the physical problem is constrained enough that a fixed barrier works remarkably well.

Semantic boundaries are harder because language allows the same subject to appear through explanation, fiction, quotation, criticism, historical discussion, technical analysis and direct instruction. Two requests can contain nearly identical words while asking for substantially different things. A simple boundary will sometimes treat them alike. A more complicated boundary can preserve more distinctions, but complexity doesn't eliminate the underlying problem. Eventually the system encounters another case we didn't represent.

This isn't unique to AI. Spam filters block legitimate email, fraud systems decline legitimate purchases, content moderation removes satire, security policies prevent employees from doing harmless work and building codes occasionally fit a strange building poorly. Each boundary exists because some problem was real enough to justify intervention. The fact that the boundary occasionally fails doesn't mean the underlying constraint was unnecessary.

The opposite mistake is easier to make: the rule exists, therefore whatever the rule blocks must be the thing we intended to prevent. Now the representation has become the principle, and a system can become very good at satisfying a measurable boundary while that boundary becomes increasingly detached from the outcome we actually care about.

This happens in software constantly. We measure response time because users dislike waiting, and eventually teams optimize response time. Usually that's useful. Then someone discovers that an interface can become technically faster while becoming harder to use, that a cached result arrives instantly while being wrong, or that optimizing one metric moved the delay somewhere we weren't measuring. The metric wasn't false. It was incomplete.

The same thing can happen with safety. A declining rate of prohibited outputs may indicate that a safeguard is working extremely well, or it may mean users have learned not to ask certain legitimate questions. An increase in refusals may represent stronger protection against a real failure, or a system losing the ability to distinguish harmful requests from harmless ones. The number alone can't tell us which interpretation is right.

This is where I once made a mistake in thinking about guardrails. I treated constraint and learning as opposites. If a system was prevented from crossing a boundary, I assumed it was also prevented from learning why the boundary mattered. From there it was easy to imagine a better architecture in which the system would encounter consequences, integrate them and gradually develop something closer to genuine understanding.

There is something useful in that picture, but it leaves out the fact that we don't require every learner to personally reproduce every failure. A child can learn that a stove is hot without receiving a serious burn. An engineer can learn from an accident report. A new generation of software can inherit a regression test created because the previous generation failed.

Constraints can contain learning too. A guardrail may be the surviving trace of a mistake we have no reason to repeat.

That changes the question. Instead of asking whether we should have guardrails or learning, I want to know what each guardrail represents, how confident we are in the distinction it makes and what happens when reality presents a case that doesn't fit. Some boundaries deserve to be extremely difficult to cross while others should be provisional. Some can be enforced mechanically while others require additional context or human judgment. Experience may show that a boundary can become more precise, or it may show that the boundary needs to become stronger because we were allowing risks we didn't initially understand. There is no general rule requiring movement toward greater permissiveness. Learning can move the boundary in either direction.

Deviation isn't automatically information we should preserve any more than compliance is automatically evidence of safety. Sometimes a prohibited action is simply the prohibited action we expected, and sometimes a refusal is exactly correct. The interesting cases are the ones where our expectation and the observed situation stop matching: a user believes a harmless request was blocked, an evaluator discovers a dangerous request that passed, two safeguards classify essentially the same situation differently, or a new application gives an old capability consequences it didn't previously have.

Something has happened at the boundary between our representation and the world, but that doesn't tell us what it means. Investigation may show that the model needs to change, the safeguard needs to change, the user misunderstood the system, the original policy was underspecified, or nothing particularly profound happened at all.

Feedback doesn't arrive with its interpretation attached.

That's why I no longer think the alternative to guardrails is "emergent alignment." The phrase makes it sound as though removing premature constraints will naturally allow something wiser to develop. Emergence doesn't promise wisdom. A system allowed to explore can discover useful distinctions, but it can also discover shortcuts, exploit weaknesses in its environment, reinforce mistakes or produce behavior nobody intended. Constraint can prevent learning or preserve it. Freedom can expose useful information or expose someone else to the cost of our experiment. The architecture has to distinguish among those cases.

This is where engineering becomes more useful to me than metaphor. If we're uncertain about a behavior, we don't have only two choices: prohibit it forever or release it into production and see what happens. We can test it, simulate it, red-team it, limit its authority, reduce the blast radius, deploy it gradually, require review for irreversible actions, record what happens and compare behavior across contexts. None of those mechanisms guarantees alignment. They give us more places to discover that our current understanding is wrong before the mistake becomes expensive.

A guardrail can also prevent an action without explaining why the action is dangerous, and sometimes that's enough. I don't need my database permission system to understand the moral significance of privacy. I need it to prevent an unauthorized account from reading a table. The understanding can exist elsewhere in the larger system: in the people who designed the policy, the law that constrains the organization, the tests that verify enforcement, the incident history that explains why the rule exists and the process through which exceptions are evaluated.

We shouldn't require every component to contain the reasoning of the whole. What matters is whether the larger arrangement retains enough of that reasoning to recognize when the implementation no longer serves its purpose.

That is harder than writing a rule because rules are inspectable while the relationship between a rule and the reason for the rule is easier to lose. Years later, someone encounters an inconvenient restriction and removes it without knowing what failure originally produced it. The opposite can happen too: everyone knows the rule exists, nobody remembers why, and the organization continues enforcing it long after the circumstances that justified it disappeared.

Both are failures of memory.

So when I encounter a guardrail now, I want to know what problem it was intended to prevent and what distinction it's actually making. I want to know what evidence tells us that distinction works, what happens when it's wrong, who bears the cost of a false positive or false negative, what we would have to observe before changing it and, if we do change it, how the next person will know why.

Those questions don't eliminate the guardrail. They keep it connected to the problem that gave it a reason to exist.

That's the part I was reaching for when I first wrote about premature sealing. The problem isn't that something became fixed, because some things should become fixed. The problem is losing the path by which a fixed answer can be compared with the changing situation it was meant to address.

A good boundary can remain stable for years because reality keeps confirming it. Another may need to move tomorrow. We don't know which merely by noticing that it's a boundary. We find out by keeping the reason, the implementation and the consequences close enough that they can still disagree with one another.

The guardrail problem isn't that guardrails prevent alignment.

It's that eventually we can forget what the guardrail was trying to align with.