Output Filters Are Not a Security Boundary

Teams building AI features often add a filter to the model's output and consider the risk handled. A filter helps with something real. It can catch unwanted content, reduce embarrassing responses and send borderline cases for review. It is one layer, though, and a boundary is something different. A boundary is the point where a decision is enforced reliably, whatever the input. An output filter inspects text, usually with rules or a classifier. Much of the harm in an AI system does not sit in the text. It happens when a system takes an action or returns data it should not.

What a filter does well

A filter works on the text that comes out of the model. It can recognize categories of content the product does not want to show and flag responses that need a human look. It is inexpensive to run and easy to explain to a product owner. Its limits follow from the same features. A classifier estimates whether text belongs to a category, so it produces false positives, which frustrate users and push them toward workarounds, and false negatives, which let some unwanted content through. Its performance also depends on the wording, language and format of the input, so it changes when those change. Filters also produce a record of what was blocked, which is useful for tuning. Logged blocks show patterns in ordinary use and reveal where legitimate requests are caught by mistake.

Why a filter cannot be the boundary

Consider an assistant that can send email on a user's behalf. A filter can check the wording of an outgoing message. It cannot know whether the recipient should receive that information, whether the user has authority to send it, or whether the message was requested by a document the user never read. The consequential decision sits in the tool call, and the filter is not there.

Bypass is also a property of the mechanism. Text can be rephrased, split across turns, moved into another language or hidden in material the model reads. A system that depends on recognizing bad wording must keep recognizing new wording, and it will always be a step behind. This does not make filters useless. It means their failures should be expected, so the design does not depend on them being perfect. The same reasoning applies to a model's own refusals. A refusal lowers the chance of a bad answer in the common case, but it is a behavior of the model, which can change between versions and cannot be counted on as a control.

Enforce controls where the effect happens

Put the boundary at the layer where the effect occurs. At the tool layer, each call should be authorized against the user's permissions, its parameters validated and its destinations restricted to an allowlist. Spending limits, row limits and read-only modes belong there too. Approval steps are one form of enforcement. For an action with large or irreversible effects, the approval should come through a channel the model does not control, and it should show the exact action and parameters being approved. At the data layer, the system of record should enforce row-level and field-level access, and retrieval should be filtered by the requesting user's identity. Take a concrete case. An assistant that can query customer records should be unable to return a field the user cannot see, whatever the prompt says, because the query itself returns only what the database permits. If the boundary depends on the model declining, it is not yet a boundary.

Layer the defenses and name what each one stops

Layers help when each one is honest about what it stops:

• Input classification flags suspicious or unwanted text before it reaches the model, with some false positives and negatives.
• Retrieval permissions keep content the user should not see out of the context entirely.
• Model instructions guide behavior as a preference and are not a control.
• Tool authorization decides whether an action may happen, whatever the model asked for.
• Output checks inspect text and catch some unwanted content before display.
• Monitoring records attempts and outcomes so patterns become visible afterward.

Write down, for each layer, what it is meant to stop and what it cannot stop. A layer described that way is easier to trust and easier to test.

Measure the boundary

Measure the controls that carry the weight, not only the filter. Useful measures include how many tool calls the model attempted and how many authorization checks refused them, whether any unauthorized data entered the model's context at all, and whether a request from a user without access returns nothing regardless of wording. Test the boundary with tests that do not depend on phrasing. One such test asserts that a user without access receives no data from the assistant, and the assertion holds however the request is written. Review blocked attempts with the team that owns the tool. A cluster of blocks in one category may mean the permissions are too tight, or that someone is testing the boundary on purpose.

Practice by listing the enforcing controls

List every action the assistant can take, including reads, writes, messages and calls to outside services. Beside each, name the control that enforces it without relying on the model. Where the entry says the instructions, a filter or the model will refuse, mark it as a gap. Then decide the control that closes each gap and the test that shows it works. The list is short for a simple assistant and grows with each new tool, so keep it as a living document rather than a one-time review.