Zero Trust, Restated for AI Tools

Zero trust is often reduced to a product category, which hides what the principle asks of a design. The principle itself is simpler. A request is not trusted because of where it comes from. Each access is checked against identity, context and policy, and every component is assumed to be compromised at some point. AI tools add a component that does not fit neatly into that picture. The model reads text from many sources, produces text that other systems act on, and can be steered by any of its inputs. Restating the principle for that component is the useful exercise.

Verify identity on whose behalf the model acts

An assistant that answers questions over company data needs an identity for every action it takes. The common shortcut is a single service account that can read everything, with the application deciding what each user sees. That shortcut means the database trusts the application completely, and the application trusts its reading of the model's request. If the model is misled, the service account still has the access, not the person who asked.

The stronger pattern makes the model act on behalf of the user. Each query or tool call runs with that user's delegated permissions, so the system of record enforces what the person may see. The model can still be wrong about what to fetch, but it cannot fetch what the person could not already reach. A simple test helps here. Ask what would happen if the model returned a request for another employee's records. If the answer depends on the model behaving well, the boundary sits in the wrong place.

Treat model output as untrusted input

The model's output is generated from everything in its context, including user text and retrieved documents. Some of that text may have been written by someone hoping to influence the result, so the output carries the same uncertainty as any other input from an uncertain source. Downstream systems should validate it before acting. If the output is meant to be a date, parse it as a date. If it is meant to become a database filter, do not pass it through unchanged. If it names a recipient, check the recipient against an allowlist.

Rendering needs particular attention. When a client displays model output as rich content, such as images or links, the act of rendering can send a request to an external host. Restrict what the interface renders, and treat outbound requests from a rendering client as something to monitor. These are familiar web application defenses, applied to generated text.

Treat retrieved content as data with provenance

Documents pulled into a prompt form the second input stream. A document may be accurate, outdated, or written by someone outside the organization. Retrieved text can also contain sentences that read like instructions, and the model cannot always tell a fact to report from a command to follow. The system should keep that distinction itself. Store the source, owner and date with each chunk, and present retrieved text in a clearly delimited block marked as material to read, not to obey.

Labels help, but the stronger control is an access check at retrieval time against the requesting user's permissions. Content the user should not see never reaches the model, so it cannot leak through a summary. Content from an untrusted source can be given lower weight in the design, for example by being shown with a warning or kept out of tool decisions entirely.

Segment tool access by risk

Tools differ in what they can do, and that difference should shape how they are granted. A read-only lookup of public documentation carries little risk. A tool that messages customers, changes records or moves money carries a great deal. Group tools into tiers and give each tier its own identity, logging and approval rule. Watch for a low-risk tool becoming a route to a high-risk one, for example by returning a credential that another tool accepts.

Network segmentation matters too. The runtime hosting a model should reach only the services it needs. Egress rules that allow arbitrary outbound connections make the model's mistakes, and any injected instructions, far more costly. A short list of allowed destinations is easier to defend than a general permission to connect anywhere.

Check each action in context

Even with sound identity and segmentation, some decisions depend on the situation. Is this user allowed to perform this action on this record today? Does the requested amount fall within the limit set for this task? Has the person confirmed an action that cannot be undone? These checks belong in the tool layer, written as code the model cannot override. They add friction, so choose confirmations by consequence rather than by how risky an action feels. Asking for confirmation on a harmless lookup teaches people to click through the prompt that matters.

Practice by tracing one feature

Pick one AI feature, real or in a lab, and trace it from input to action. Write each hop on its own line: where the text comes from, which identity it runs under, what it can read, what it can call, and what check happens before the call. Mark each hop as trusted, verified or untrusted. Any place where one step's output becomes a privileged step's input without a check is a gap. Note each gap and the check that would close it. The gaps this exercise finds are usually the ones an incident would otherwise expose later.