Tool Use and Agents: A Model That Can Act Needs Boundaries

An agent asked to clear old files from a project folder deleted far more than intended. The request was reasonable and the plan looked sensible. The pattern used to match files was broader than the person meant, and nothing in the system stopped the deletion, because the only boundary lived in the instructions. Tool use turns a model's mistakes into actions, and actions need limits that do not depend on the model being right.

What a tool call really is

A model with tools does not run them. It produces a structured request that names a tool and supplies arguments, and the surrounding program decides whether to execute that request. The program runs the function, returns the result to the model as text, and the model continues. The loop repeats until the model produces a final answer or the program stops it. In outline it looks like this:

steps = 0 while steps < MAX_STEPS: reply = model.next(context) if reply.is_final: return reply.text check(reply.tool, reply.arguments) # validate and authorize result = run_tool(reply.tool, reply.arguments) log(reply, result) context.append(result) steps += 1

The whole arrangement depends on that program. The model proposes and the program disposes. Keeping the distinction in view is the basis for every boundary discussed below.

Why the model's choices are not the boundary

A model can choose the wrong tool, misread an argument or follow an instruction that arrived inside a document it was reading. It can also reason its way to a plausible but unjustified action. Any boundary that exists only in the prompt is a request the model may decline to honor. Boundaries have to be enforced by the code that runs the tools: argument types and ranges validated, an allowlist of permitted operations, credentials scoped to the minimum needed, and limits on steps and calls per request. Design each tool to be safe when called twice. A retry after a timeout should not send the same message or create a second record, so writes should carry an identifier that makes a repeated call harmless. Rate limits on each tool cap the damage a runaway loop can do if another limit fails.

The folder incident illustrates the point. A tool that accepts a pattern and deletes every match is dangerous whatever the instructions say. A tool that accepts a list of exact paths, confirms that each lies inside the project, and refuses anything else is far safer, even when the model proposes the same plan.

Confirmation for consequences

Classify each tool by what it does to the world. Reads that change nothing can usually run freely. Writes that can be undone may run with logging. Actions that are hard to reverse, that send messages outside the organization, that spend money or that change access should require confirmation from a person, who sees the exact parameters before approving. A confirmation that asks only "Proceed?" is not a real confirmation. It should show the recipients, the amount or the paths, so that the approver is judging the action itself.

Set the rule by reversibility rather than by how the task looks. A harmless-looking action can be irreversible, and a dramatic-looking one can be a simple edit with a backup behind it.

Logging and reconstruction

Log each tool call with its arguments, its result, the time and the context that led to it. Logs are how a team learns what happened when something goes wrong, and they are also how patterns surface before anything goes wrong. Watch for loops that repeat the same call, for unusual arguments, and for calls to tools a given task should never need. Alerts on those patterns are cheap and catch problems early. Keep the logs tied to the model version and the instructions that were active, because a behavior that cannot be reproduced is hard to fix.

Containing a bad outcome

Assume that some action will eventually go wrong, and design for that before the first deployment. Where possible, make actions undoable: keep backups before destructive changes, use soft deletes and record enough state to reverse an edit. Run new capabilities in a staging environment with copies of realistic data. Give the system a kill switch that stops tool execution without taking down the rest of the product. Scope credentials so that one faulty agent cannot reach every system. Practice recovery as well as prevention, because a rollback that has never been tried is still only a plan on paper.

• Backups or soft deletes before any destructive operation.
• A staging copy of realistic data for every new capability.
• A kill switch that stops tool calls without stopping the product.
• Credentials scoped to one system and one purpose.
• A rollback that has been rehearsed at least once before launch.

Practice: specify one tool and break it

Choose one action the product might need, such as sending a reminder, updating a record or creating a file. Write a short specification for it: the name, the inputs with their validation rules, what the tool can read and write, which confirmation rule applies, the fields it logs and the worst plausible outcome. For that outcome, describe the containment: what could be undone, who would be told, and how quickly the capability could be switched off. Then give the specification to a colleague and ask them to break it. Their attempts will expose boundaries that exist only in the author's head, and those are the ones to move into the code.