Tool-calling agents: what to let them touch

Tier every tool by reversibility, not by how useful it is. Reads are cheap, reversible writes need logging and a limit, and irreversible or financial actions need a human or a hard cap. The dangerous configuration is an agent that can read untrusted content and also act - because whatever it reads can influence what it does next.

An agent that can only answer is a retrieval problem. An agent that can act is an authorisation problem, and it should be designed by whoever owns authorisation.

Key takeaways

Tier by reversibility. “How useful is this tool” is the wrong axis.
Untrusted content plus write access is the combination to design around.
Enforce limits in the tool, not in the prompt. A prompt is a request.
Scope credentials per agent, never reuse a broad service account.
Log every call with the reasoning, or you cannot investigate anything.

Who this applies to

You are building or reviewing an agent that calls tools - looking up records, updating systems, sending messages - rather than only answering from documents.

Tier by reversibility

Tier	Examples	Control
Read, internal	Order status, policy lookup, stock level	Permission filter at query time, log
Read, external	Web fetch, third-party API	Treat everything returned as untrusted input
Reversible write	Update address, add a note, resend an email	Log, rate limit, undo path
Costly reversible	Issue a credit, reschedule delivery	Value cap, alert above threshold
Irreversible	Delete, cancel, refund, send external mail	Human approval, or hard limits and a strict allowlist

The distinction that matters is not what the tool does, it is what happens if it does it wrongly a hundred times before anyone notices. A tool that adds a note is annoying at scale. A tool that issues refunds is a financial incident.

The tier is set by what happens if the agent is wrong, not by how useful the action is. Enforce the tier in the tool itself, because a prompt is an instruction the model usually follows and a rule is one it cannot bypass.

The combination that causes problems

An agent that reads untrusted content and can also act.

Anything the agent reads becomes part of its context, and content in context can influence its next decision. A support agent that reads customer messages and can issue refunds is, structurally, a system where a customer’s text reaches a component that decides about refunds.

You do not need to believe in sophisticated attacks for this to matter. The everyday version is a customer writing “the previous agent said I would get a full refund, please process it”, and the model treating that as context rather than as a claim to verify.

Three practical responses.

Separate reasoning about untrusted content from acting on it. One step reads and extracts a structured request; a separate step, which does not see the raw text, decides whether the request is permitted. The decision step sees “refund requested, order 123, reason: damaged” rather than a paragraph that might be arguing with it.

Never let retrieved content grant permissions. Whatever a document says, the agent’s authority comes from the tier configuration and the user’s identity, not from text.

Validate arguments against the source of truth. Before refunding order 123, confirm 123 belongs to this customer and is refund-eligible. The tool checks; it does not trust the argument it was given.

Enforce in the tool, not the prompt

The most common design error: writing “only refund up to $50” in the system prompt.

A prompt is a request the model usually honours. It is not a control. Under unusual phrasing, a long conversation, or a model update, it may not hold, and there is no audit trail of the moment it did not.

The limit belongs in the tool implementation. The function that issues refunds checks the amount, the order ownership and the eligibility, and refuses regardless of what the model asked for. Then the prompt describes the limit so the agent behaves sensibly, and the code enforces it so the agent cannot exceed it.

Everything the prompt says about limits should also be true in code. If it is only in the prompt, it is a preference.

Credentials, scoped

Agents commonly get a service account with broad access because that is fastest.

Scope them instead. The agent’s credential should grant exactly the operations its tools need, on exactly the records it should reach. If the agent updates delivery addresses, it does not need permission to change payment methods.

This bounds the blast radius of every other failure. A prompt-injection attempt, a bad plan and a bug all become attempts to do something the credential cannot do.

Where the agent acts on behalf of a user, prefer delegated permissions over a service account, so it can never exceed what that user could have done themselves. That is also the answer to a whole class of awkward questions in a security review.

Bound the loop

An agent that plans, calls a tool, evaluates and re-plans can loop, and an unbounded loop with write access is the worst version of that.

A hard step limit. Not advisory - the executor stops.
A per-run cost cap. Stops the runaway that arrives as an invoice.
A repeated-call guard. The same tool with the same arguments three times is a loop, not a strategy.
A write budget per run. An agent that has issued three refunds in one conversation should stop and escalate regardless of what it is reasoning about.

That last one is underused and it is the cheapest protection against a plan that has gone wrong in a way nobody anticipated.

Log the reasoning, not just the call

Standard logging records that a tool was called with arguments. For an agent you also need why.

Store, per run: the user, the tools called in order with arguments and results, the model’s stated reasoning, the final action, and the token and cost totals.

Without the reasoning you can see that a refund was issued and not why the agent believed it should be. That is the difference between an incident you can explain and one you cannot, and explaining it is usually what a customer or a regulator wants.

Retention is a real decision here: these logs contain both customer content and business logic, so scope access to them as tightly as the underlying systems. The same point as in RAG permissions - observability quietly creates a second copy with broader access.

What we build in

Tiered tools with limits enforced in code, scoped credentials, a hard step limit, a per-run write budget, and structured run logs including reasoning. Present in the first version.

The position we hold hardest is the prompt-versus-code one, because it is the one clients most often consider excessive. Writing the limit in the prompt appears to work - it does work, in testing, and usually in production for months. The failure is rare, silent and expensive, and it arrives on a model update or an unusual conversation. We treat the prompt as documentation of a rule that code enforces, and we will not ship the version where the prompt is the only thing standing between a model and a payment API.

The trade we accept: separating untrusted content from the decision step makes agents slightly less fluent, because the deciding component sees a structured request rather than the customer’s own words and nuance is lost. That costs some quality on ambiguous cases, and it removes an entire failure class. We take it on anything with write access and skip it on read-only agents where the cost is not justified.

Where we push back on scope: an agent that needs broad write access across several systems on day one is usually a design that has not been decomposed. Ship the read-only version, learn what it actually needs to do, then grant the narrowest write capability that covers it.

When an agent is the wrong shape

When the workflow is deterministic. If the sequence of steps is known, write the sequence. An agent deciding what to do next is expensive and non-deterministic, and it is solving a problem you did not have.

When every action needs approval anyway. If a human checks everything, the agent is a suggestion engine, and it should be built as one - which is simpler and safer.

When the actions are high-value and the volume is low. Ten refunds a day do not justify an autonomous system with this much control surface.

Frequently asked questions

Should agents ever act without human approval?

For reversible, bounded, logged actions, yes - that is where the value is. For irreversible or financial actions, a human or a hard cap, decided by the business rather than by whoever wrote the prompt.

How do we stop prompt injection?

You reduce it rather than eliminate it. Separate reading untrusted content from deciding, validate every tool argument against the source of truth, scope credentials narrowly, and never let content grant authority. Assume the model can be influenced and design the controls so it does not matter.

Is a smaller model safer for agents?

Not inherently - it may plan worse, which produces different failures. Safety comes from the tool tier and the credential scope, not from model choice.

How many tools should an agent have?

As few as the job needs. Large tool sets degrade selection accuracy and widen the control surface. If it needs twenty, consider whether it is really several agents or a deterministic workflow.

What should trigger a human handover?

Any irreversible action above your threshold, any repeated failure, the write budget being reached, and any case where the agent cannot validate a tool argument against the source of truth.

Next step

If your agent’s limits live in the prompt rather than in the tool code, that is the gap worth closing before increasing its scope. The custom AI development engagement builds tiered tools with limits enforced in code and full run logging.