On this page 11 sections
If a task has a right answer that can be computed, a model is the wrong tool for it. Lookups, arithmetic, format conversion, deduplication, threshold rules and anything already encoded in a system should be code. Use a model where the input is genuinely unstructured or the judgement is genuinely fuzzy, which in most workflows is one or two steps out of eight.
The question is not whether a model can do it. It is whether anything else can do it correctly every time, faster and for nothing.
Key takeaways
- A model is a probabilistic component. Putting one where a deterministic answer exists trades correctness for nothing.
- Cost and latency are the small objections. Untestability is the large one.
- The tell is whether two competent people would give the same answer from the same input. If yes, it is code.
- Most real workflows are mostly deterministic with one or two genuinely fuzzy steps.
- Wrapping a rule in a model does not make a system intelligent. It makes it unpredictable and slower.
Who this applies to
You are designing or reviewing a workflow that involves a model, and want to know which parts of it should. This is about the task, not the architecture. Whether a job needs one agent, several, or a pipeline is multi-agent systems: when the complexity pays, and which lever fixes a quality problem is fine-tuning, retrieval or a better prompt.
The test
Would two competent people, given the same input and the same rules, produce the same output?
If yes, the task is deterministic and belongs in code. There is a correct answer, it can be computed, and a model can only approximate it.
If no, because the input is ambiguous or the judgement genuinely varies, a model is a reasonable tool and you should expect to measure it.
That single question resolves most of these arguments in a meeting, and it is more useful than any list, because it also handles the cases nobody anticipated.
Eight jobs that keep getting given to a model
| Task | What it should be |
|---|---|
| Looking up a status, price or entitlement | A query against the system that holds it |
| Arithmetic, totals, pro-rata, tax | Code, always, with no exceptions |
| Reformatting between structured formats | A parser or a mapping |
| Validating an email, VAT number, postcode, SKU | A validator, and the authoritative check where one exists |
| Deduplicating records | A matching rule, then a model only for the genuinely ambiguous remainder |
| Routing on a value you already have | A conditional |
| Applying a documented threshold | A rule, so it can be shown to an auditor |
| Extracting from a fixed template | A template parser |
Two of these deserve the emphasis.
Arithmetic. A model producing a number that a calculation could have produced is the clearest example of the whole category. It is slower, it costs money per attempt, and it can be wrong in a way that looks entirely plausible. If a total appears on an invoice, a customer’s screen or a report, it should be computed.
Thresholds and documented rules. “Refunds under twenty pounds are automatic” is a business rule. Implemented as a prompt instruction it is a strong suggestion. Implemented as a conditional it is a policy you can show to a regulator, test in a unit test, and change without re-testing everything else.
The reason that is not cost
Cost and latency are the objections people reach for, and they are the weaker two.
The real problem is that a deterministic function can be tested exhaustively and a model cannot. A rule with four branches has four test cases and is then permanently settled. The same logic in a prompt has an input space you cannot enumerate, needs an evaluation set, needs re-testing whenever the model version moves, and will still surprise you on an input nobody thought of.
Every model call you add is a component that has to be measured forever. That is affordable for the steps that need one, and it is a strange price to pay for a conditional.
There is a second-order effect worth naming. Systems that use a model for their deterministic steps are hard to debug, because a failure could be anywhere. When the model is doing one clearly bounded job, a wrong output tells you immediately which component failed.
Where a model genuinely earns its place
The complement of the list above, and it is not small.
Unstructured input. A free-text message, a document with inconsistent layout, an email thread. Anything where the shape varies unpredictably is exactly what models are for.
Classification with fuzzy boundaries. Intent, sentiment, urgency, topic. There is no rule that separates a complaint from a firm question, and both a rule and a person would disagree with themselves.
Generation. Drafting, summarising, rewriting for an audience.
Judgement under ambiguity. Which of these three policies applies to an unusual case. This is where measurement matters most, and where confidence thresholds belong.
The pattern that works is a deterministic pipeline with model calls at the specific steps that need one. Code decides what happens next; the model handles the step where the input is genuinely messy. That structure is testable, debuggable and cheap, and it is what most well-built systems in this space actually are underneath.
A worked example
A support workflow that looks like it needs a model throughout.
An email arrives. Classify the intent - model, because the input is free text. Identify the customer from the address - code, it is a lookup. Fetch the order - code. Decide whether the return window is open - code, it is a date comparison against a documented rule. Decide whether this unusual case merits an exception - model, with a threshold and an escalation path. Draft the reply - model. Calculate the refund amount - code, always. Send it - code, behind whatever approval the amount requires.
Eight steps, three model calls, and the three are the ones where something is genuinely ambiguous. Built the other way, as one prompt that receives everything and decides everything, the same workflow is slower, more expensive, impossible to unit test, and will eventually get a refund amount wrong in front of a customer.
What we push back on
The request we decline most often is a model call in the middle of a workflow that has a lookup on both sides of it. Usually it arrived because the workflow was designed as “an AI workflow” and every step inherited the assumption.
The conversation that works is the two-people test applied step by step, out loud, in the first session. It takes twenty minutes and it typically removes half the model calls from the design, which makes the remaining ones cheaper to run and much easier to defend when they are wrong.
Where this costs us: a workflow with two model calls is a smaller engagement than a workflow with eight, and it is easier for a client to conclude that the thing they were sold as AI is mostly ordinary software. That conclusion is correct. Most good systems in this field are mostly ordinary software, and the part that is not is the part worth paying attention to.
Where we have been wrong: we have used a model for fuzzy record matching where a normalisation step and an edit-distance rule would have handled almost all of it. The model was better on the genuinely ambiguous remainder, which is the correct place for it, and we had it doing the easy nine-tenths as well, invisibly, at cost, until somebody looked.
When this does not apply
When the deterministic version is a research project. If encoding the rule means reverse-engineering fifteen years of undocumented exceptions, a model over examples may genuinely be the cheaper path. Measure it like anything else.
Prototypes. Getting something in front of people quickly is a legitimate reason to do the crude version. It becomes a problem when the prototype ships.
When the volume is trivial. Ten runs a day makes the cost argument irrelevant, though the testability argument survives.
Frequently asked questions
Is it not simpler to have one model do everything?
Simpler to write and harder to own. One prompt handling eight steps cannot be tested per step, so every change re-opens all of them, and a failure gives you no information about where it happened.
What about a model that calls tools for the deterministic parts?
That is the right pattern, and the point stands about which parts are tools. Let the model call a calculator rather than doing arithmetic, and validate every argument it passes, as in tool-calling agents.
How do I convince a stakeholder who wants it to be AI?
Show the two-people test on their own workflow. It is concrete and it usually converts, because nobody defends spending money to make arithmetic unreliable once it is put that way.
Does this change as models get better?
The economics shift, the argument does not. A better model still cannot be exhaustively tested, and a lookup will still be correct every time for nothing.
Where is the boundary on classification?
If the categories are defined by values you already hold, it is a conditional. If they are defined by what somebody meant, it is a model. Order status is the first; whether a message is a complaint is the second.
Next step
If a workflow in front of you has a model at every step, the two-people test applied step by step will usually remove most of them in one session. Business process automation designs these as deterministic pipelines with model calls where they are earned.
Related: Multi-agent systems: when the complexity pays · Fine-tuning, retrieval or a better prompt · Structured output: reliable JSON · Where the hours actually go · Business process automation