Scoping an AI project that actually ships

There is a reliable predictor of whether an AI project reaches production, and it has almost nothing to do with the technology. It is whether the scope was drawn around a workflow or around a capability. A capability has no end state, no owner and no way to fail, so it runs until the budget does. A workflow has an input, a defined done, a defined escalation, and a number that moves. Four questions settle which one you have, and none of them are technical.

Key takeaways

A capability scope cannot be finished, only abandoned. That is the failure, and it happens before any code.
The four scoping questions are: what number moves, what is the system allowed to do, what happens when it is unsure, and who owns it in six months.
If the baseline metric cannot be computed for last month, establishing the measurement is the first deliverable, not the model.
Each phase should end with evidence and a real option to stop. A phase that cannot produce evidence is not a phase.
Start with high-volume and boring. The workflow that annoys you most is usually the one with the most exceptions.

Who this applies to

You are about to commission an AI build, or you are writing the brief for one. Also relevant if a project is already running and nobody can say what “finished” looks like.

Capability scope and workflow scope

“Add AI to customer service” is a capability. “Resolve order-status enquiries end to end in the helpdesk, escalating anything involving a refund over $200” is a workflow.

The difference is not phrasing. It is that the second one has edges.

Capability scope “Add AI to support” chatbot?summaries?ticket tagging? NO INPUT, NO DONE, NO OWNER Nothing here can be completed, so nothing here can be stopped.

Workflow scope Order-statusenquiry Retrieve order,draft answer Resolved inhelpdesk Named humanqueue MEASURED REFUND > $200 Two exits, both defined. Anything that is neither is a bug you can see.

The test is not whether the sentence sounds specific. It is whether you can name the input, both exits, and the number that moves. A scope with only one exit is a capability wearing a workflow's clothes - it has no defined behaviour for the cases it cannot handle, which is where the damage happens.

Four questions that decide the outcome

Before any technical design, push for answers to four things. Where a client cannot answer them, that gap - not the model - is the real first task.

1. What number moves, and what is it today?

Not “efficiency”. A number with a current value: 41% of tickets resolved without human touch, 6.2 hours per week per person on invoice entry, 3.1% chargeback rate.

If it cannot be computed for last month, establishing the measurement is the first deliverable. This is worth accepting cheerfully rather than treating as a delay, because a project that launches without a baseline can never answer whether it worked - see how to measure whether an AI system works for what the measurement itself involves.

2. What is the system allowed to do?

There is a large difference between a system that drafts a reply for a human to send and one that sends it. Between one that recommends a refund and one that issues it.

Each step up the permission ladder multiplies both the value and the blast radius. Decide this early, because it drives the entire architecture including the audit logging you will wish you had - see tool calling and what an agent is allowed to do.

3. What happens when it is unsure?

Every workflow needs a defined answer. Route to a named queue. Ask a clarifying question. Refuse and explain.

“It should not happen” is not an answer, it is the absence of one, and it is how confident wrong answers reach customers. The design of that gate is its own decision with real trade-offs - see confidence thresholds and escalation design.

4. Who owns it in six months?

Someone has to read the dashboard, review escalations, and decide when it needs tuning. If the answer is “the consultancy”, you have bought a dependency rather than a capability.

Name the internal owner during scoping and involve them in the build. The unowned system is the one that degrades quietly, because degradation is only visible to somebody who is looking.

What the sequence actually looks like

For a single well-scoped workflow, the shape is fairly consistent. The durations below are for a small workflow with a cooperative data situation; a slower team needs longer for the same sequence.

Phase	Rough span	Deliverable	Evidence produced	The stop decision
Diagnose	1-2 days	Ranked candidate workflows, effort and payback against each	Real volumes, measured handling time, baseline metric	Is any of this worth building at all
Prove	~1 week	Core workflow run offline against historical data	Accuracy, escalation rate, cost per task, and the failure cases	Does the quality clear the bar before anything is live
Integrate	~1 week	Systems of record, permissions, audit logging, PII handling, escalation path	Shadow-mode comparison against the current process	Does it behave the same on live traffic as on history
Operate	~1 week	Monitoring, alerting, runbook, handover	Live performance on a slice of traffic, held against the baseline	Do we widen the slice or roll back

The weeks are not the point. The sequence is, and specifically that each phase ends with evidence and a genuine option to stop. What nobody should do is drop a phase to hit a date, because the phases are not equally droppable - Prove is the one under pressure and the one that makes the rest cheap.

What the brief should actually say

The difference between the two scopes is usually four edits to one sentence. It is worth doing them on paper before anyone quotes, because a vague brief gets quoted vaguely and you will be comparing two guesses.

Take a real starting point: “We want AI to help with our support inbox.”

Add the input. Which messages? “Order-status enquiries arriving in the helpdesk” excludes the hard categories you were not thinking about when you wrote the sentence.

Add the permission. Drafts a reply, or sends it? Say which. This single word changes the architecture, the review burden and the risk, and leaving it out means the quote you receive has silently assumed one of them.

Add the escalation. “Anything involving a refund over $200 goes to the returns queue” gives the system a defined behaviour for what it cannot do. Without it, the system’s behaviour on the hard cases is whatever falls out.

Add the number. “Currently 41% of these resolve without a human, measured in the helpdesk” is what turns the whole thing from a description into a target.

What you end up with is longer and considerably less exciting, and it is the version two agencies can quote comparably. It is also the version that can be finished, which is the point - the four additions are exactly the four questions above, written down.

If you cannot make those four edits, that is the useful finding. It means the decision about what the system is allowed to do has not been made yet, and no amount of technical detail in a proposal will substitute for it.

The scoping mistakes we see most

Starting with the hardest case. Teams pick the workflow that annoys them most, which is usually the one with the most exceptions. Start with high-volume and boring, and earn the hard one.

Automating a broken process. If the process is bad, automation makes it bad at scale, faster. Sometimes the diagnose phase concludes that three steps should be deleted and no AI is needed. That is a good outcome, not a failed engagement - see where the hours actually go for how to tell the two apart.

Skipping the offline evaluation. It feels faster to go straight to a live pilot. It is not. It moves discovery of the failure modes to the point where customers find them, and it forfeits the launch baseline permanently - see we shipped without an eval set.

No holdout. Without a control group you will attribute every good number that quarter to the new system, and so will everyone else, until someone asks a hard question you cannot answer.

Scoping the build without scoping the running. The proposal covers delivery and stops. Six months later nobody has budget for the monitoring, the evaluation refresh or the person who reads the escalations, so those things do not happen.

What we do differently because of this

The diagnose phase is a fixed-fee deliverable that ends in a written roadmap, and the roadmap is the client’s whether or not they build with us.

The reason is that it removes the incentive to find a project. An agency that only earns money if there is a build has a bad reason to conclude that there is one, and clients are right to notice. Separating the two means we can say “delete these three steps and you do not need us”, which we have done, and which is the single most useful thing we produce on some engagements.

The question we push hardest on is the fourth one, ownership, because it is the one clients most want to defer. It sounds like an org chart question and it is actually the load-bearing one: everything else on this list decays without somebody whose job includes reading the output.

Where our own position costs us: quoting four phases against a competitor quoting a build makes our number look larger for the same visible outcome. We have lost work on that comparison. The honest framing is that the competitor’s number is not smaller, it is less complete, and a client cannot see that from two PDFs - see how to evaluate an AI agency proposal for the specific things to check.

When this shape is wrong

Genuinely exploratory research, where the output is a finding rather than a system. Forcing it into phased delivery produces a fake roadmap.

A custom model on proprietary data, a regulated workflow needing formal validation, or a multi-system migration. The response is not to compress it. It is to break it into pieces that each end with something demonstrable, so the programme keeps producing evidence instead of promises.

Very small internal tools where the whole build is two days. The overhead of four phases exceeds the work.

The rule of thumb: if you cannot describe what you will be able to show at the end of the next three weeks, the scope is still too vague to start.

Frequently asked questions

What if we cannot compute the baseline metric?

Then that is the first project, and it is usually days rather than weeks. It is also independently valuable, because a business that cannot measure a workflow cannot tell whether any intervention improved it, AI or otherwise.

Should the pilot use real customer traffic?

Not at first. Run offline against historical data, then in shadow mode alongside the existing process, then on a slice of live traffic. Each step is cheap to reverse and the order is what makes the failure modes surface somewhere they cost nothing.

How narrow is too narrow for a first workflow?

If the volume is under roughly fifty instances a week you will struggle to evaluate it, because differences will not be distinguishable from noise. That is the practical floor, not the value of the task.

Who should own it internally?

Someone in the team whose work the system touches, not someone in IT who has no view of the outcome. The useful test is whether they would notice the system getting worse without being told.

Can we scope several workflows at once?

Scope them, yes - the diagnose phase should rank more than one. Build them at once, no. The second workflow is much cheaper after the first, because the integration, logging and evaluation harness already exist.

Next step

If you have a project that cannot answer the four questions above, that is the work to do first and it is smaller than the build. The AI consulting and strategy engagement runs the diagnose phase as fixed-fee work and the roadmap is yours either way.