What you own after an AI project: code, prompts and evals

Owning an AI system means holding eleven things: the source code, the prompts, the evaluation set and harness, the retrieval configuration, the infrastructure definition, the data pipeline, the credentials, the model configuration, the runbook, the decision log and the training data if any exists. Contracts routinely transfer the first and stay silent on the rest, which is enough to make you dependent without anyone intending it.

The test is simple: if the vendor stopped answering email tomorrow, could another team keep this running and improve it? Most buyers assume yes and discover otherwise.

Key takeaways

The evaluation set is the artefact most often retained by vendors and the most expensive to rebuild.
“You own the code” is not the same as “you can run the system”.
Prompts are the operating logic. They must be in your repository, versioned, with the reasoning recorded.
The decision log is worth more at month eighteen than the code is.
Test ownership at handover with a dry run, not with a clause.

Who this applies to

You are contracting an AI build, or taking delivery of one, and you want to know what to ask for. Also relevant if you already have a system and want to establish what you actually hold.

The eleven artefacts

#	Artefact	Why it matters	Commonly withheld
1	Application source code	Obvious, and usually transferred	Rarely
2	Prompts, with version history	The operating logic of the system	Sometimes
3	Evaluation set and scoring harness	The only way to verify quality or safely change anything	Often
4	Retrieval configuration	Chunking, embedding, ranking, thresholds. Changes behaviour more than the prompt	Often
5	Infrastructure as code	Reproducing the environment without archaeology	Sometimes
6	Data pipeline and transformations	How your content became indexable	Sometimes
7	Accounts and credentials	In your name, on your billing	Often
8	Model configuration	Versions, parameters, routing rules	Sometimes
9	Runbook	What breaks, what to check, how to roll back	Usually
10	Decision log	Why it was built this way, what was tried and rejected	Usually
11	Training or fine-tuning data	Only if fine-tuning was used	Sometimes

The four marked “often” or “usually” are the ones that decide whether ownership is real.

The right-hand two columns are the ones to write into the contract. The evaluation set and the decision log are what you need to change the system safely, and they are the two most often missing at handover.

The three that matter most

The evaluation set

The most valuable artefact in an AI project and the one most likely to sit on a vendor’s laptop.

Without it you cannot verify a quality claim, safely change a prompt, detect a regression when the provider updates the model, or hand the system to anyone else. Rebuilding it means re-labelling several hundred real cases: days of domain-expert time, and often the most expensive part of an inherited system.

It is also the artefact that vendors most reasonably forget to transfer, because it feels like their working material rather than your deliverable. Name it explicitly. See building an eval set from real tickets.

The prompts, with reasoning

Prompts are not configuration, they are logic. They encode which cases escalate, what tone is used, which policies are enforced, and dozens of small decisions made in response to specific failures.

A prompt without its history is a set of instructions nobody dares change, because every line might be load-bearing and nobody knows which. That is how systems become frozen.

Ask for them in your repository, in version control, with commit messages that say what each change was fixing. If a vendor stores prompts in their own platform rather than your repository, you have a dependency regardless of what the contract says about ownership.

The decision log

The least glamorous and the one that matters most at month eighteen.

A short record of the significant choices: why this retrieval strategy, why this model, what was tried and rejected and why, what the known limitations are, and what was deliberately deferred.

Without it, the next team re-derives everything by experiment, including the experiments that already failed. A few pages written during delivery saves weeks later, and no one ever writes it retrospectively.

The credentials problem

A specific trap worth its own section.

If the model API account, the vector database, the hosting and the monitoring are in the vendor’s name and on the vendor’s billing, you do not control the system regardless of who owns the code. Ending the relationship means migrating infrastructure under time pressure, and in the worst case the system stops when an invoice is disputed.

All production accounts should be in your name, on your billing, with the vendor holding scoped access you can revoke. This is normal practice, costs nothing, and a vendor who resists it is telling you something.

The same applies to the domain, the repository organisation and the monitoring dashboards.

Contract language that works

Ownership clauses in software contracts are frequently written for ordinary software and miss AI-specific artefacts. Practical additions:

Define deliverables by enumeration. List the eleven artefacts rather than saying “all work product”. Enumeration removes the argument about whether an evaluation set is a deliverable or a tool.

Require the repository to be yours from day one. Not delivered at the end. Work happens in your organisation’s repository, and this also means you can see progress.

Name the accounts. Explicitly: production accounts in the client’s name, vendor access scoped and revocable.

Require a handover package as a milestone. With payment attached. Runbook, decision log, evaluation set, architecture overview. Tie the final payment to it, because handover documentation written after final payment does not get written.

Address third-party components. Which open-source licences are in the dependency tree, and does any of it restrict commercial use? Worth a clause and worth an actual check.

Cover the model provider relationship. Whose terms govern, whose data-processing agreement applies, and what happens to conversation logs.

The handover test

The clause is not the proof. Test it before final payment:

The dry run. A person on your side, or a third party, deploys the system from your repository into a clean environment using only the documentation. Time it. Note every point they had to ask a question.
The change test. Make a small change - add an intent, adjust a threshold - run the evaluation set, and deploy. If your team cannot do this unaided, you have documentation rather than ownership.
The incident test. Take a real failure from the logs and ask someone to diagnose it with the runbook alone.

Failing these is normal and useful. It shows exactly where the handover is thin while the vendor is still engaged and still being paid.

Where we land on this

We publish “full handover, no lock-in” on the home page, so it is fair to say what it means concretely and where it costs us.

Work happens in the client’s repository from the first commit, production accounts are in the client’s name, and the evaluation set is a named deliverable rather than our working material. The handover package - runbook, decision log, architecture overview - is a milestone with payment attached, because we know from the shape of the incentive that documentation written after final payment does not get written well.

What this costs us is straightforward: it makes us easy to leave. A client who can hand the system to their own team, or to a cheaper supplier, at any point, is a client we have to keep by being useful rather than by being difficult to remove. We think that is the right trade for a services business, and it is also the honest position given how often we are called in to rescue systems that nobody can modify.

The place we do not go: we will not write a runbook for a system we did not build without inspecting it first, and we will not claim a decision log can be reconstructed accurately after the fact. Both of those are things clients sometimes ask for on inherited systems, and the truthful answer is that some of that context is simply gone.

When ownership matters less

Genuine SaaS. When you buy a subscription product, you are renting deliberately and the economics reflect that. Ownership questions there are about data portability and exit, not source code.

Short-lived systems. A campaign tool for one quarter does not need a decision log.

When you have no capacity to own it. Ownership without a technical owner is theoretical. If nobody will maintain it, a managed arrangement may be the honest choice - just make it a deliberate decision rather than an accident.

Frequently asked questions

Is “we own the code” enough?

No. Code without the evaluation set, the retrieval configuration and the credentials produces a system you legally own and practically cannot operate.

What if the vendor uses their own framework?

Ask two questions: is it open source with a permissive licence, and can we modify and deploy it without their involvement? A proprietary internal framework is a dependency even when the application code is yours.

Who owns the prompts if they were written by their team?

You should, and say so explicitly. Prompts are work product created for you. Vendors sometimes claim a general methodology, which is reasonable for their internal patterns and not for the specific prompts running your system.

Do we own the model?

You own your fine-tuned artefacts if fine-tuning was used, subject to the provider’s terms. You do not own the base model. Where the fine-tuned weights live and whether they are portable is worth checking before choosing fine-tuning at all.

What about conversation data?

Yours, and confirm where it is stored, how long it is retained and whether the model provider may use it. This belongs in the data-processing agreement, not in a verbal assurance.

Next step

If you have inherited a system and cannot answer these questions about it, the AI system rescue engagement is a read-only assessment: what it does, how well, what it costs, what would break it, and what is missing from the handover.

If nobody internally can hold that ownership yet, the interim answer is usually a fractional AI lead rather than a hire, and the longer answer is building the capability in the team - see AI training that changes what people do on Monday.