On this page 12 sections
- Key takeaways
- Who this applies to
- Why one number is the wrong target
- Measure “unsupported”, not “wrong”
- Defensible thresholds by category
- What actually reduces fabrication
- Measuring it without a research team
- The trade nobody states plainly
- What we hold ourselves to
- When this is not worth measuring formally
- Frequently asked questions
- Next step
There is no single acceptable hallucination rate, because fabrications differ by cost. The useful target is per-category: near zero for anything a customer will act on financially or legally, and a tolerated few percent for low-stakes informational answers. Measure “unsupported by source” rather than “wrong”, because that is the failure you can actually engineer against.
Asking for a system’s hallucination rate is a reasonable instinct and an unhelpful question. The answer is either meaningless or misleading, and the reason why points at how to fix it.
Key takeaways
- Measure unsupported claims, not incorrect claims. Support is checkable; truth is not always.
- One global rate hides the failures that matter. Segment by consequence.
- Retrieval quality gating removes more fabrication than any prompt instruction.
- Requiring citations makes fabrication visible rather than preventing it. That is still worth a lot.
- A system that refuses appropriately will score worse on coverage and better on trust.
Who this applies to
You are setting a quality bar for a production AI system, accepting delivery of one, or answering an internal question about whether it is safe to put in front of customers.
Why one number is the wrong target
Consider three fabrications from the same support agent:
- It states your office closes at 6pm when it closes at 5:30pm.
- It invents a returns window of 60 days when your policy is 30.
- It tells a customer their warranty covers water damage when it does not.
A single hallucination rate treats these identically. In reality the first is trivial, the second creates an obligation you may have to honour, and the third may be a regulatory problem depending on your sector.
Any system will produce some rate of fabrication. The engineering question is not how to get one number to zero. It is which categories must be near zero, and what you are willing to spend to get them there.
Measure “unsupported”, not “wrong”
The practical shift that makes this tractable.
Judging whether an answer is true requires a source of truth for every possible question, which you do not have. Judging whether an answer is supported by the retrieved source requires only the source you already retrieved, and it is checkable automatically.
So the metric becomes: for each factual claim in the answer, is there a passage in the retrieved context that supports it?
This is measurable at scale, it can be scored by a second model with reasonable reliability, and it targets the exact failure that hurts: the system stating something about your business that your documentation does not say. It also catches the dangerous case that a truth-based metric misses - an answer that happens to be correct but was not grounded in anything, which will be wrong next time.
Two secondary metrics worth tracking alongside:
- Citation validity: does the cited source exist and does it actually contain the claim? Fabricated citations are common and are the most damaging kind, because they defeat the customer’s ability to check.
- Appropriate refusal rate: of the cases where nothing in your content answers the question, what share did the system decline rather than improvise?
Defensible thresholds by category
Targets that hold up in practice for a retrieval-based business system, measured as unsupported-claim rate:
| Category | Target | Rationale |
|---|---|---|
| Financial: prices, refunds, fees, billing | < 0.5% | Customer acts on it; you may be held to it |
| Policy and contractual terms | < 0.5% | Creates obligations, and may be legally binding |
| Regulated: medical, legal, safety | ~0%, or do not answer | The correct design is usually to refuse and route |
| Account-specific facts | < 1% | Should come from a system lookup, not generation |
| Product capability claims | < 2% | Drives purchase decisions, creates returns |
| General informational | < 5% | Low cost, high volume, cheap to correct |
The first three are not achieved by better prompting. They are achieved by not letting the model generate them freely - retrieve the number from the system of record, template the response, and refuse when the lookup fails.
That is the underlying principle: for high-stakes facts, generation is the wrong mechanism. Use the model to understand the question and to write the sentence, and use a database to supply the fact.
What actually reduces fabrication
In rough order of effect.
1. Gate on retrieval quality. If nothing relevant was retrieved, do not answer. This is the single biggest reduction available and it is a configuration change rather than a research problem. See confidence thresholds and escalation design.
2. Get facts from systems, not from the model. Order status, balances, entitlements, prices. A lookup that fails is a refusal; a model that guesses is a fabrication.
3. Require citations. This does not prevent fabrication, but it makes it visible - to the customer, to your reviewers, and to your evaluation harness. A claim with no citation is a flag. A claim with a citation that does not support it is a scoreable failure.
4. Fix the content. A large share of apparent hallucination is the system faithfully retrieving a page that is out of date. The model is behaving correctly and your documentation is lying. This is cheap to fix and frequently the largest single cause.
5. Narrow the scope. A system that declines to discuss anything outside a defined domain fabricates less than one that attempts everything. Coverage and accuracy trade against each other, and the trade should be deliberate.
6. Prompt instructions. Last, deliberately. “Only answer from the provided context” helps somewhat and is not reliable under pressure. Treat it as a nudge, not a control.
Measuring it without a research team
A workable method:
- Take 200 cases from your evaluation set, stratified by consequence rather than by volume.
- Run them. Capture the answer and the retrieved context together.
- For each answer, split it into factual claims. A model can do this.
- For each claim, score supported / unsupported / partially supported against the retrieved context.
- Calibrate: have a human score 40 of them and measure agreement with the automatic scoring. If agreement is poor, fix the rubric before trusting the numbers.
- Report per category, with the count of cases in each so nobody over-reads a rate computed on nine examples.
Run monthly, and on every change to prompts, retrieval or model version. Provider updates shift this metric without any change on your side - see your provider changed the model, what broke.
The trade nobody states plainly
Reducing fabrication reduces coverage. A system tuned to refuse whenever grounding is weak will answer fewer questions, and its deflection rate will fall. It also tends to answer more slowly, because verification costs time - see latency budgets for conversational AI.
That trade is usually worth making and it is usually made in the wrong direction, because deflection rate is on a dashboard and fabrication rate is not. A system that answers 70% of contacts with a 6% fabrication rate looks better in a monthly report than one that answers 55% with 1%, and it is worse for the business by any measure that includes complaints, rework and trust.
The fix is procedural rather than technical: report both numbers together, always, and never let deflection appear in a review without the fabrication rate beside it.
What we hold ourselves to
Our default position on new builds is that any answer stating a price, a policy term, or an account-specific fact must be traceable to a retrieved source or a system lookup, and where it cannot be, the system refuses and escalates. That is a design constraint set before tuning, not a target negotiated afterwards.
It has a visible cost. Systems built this way have lower deflection rates in week one than systems that answer more freely, and that is a genuinely uncomfortable number to present at a first review. We present it anyway, alongside the fabrication rate, because the alternative is a system whose apparent success is being purchased with unmeasured risk that lands later.
The other thing we do is treat “answered from a weak source” as its own failure category rather than folding it into general inaccuracy. It has a different fix - content or retrieval, not reasoning - and separating them is what makes the report actionable rather than merely informative.
When this is not worth measuring formally
Internal tools with expert users. A researcher using an assistant to draft summaries will notice fabrication immediately and the cost of an error is a moment of their time.
Systems that only classify or route. There is nothing to fabricate. Measure precision and recall instead.
Creative or drafting tools where the output is a starting point a human edits. Fabrication is a normal part of a draft.
Before a system does anything real. Do not build a claim-level scoring pipeline for a prototype. Read a hundred outputs.
Frequently asked questions
What hallucination rate do commercial systems achieve?
Published figures vary so widely by task, measurement method and definition that cross-vendor comparison is close to meaningless. Measure your own system on your own cases. Treat any vendor quoting a single headline rate without a method as marketing.
Does retrieval eliminate hallucination?
No. It reduces it substantially and changes its character - a grounded system fabricates less often but can still misread, over-generalise, or blend two sources. It also introduces a new failure: faithfully retrieving something out of date.
Can we just tell the model not to make things up?
It helps at the margin and it is not a control. Instructions are followed most of the time, and “most of the time” is not a threshold you can put in a policy document.
Is a bigger model less prone to this?
Generally somewhat, but the relationship is not reliable enough to plan around, and it costs more per request. Better retrieval usually buys more accuracy per dollar than a larger model.
How do we explain this risk to a non-technical stakeholder?
The honest framing: the system will occasionally state something your documentation does not say. We measure how often, we make it near zero where it costs money, and we accept a small rate where it does not. That is a normal engineering trade, and it is a better position than a system where nobody has measured it at all.
Next step
If nobody has measured this on your system, that is the common case. The AI evaluation and QA engagement produces the claim-level baseline, per category, and wires it into a schedule.
Where the answer has to be defensible afterwards rather than merely correct, the record you need is a logging decision taken in week one - see responsible AI as an engineering decision.
Related: How to measure whether an AI system works · Confidence thresholds and escalation design · Building an eval set from real tickets · AI evaluation and QA