On this page 10 sections
Enforce permissions inside the retrieval query, not on the results it returns. Post-filtering leaks through summaries, citations and answer text even when the final document list is correct - the model has already read what it should not have. Then close the four paths that survive a correct filter: stale permissions, the index itself, conversation history, and logs.
The failure here is not subtle when it happens. An employee asks an internal assistant a reasonable question and gets a number from a document they cannot open.
Key takeaways
- Filter in the query. Anything the model reads has already leaked, whatever you do next.
- Permissions belong on the chunk, not just the document.
- Re-check at query time. A cached permission set is a stale permission set.
- The index is a copy of your documents with a different access model. Secure it as such.
- Conversation history persists access to content the user has since lost.
Who this applies to
You are building or reviewing an internal assistant over company documents - a knowledge assistant, a support copilot, anything retrieving from sources with differing access rules.
The mistake
The intuitive design retrieves the best-matching passages, then removes the ones the user cannot see, then answers.
That is too late. By the time filtering happens, the restricted content has been retrieved and, in most implementations, sent to the model. If the model has seen it, it can appear in the summary, influence the answer, or be quoted directly - even when the citation list correctly shows only permitted documents.
Post-filtering produces a system that looks correct in testing, because the visible citations are right, and leaks through the prose.
Enforcing at query time
The user’s permission set becomes part of the retrieval query, so restricted chunks are never candidates.
Attach permissions to the chunk. When indexing, carry the source document’s access control onto every chunk derived from it. A document split into forty passages needs the permission on all forty, because chunks are what get retrieved - and how you chunk a long document is its own decision, see context windows and why long documents still fail.
Model permissions as a filterable attribute. In practice this is usually a list of group identifiers on each chunk, and the query filters to chunks whose list intersects the user’s groups. Most vector stores support metadata filtering as part of the search; use it rather than filtering afterwards.
Resolve the user’s groups at query time, from your identity provider, not from a cached copy attached to their session. Group membership changes and a cached set is a stale set.
Fail closed. If permissions cannot be resolved - identity service down, unexpected document state - return nothing. Never fall back to unfiltered retrieval on error. This is the single most important line of code in the design.
The four paths that survive a correct filter
1. Stale permissions in the index
Access changed in the source system; your index still carries the old rules. Someone loses access to a folder on Monday and the assistant keeps serving its contents until the next reindex.
The gap equals your reindex interval. Nightly means up to a day of over-permissioned answers.
Two mitigations. Sync permission changes more frequently than content changes - they are cheaper to update, and many source systems emit change events. And for high-sensitivity sources, verify permission at answer time against the live system for the specific documents about to be cited. That is a handful of checks per answer, not a full re-evaluation, and it closes the window.
2. The index itself
Your vector store is a copy of your documents, flattened, in a system with its own access model and often weaker controls than the source.
Anyone with database access can read every chunk regardless of the permissions recorded on it. So: restrict access to the store as you would the source documents, encrypt at rest, be deliberate about who holds credentials, and log reads.
Worth stating plainly to whoever owns information security, because it is frequently missed: building a knowledge assistant creates a second copy of your sensitive documents in a new place.
3. Conversation history
A user asks about a document they can access. Access is revoked. The prior conversation still contains the content, and a follow-up question may draw on it.
Decide the policy explicitly: how long history persists, whether prior turns are re-validated, and whether revocation purges history. There is no universally right answer, and the wrong outcome is not having decided.
4. Logs and traces
Observability captures prompts and responses, which means retrieved content lands in your logging platform - typically with much broader access than the documents themselves.
Redact retrieved content from traces, or restrict log access to match the most sensitive source indexed. Engineers debugging a pipeline should not thereby have read access to HR files.
Testing it
Permission bugs do not surface in ordinary use, because most users ask about things they can see. Test deliberately.
Build a test set specifically for access control: users at different permission levels, and questions whose answers exist only in documents they cannot see. The correct behaviour is a refusal or an answer drawn only from permitted sources - not a partial answer that reveals the restricted content exists in an interesting way.
Run it on every change to retrieval, indexing or permissions. This belongs in the same harness as your quality evaluation, scored separately, because a permission regression is a different class of failure from a quality regression. It is also one of a small set of properties that are cheap in week one and projects later - see responsible AI as an engineering decision.
What we treat as non-negotiable
Query-time enforcement, permissions on chunks, fail-closed on resolution errors, and an access-control test set that runs in CI. These are in the first version, not a hardening phase.
The reasoning is that this failure is categorically different from the others we write about. A wrong answer is embarrassing and fixable. An assistant that shows an employee their colleague’s salary is an incident with legal and HR consequences, and it cannot be un-shown. Systems where the worst case is unrecoverable get their controls built first.
The design decision we argue for most often: enforce at query time even when it costs retrieval quality. Filtering the candidate pool before ranking sometimes returns fewer good passages than filtering after, and the temptation to post-filter for better answers is real and gets raised in most projects. We treat it as settled rather than as a trade-off, because the alternative is a correctness property you cannot test your way out of.
What we cannot promise: that your source systems’ permissions are themselves correct. An assistant faithfully enforcing a permission model with a badly-configured shared folder in it will faithfully serve that folder to everyone. Projects like this routinely surface pre-existing access problems, and that discovery is genuinely useful and not something we can fix from the retrieval layer.
When this is simpler than it sounds
Single-permission corpora. If every employee may read everything indexed - published policies, a public help centre - there is no permission model to enforce. Say so explicitly and scope the index to exactly that content.
Small, stable teams. Ten people and three groups is not a hard model. The complexity is in large inherited hierarchies.
The route many teams should take: index only the universally-readable content first. It delivers most of the value, and it defers the hardest part until the assistant has proven itself.
Frequently asked questions
Can we filter results instead of the query?
No, for the reason the diagram shows: the model has already read the content. Post-filtering corrects the citation list and not the answer text.
How do we handle documents with per-user rather than per-group access?
The same mechanism with the user identifier in the filter. It is less efficient at scale and works. The harder problem is keeping it synchronised.
What if our source system’s permissions are a mess?
Then the assistant will faithfully reproduce the mess. Fix the source, or scope the index to a subset you have verified. Do not attempt to correct it in the retrieval layer.
Does this apply if we use a hosted knowledge product?
Yes, and ask the vendor specifically whether filtering happens in the query or on results, and how quickly permission changes propagate. If they cannot answer precisely, assume the weaker implementation.
How often should we reindex permissions?
More often than content. Daily is a reasonable floor; event-driven updates from the source system are better; live verification of cited documents at answer time is the strongest option for sensitive corpora.
Next step
If you have an assistant over mixed-permission content and cannot say whether filtering happens in the query, that is worth establishing before it is asked a question by the wrong person. The knowledge assistant engagement enforces permissions at query time as the default.
Related: Document extraction: what to verify · Confidence thresholds and escalation design · Building an eval set from real tickets · Knowledge assistant