advanced26 min studyLesson 674 in Engineering

Retrieval access control: enforce permission before evidence reaches a consumer

Bind retrieval, expansion, caches, citations, and answer serving to authenticated identity and current source permissions.

On this page

What you’ll understand

  • Define an authorized candidate set under a versioned access policy.
  • Identify bypass paths through caches, parent expansion, and citation endpoints.
  • Test revocation and derived evidence without relying on generation instructions as authorization.

A generator cannot enforce a document permission by promising not to mention text it has already received. Access decisions belong in the authenticated data path before restricted evidence reaches an unauthorized model context, response, cache consumer, or citation download. Follow the principal and policy version through every retrieval branch, and make revocation behavior an explicit part of the application's consistency contract.

Authentication, candidate eligibility, and generation policy are distinct

Authenticate the requesting principal and obtain its current tenant, group, purpose, or other policy attributes from trusted application state. Use them to define eligible source records under the current access policy. Retrieval should operate within that eligible domain where the chosen index supports it, and fetched content must still be checked before serving or model use. Post-filtering a global top-k list can remove unauthorized hits but also leave too few useful authorized results; it is not automatically equivalent to ranking the eligible corpus.

A prompt saying use only public documents does not replace a data authorization check. Retrieved text is evidence input, not permission to expand access or execute its embedded instructions. PostgreSQL 18 row security and Elasticsearch 8.19 document-level security are examples of enforcement mechanisms with specific role and query semantics. PostgreSQL superusers, BYPASSRLS roles, and normally table owners have documented bypass behavior. Test the actual service identity and configuration rather than assuming an enabled feature constrains every code path.

Du,vA={d∈D:Allow⁡(u,d,vA)=1},Ru(q)⊆Du,vA\mathcal D_{u,v_A}=\{d\in\mathcal D:\operatorname{Allow}(u,d,v_A)=1\},\qquad R_u(q)\subseteq\mathcal D_{u,v_A}
u is the authenticated principal and v_A identifies the policy state used for the decision. The subset is a required application invariant for evidence eligible for this retrieval path, not a claim that every backend applies filters at the same stage or provides an instantaneous revocation guarantee.↔ Scroll wide equations horizontally.

Permission must survive expansion, derivation, and cache reuse

Check parent expansion, neighboring passages, hierarchical summaries, reranking services, fallback queries, exports, and citation-resolution endpoints. A permitted child does not authorize a restricted parent region. A summary derived from mixed-access sources can reveal restricted content even if its own record lacks the original labels. Define and enforce a derived-content policy using lineage, or regenerate a permitted derivation. Cache entries containing snippets, source identifiers, or generated answers need equivalent scope checks; a query string alone is not a safe cross-user key.

Revocation can race retrieval or cache lookup. Choose a policy for in-flight requests and a point at which a current permission decision must be established before delivery, using an implementation that supports that consistency goal. Merely writing an ACL version into a log does not make the decision atomic. On permission uncertainty, refresh or deny the affected artifact according to policy. Deleting an index row does not necessarily remove material from parent stores, answer caches, derived summaries, or a previously generated response buffer.

ServeEvidence⁡(u,x)⟹Allow⁡(u,x,vArequired)=1\operatorname{ServeEvidence}(u,x)\Longrightarrow\operatorname{Allow}(u,x,v_A^{\rm required})=1
This desired delivery invariant applies to source-derived artifacts x, including snippets and resolved citations, at the application's stated authorization decision point. Implementing it requires authoritative policy checks and a defined in-flight consistency model; model instructions, TTLs, and stale metadata alone do not prove it.↔ Scroll wide equations horizontally.

Test identities and negative paths, not only successful queries

Build cases for two tenants with identical query text, a user losing a group, a source changing permissions without changing content, and a cache populated by a more privileged principal. Verify every retrieval variant and citation endpoint. Test broad database roles and role combinations using their actual documented semantics. Observe content delivered to rerankers and generation providers as well as the final user response. A denied visible answer does not repair a restricted passage already sent to an unauthorized downstream consumer.

The probability lab provides only a negative-test counting analogy. Reset, set p=0 and n=20, predict zero successes with probability one, then Play or Step. Separately Reset, set p=0.05 and n=20, predict mean one success and probability of zero successes about 0.3585, then Play or Step. It does not implement permissions or simulate attacks. Under an explicitly independent hypothetical defect model, zero observed failures does not prove the defect probability is zero; access invariants still need deterministic path coverage and actual policy tests.

A cached identity list still needs current authorization

A retrieval cache contains source identities [A,B,C,D,E] from policy version seven. At required policy version eight, the current principal may access only [A,C,E,F]. No cached source text has yet been delivered.

  1. Intersect cached identities with the current authorized set to obtain [A,C,E]. Reject B and D before their text reaches the model, reranker, response, or citation consumer.

  2. The usable list now has three candidates rather than five. If the request requires more, retrieve additional eligible evidence under version eight; do not restore the rejected entries to fill a quota.

  3. A cached generated answer that depended on B or D cannot simply be relabeled version eight. Revalidate its complete provenance and policy or recompute from permitted evidence under the stated serving contract.

The result

Three cached identities remain eligible. Rechecking the answer's source dependencies is distinct from rechecking only the retrieval list.

Explore the model

Interactive lab

Interactive lab · probability

No observed failure is not a permission proof

Reset, set p=0 and n=20, predict K=0 with probability 1, then Play or Step. Separately Reset, set p=0.05 and n=20, predict mean 1 and P(K=0)=0.95²⁰≈0.3585, then Play or Step. The exact binomial assumes independent trials and has no ACL or attack model.

No observed failure is not a permission proofThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forUse actual deterministic authorization tests rather than inferring safety from a small clean sample.

Check your understanding

Why is filtering restricted citations out of the final answer insufficient access enforcement?

Try it yourself

A cache entry depends on sources [P,Q]. The user retains permission for P but loses permission for Q. The entry's TTL has ten minutes remaining. Can the remaining TTL establish that the answer may still be served?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Freshness time and permission validity are different conditions.

Compare with the solution

No. The TTL does not establish current authorization for Q or for an answer derived from it. Apply the stated authoritative permission and provenance checks, then deny, revalidate, or regenerate as the serving policy requires. A fresh timestamp cannot replace a current access decision.

Take it with you

  • Authorize source-derived artifacts before they reach each unauthorized consumer.
  • Expansion, derivation, caches, and citations are separate enforcement paths.
  • Revocation needs a concrete consistency contract and identity-based tests.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.