Propagate Access Policies into Vector Indexes and Test for Leakage

Tier: Advanced · Workflow area: Docs, lineage & governance

Securing RAG against leakage is named in security postings for a reason — and the data side of it is a data engineering job. This playbook designs metadata so every vector-store entry carries ACL tags, filters retrieval by the caller's entitlements, and — the part most teams skip — adversarially tests that the filter actually holds.

When to reach for this

• •
  Any RAG index built over documents or tables that have differing access levels (HR docs, customer contracts, regional data subject to residency rules).
• •
  Multi-team or multi-tenant assistants where "who asked" must change "what can be retrieved".
• •
  A security review of an existing RAG pipeline that was built permissions-blind (most first versions are).

If the corpus is genuinely uniform-access (public docs, one team's own runbooks), skip the machinery — but write down that decision, because corpora grow.

The workflow

1. 1.

   Make the source of truth explicit. Decide where entitlements live — warehouse grants, an IdP group mapping, a document-system ACL — and treat the vector store as a downstream consumer of it. The index never gets its own hand-maintained permission model; it inherits one.

2. 2.

   Design ACL metadata at chunk level. Every chunk record carries the tags needed to answer "may this caller see this?" at query time: e.g. {"source_doc": ..., "acl_groups": ["finance-fp&a"], "sensitivity": "internal", "tenant_id": ...}. Derive tags in the ingestion pipeline from the source ACL — never let humans (or an LLM) assign them ad hoc. A chunk with no resolvable ACL gets quarantined, not indexed with a default-open tag.

3. 3.

   Enforce with the store's native mechanism. Filtering belongs in the database, not in application code that remembers to add a WHERE clause:

   • •

     pgvector + row-level security — the strongest teaching pattern, because the database enforces it even if the app forgets:

     Copy

     ALTER TABLE rag_chunks ENABLE ROW LEVEL SECURITY;
     CREATE POLICY chunk_acl ON rag_chunks
       USING (acl_group = ANY (string_to_array(current_setting('app.caller_groups'), ',')));
     

     The API layer sets app.caller_groups from the verified identity per connection; similarity search then physically cannot return out-of-scope rows.

   • •

     Qdrant / Weaviate / Milvus — mandatory payload/metadata filter injected server-side or by a thin gateway the caller cannot bypass.

   • •

     Pinecone — namespaces for hard tenant isolation plus metadata filters for group-level ACLs within a namespace.

   • •

     OpenSearch — document-level security (DLS) mapped from the caller's roles.

4. 4.

   Propagate changes, especially revocations. Access is dynamic: a user leaves finance, a document is reclassified. Wire your CDC/refresh pipeline (the same one that handles content updates) to re-derive ACL tags on change, and measure revocation lag — the time between a source-system revoke and the index no longer serving that chunk to that caller. Agree an SLA for it; hours is usually acceptable, days is a finding.

5. 5.

   Adversarially test for leakage. This is the step that separates "designed" from "secured". Build a leakage test suite that runs in CI and after every index rebuild: for each persona, a set of queries specifically crafted to pull content they must not see (ask about the restricted content by topic, by paraphrase, by asking the assistant to summarize "everything about X"). Assert zero restricted chunks in the retrieval results — check the retrieval layer's returned chunk IDs, not just the generated answer, because generation can leak from a retrieved-but-unquoted chunk. An LLM is useful here as the red-team query generator: give it the restricted docs' titles/topics and have it produce paraphrase attacks.

6. 6.

   Log retrieval with identity. Every query logs caller identity, filter applied, and chunk IDs returned. Without this you cannot audit an incident, and with it you can continuously mine for near-misses.

Verify before you trust it

• •
  RLS/filter enforcement is server-side: a raw similarity query with a persona's credentials but no application filter still returns only permitted chunks.
• •
  The leakage suite (≥20 adversarial queries per persona) passes with zero restricted chunk IDs returned, and runs in CI on every ingestion-pipeline change.
• •
  Revocation lag measured end-to-end (revoke in source → chunk unretrievable) meets the agreed SLA.
• •
  Quarantine works: a document with unresolvable ACL demonstrably does not appear in the index.
• •
  Deletion propagates: a deleted source document's chunks are gone from the store (check by ID), not just unreferenced.

Pitfalls

• App-layer-only filtering. A forgotten filter parameter in one code path is a silent full-corpus exposure. Prefer RLS/DLS or a mandatory gateway; treat client-supplied filters as untrusted.
• Embedding leakage via shared indexes. Even with filtered retrieval, aggregate artifacts (cluster summaries, "similar documents" features, cached results) can leak across boundaries. Audit every feature that touches the index, not just the main query path.
• Default-allow on missing tags. Pipelines that index untagged chunks "temporarily" create permanent leaks. Quarantine is the only safe default.
• Stale group membership. Caching the caller's groups for a day quietly extends access a day past revocation. Bound the cache by your revocation SLA.
• Testing with friendly queries. Retrieval tests that only ask polite, on-topic questions miss paraphrase and summarization attacks. Generate adversarial queries deliberately.

Tools

• pgvector + Postgres RLSfree, ubiquitous, and the only option here where the database itself enforces the policy; the default teaching and small-scale production choice.
• Qdrant / Pineconepayload filters and namespaces for dedicated-store deployments; namespaces give hard multi-tenant walls (pairs with multi-tenant RAG design).
• OpenSearch (document-level security)when the RAG stack is search-engine based and roles already live in the security plugin.

Client-safe variant

In client engagements this workflow usually is the deliverable of a RAG security review. Run everything inside the client's environment — the restricted corpus and the leakage-test transcripts are both sensitive. If an external LLM is used only to generate red-team queries from document titles/topics, confirm that even titles are shareable; otherwise generate attacks with a self-hosted model or by hand. Deliver the leakage-test suite itself (runnable, in their CI) plus the revocation-lag measurement — a client who can re-run the proof after every index rebuild has received security, not a report about security.