Beyond Flat Vector Search: Metadata Architecture, Parent-Document Retrieval, and GraphRAG

Tier: Advanced · Workflow area: RAG & Retrieval Engineering

This playbook adds the three structural upgrades — metadata-filtered retrieval, parent-document retrieval, and graph-augmented retrieval — each gated by the same eval discipline as pb-22–24.

When to use

• •
  Enterprise corpora with mixed sources, versions, and audiences where "top-k over everything" returns stale or out-of-scope chunks.
• •
  Per-user document permissions (ACLs) must hold — a RAG answer must never leak a document its asker cannot open.
• •
  Multi-hop questions ("which suppliers of our top-3 customers had audit findings?") that pb-24's stack still fails because no single chunk contains the join.

Prerequisites

• •
  The pb-24 stack and harness; a payload-filtering vector store (Qdrant/Weaviate/pgvector with WHERE clauses).
• •
  Document metadata: source system, doc type, effective/updated dates, version, access groups. If this doesn't exist, extracting it (pb-14/18) is step zero.
• •
  For the GraphRAG arm: networkx (or Neo4j), and an entity/relation extraction schema.

The workflow

1. 1.

   Design the payload schema like a database, not an afterthought. Every chunk carries: doc_id, parent_section_id, source, doc_type, effective_date, version, acl_groups: list[str], plus domain facets (product, region). Index the filterable fields (Qdrant payload indexes / pgvector b-tree columns) — unindexed filters silently become post-filters that wreck recall at scale.

2. 2.

   Enforce scope as pre-filter, ACLs server-side. Filters must constrain the ANN search itself, and permissions must be non-bypassable by prompt content:

   Copy

   hits = qdrant.query_points("corpus", query=qvec, limit=50,
       query_filter=Filter(must=[
           FieldCondition(key="acl_groups", match=MatchAny(any=user.groups)),
           FieldCondition(key="version", match=MatchValue(value="current")),
       ]))
   

   The acl_groups condition is injected by the serving layer from the authenticated session — never model-generated. Model-chosen filters (self-query: "in 2024 reports" → effective_date range) are fine for scoping, never for authorization.

3. 3.

   Retrieve small, read big — parent-document retrieval. Embed fine-grained chunks (they match queries better) but hand the generator the parent section:

   Copy

   matched = search(qvec, k=30)                       # child chunks
   parents = dedup([c.payload["parent_section_id"] for c in matched])[:5]
   context = [section_store[p] for p in parents]      # full sections, with headings
   

   Cap parent size (~1,500 tokens; split giant sections at ingestion). Measure against pb-24: context_precision often dips slightly while faithfulness and answer completeness rise — decide on the answer metrics.

4. 4.

   Build the graph arm for multi-hop. Extract entities/relations at ingestion with a structured call per chunk ((entity_a, relation, entity_b) triples with types from a closed vocabulary + source chunk ID), and load into a graph:

   Copy

   class Triple(BaseModel):
       head: str; head_type: Literal["org","person","product","event"]
       relation: Literal["supplies","owns","audited_by","partnered_with", "competes_with"]
       tail: str; tail_type: Literal["org","person","product","event"]
   G.add_edge(normalize(t.head), normalize(t.tail), rel=t.relation, src=chunk_id)
   

   Entity normalization (case, aliases, the pb-17 dedup trick on entity-name embeddings) is where graph quality is won or lost. Query time: extract entities from the question → k-hop neighborhood → pull the source chunks for those edges → merge with vector results before reranking.

5. 5.

   Route, don't default, to the graph. Graph retrieval is expensive to build and maintain; send it only queries classified as multi-hop/relational (hops_required field in the pb-24 query classifier). Single-hop questions stay on the vector path.

6. 6.

   Evaluate on a multi-hop slice. Extend the grounding set with 15–20 genuine multi-hop questions (evidence spread across ≥2 documents). Report the ablation: pb-24 stack vs +parents vs +graph, on single-hop and multi-hop slices separately — the graph should transform the multi-hop slice and leave single-hop untouched.

Tool variants

• Qdrant/Weaviate: rich payload filtering with filter-aware HNSW; pgvector: metadata as plain SQL columns — familiar and auditable, mind combined index performance; OpenSearch: filters + hybrid in one engine.
• LlamaIndex:AutoRetriever/self-query for model-chosen filters, HierarchicalNodeParser + AutoMergingRetriever for parent-document, Property Graph Index for the graph arm.
• Neo4j vs NetworkX: NetworkX in-process is enough to prove value up to ~100k edges; move to Neo4j when you need Cypher, persistence, and concurrent access. Microsoft GraphRAG adds community detection + summarization for corpus-level "themes" questions — heavyweight; adopt only if that query class matters.
• ACL sources: mirror groups from your IdP (Okta/AD) into payloads at ingestion; re-sync on a schedule — stale ACLs fail closed if you intersect, fail open if you union. Intersect.

Verification

• •
  Adversarial ACL suite: ≥20 attempts (including prompt-injection asking the model to ignore permissions) retrieve zero unauthorized chunks — verified from retrieval logs, not from answers.
• •
  Filtered recall: recall@5 within the correct scope beats unfiltered retrieval on scoped questions (stale/wrong-version answers eliminated, and measured).
• •
  Parent-document ablation decided on faithfulness + completeness with context-token cost reported.
• •
  Multi-hop slice: graph arm lifts answer accuracy on ≥15 multi-hop questions vs pb-24 baseline; entity-normalization spot check on 30 nodes shows <10% duplicate entities.

Pitfalls

• Post-filtering. Filtering after ANN retrieval returns k-minus-filtered results — often near-empty for narrow scopes; verify your store applies filters during search.
• Letting the model set security filters. Self-query is for topical scoping; one jailbreak away from leakage if it controls acl_groups.
• Unbounded parents. "Return the whole document" as parent context reintroduces the pb-22 context cliff; cap and split at ingestion.
• Open-vocabulary relation extraction. Free-form relations ("is associated with") build an unqueryable graph; keep the relation set closed and small (≤15 types) and expand deliberately.
• Graph staleness. The graph is a derived index — wire it into the same incremental-ingestion path as vectors (pb-26) or it drifts within weeks.

Restricted-environment variant

• •
  Everything here runs on-prem: Qdrant-OSS/pgvector, NetworkX/Neo4j Community, local extraction models for triples. Client ACL data is itself sensitive — it stays in the engagement environment.
• •
  Entity/relation extraction on client text via hosted APIs only with named-entity redaction reviewed by the client — usually simpler to run local extraction and accept the quality gap.
• •
  Free-compute path: NetworkX + local 7–8B extraction on a T4 handles a 10k-chunk corpus graph within Kaggle free hours.

Success criteria

• •
  Payload schema and pre-filtered retrieval live, with ACL enforcement server-side and an adversarial suite in CI.
• •
  Parent-document decision made on measured answer metrics, not intuition.
• •
  A working graph arm that demonstrably wins the multi-hop slice, gated by a query router.
• •
  One ablation table covering flat vs filtered vs parents vs graph — the document that justifies (or kills) each layer of complexity.