The RAG bot answers confidently and the citation checks out — against the wrong document. An HR question about maternity leave returns the Sales travel advance policy. A Hindi query about leave ("एक साल में कितनी छुट्टियाँ?") retrieves three English chunks from a different department's SOP. No error, no refusal, no crash: the retrieval log looks structurally healthy, and the model faithfully quotes whatever it was handed.
A single-document pipeline (module 3.5 style) that grew: other teams pushed their documents into the same collection, in more than one language, with no labels on any chunk. Common in week two of a successful pilot — the demo worked, so every department wanted in, and ingestion stayed one loop that embeds text and nothing else. Linked module: 3.6 (RAG that survives production).
Embeddings measure closeness of meaning, not jurisdiction. "Travel advance" from Sales and "travel reimbursement" from HR sit near each other in vector space, and near enough to a travel question that either can win the top-k. Across languages the effect worsens: a multilingual embedding model maps a Hindi leave question near English leave text from any document that discusses leave. The store cannot respect boundaries it has never been told about — if no chunk carries a department or language label, "search only HR" is not an expressible query. Confirm in one step: print the retrieval log with each hit's source document. If the top-k mixes documents that should never answer the same question, you have this failure. Note what is NOT broken: retrieval ranked honestly and the citation is real. The pipeline did exactly what it was asked; the ask was wrong.
Three moves, in order. (1) Design the metadata dict: at minimum doc,
department, language, section, version. (2) Re-ingest every document with
that metadata attached to every chunk — metadata you did not capture at ingestion
does not exist at query time, so this is a full re-index, not a patch. (3) Derive
a where filter for every query — from the asking user's department, an explicit
UI choice, or a detected language — and pass it to the store so similarity runs
only inside the allowed slice. Verify with a leakage test: a list of questions
that must never cross a boundary, each asserting the top-k contains only
same-department, same-language chunks.
Decide the metadata schema before indexing document number two — retrofitting labels onto an unlabelled corpus costs a full reprocess every time. Print metadata in every retrieval log line so a cross-department hit is visible the day it first happens, not the day a user screenshots it. Keep the leakage test in your regular eval run, and add a case for every new department or language that joins the collection.
Q: "Your document assistant serves HR, Sales, and IT from one vector store. How do you stop an HR question being answered from a Sales document?" Strong answer: name the mechanism (metadata stamped at ingestion, filters applied at query time), state the deeper rule (embeddings rank by meaning and cannot enforce boundaries on their own), admit the migration cost (unlabelled chunks force a re-index), and finish with the leakage test — a test that proves isolation continuously, not a one-time manual check.