The "I don't know - this is not in the document" line has quietly disappeared from your bot's vocabulary. Out-of-scope questions — the office wifi password, a policy that never existed, even gibberish — now get fluent answers with citations. The retrieval log shows nothing alarming at a glance: every query returns k chunks with scores that look like the scores you have always seen.
A pipeline whose score floor was tuned when the collection was small (module 3.5's sweep, done at fourteen chunks) and never revisited while the collection grew to hundreds or thousands of chunks across many documents. Surfaces weeks after the growth, usually when someone asks a question from a document that was never ingested and gets a confident wrong answer instead of a refusal. Linked module: 3.6 (RAG that survives production).
Two facts combine. First, top-k retrieval returns k results by construction — it ranks whatever exists and never says "no" on its own; the floor was your only refusal mechanism. Second, scores compress as a collection grows: with 4,000 chunks covering many topics, some chunk is always moderately close to any question, so best scores crowd into a narrow band (say 0.38-0.55) instead of the wide spread you tuned against. A floor of 0.45 chosen at 14 chunks now sits inside the noise band: too low to block off-topic hits, while genuinely relevant hits barely clear it. Confirm by replaying your old out-of-scope eval questions and printing top scores — if refusals that once fired now pass the floor with mid-band scores, you have this failure. The floor did not break; the corpus it was calibrated against no longer exists.
Rebuild the honesty gate at the current scale. (1) Assemble an eval set that matches today's corpus: for each major document, several answerable questions plus genuine out-of-scope ones. (2) Re-run the floor sweep from 3.5 exercise 4 against this set and pick the new operating point; expect a different number. (3) If answerable and out-of-scope scores overlap so much that no single floor separates them, add a reranking pass: retrieve a wider shortlist cheaply, then re-score it with a stronger relevance check and refuse when even the best reranked candidate is weak. (4) Record the floor next to the collection it was tuned for, with the date and chunk count.
Treat the floor as configuration bound to a corpus version, not a constant — every large ingest event triggers a re-tune. Keep the out-of-scope questions in a weekly automated run; the day refusals stop firing should be the day you find out. Watch score-spread statistics over time: a shrinking gap between the best answerable score and the best out-of-scope score is this failure announcing itself early.
Q: "Top-k retrieval always returns k results. How does your RAG system ever say 'I don't know', and how does that mechanism survive corpus growth?" Strong answer: explain the score floor and why it is corpus-relative, describe score compression as collections grow, give the re-tuning discipline (eval set, re-sweep per ingest wave), and mention reranking as the escalation when raw similarity scores can no longer separate relevant from plausible.