A filter such as if score >= 0.7: accept behaves absurdly in one of two
directions. Either almost nothing passes — genuinely relevant chunks score 0.55
and get discarded, so the bot answers "not found" to questions the corpus covers.
Or almost everything passes — unrelated chunks score 0.72 and flood the prompt
with noise. The behaviour often flips overnight after an embedding-model change,
with no code diff anywhere near the threshold line.
A fresher copies a relevance cutoff from a tutorial, blog post, or an older internal repo. The number came from someone else's model and someone else's documents. Some embedding models spread scores widely (relevant 0.8, unrelated 0.2); others compress everything into a narrow band where unrelated text still scores 0.6. The copied constant meets the new distribution, and the filter becomes either a wall or a hole. Linked module: 3.4 (Embeddings & vector search).
Cosine similarity is model-relative. A score is a position inside one model's score distribution, not a universal grade — 0.7 can mean near-duplicate under one model and background noise under another. Measure instead of guessing: take ten query-chunk pairs you know are relevant and ten you know are not, score all twenty with the production model, and print both lists sorted. A healthy setup shows a visible gap between the two groups; your threshold is meaningful only if it sits inside that gap. If the two groups overlap heavily, no threshold exists and the retrieval design itself needs work. Checking the model's version history against the date the symptom started usually explains the "it broke overnight" report.
Calibrate on your own data. Place the cutoff inside the measured gap between your known-relevant and known-irrelevant scores — not at any number imported from elsewhere. Where the gap is thin, stop filtering by absolute score: take the top-k chunks by rank, optionally dropping results that fall far below the best hit (a relative margin), which survives distribution shifts far better than a fixed constant. Re-run the twenty-pair check and keep the sorted output in the pull request as evidence.
Keep the labelled pairs in the repo and re-run the calibration after any change of embedding model, chunking, or corpus — the threshold is part of the model configuration, never a universal constant. Prefer rank-based selection over absolute cutoffs in new designs. In reviews, treat any hard-coded similarity number as a question: "calibrated against what, and when?"
Q: "Is a cosine similarity of 0.8 a strong match?" Strong answer: refuse the number at face value — scores are model-relative, so 0.8 means nothing without that model's score distribution; describe the twenty-pair calibration, the gap test, and the top-k-with-margin alternative; mention that thresholds must be re-calibrated whenever the model or corpus changes, which is what breaks copied tutorials in production.