Users ask questions you can answer yourself by opening the source document, yet the search returns unrelated chunks — or chunks that start and end mid-sentence. Two tells appear in the logs: top results whose scores sit in a tight mediocre band (0.42, 0.41, 0.40), and retrieved text that opens with fragments like "...as stated above, the fee is waived" with no clue what "above" said.
A fresher's first document search or RAG build over PDFs, policies, or wiki exports. Chunking was done the quick way: a fixed character count — often either one giant chunk per page (or per file) or tiny slices — with no overlap and no respect for headings, paragraphs, or tables. The demo worked because the demo question happened to land inside one chunk. Linked module: 3.4 (Embeddings & vector search).
Two failure shapes, one root cause: chunk boundaries ignore meaning. Shape one — chunks too big. A 4,000-character chunk mixes several topics, so its embedding is an average of all of them; every query scores it as "vaguely related" and nothing scores sharply, which is exactly what a flat 0.4-ish score band is telling you. Shape two — answers straddling a boundary. Fixed-size cuts split a rule from its condition or a table from its header, so the subject lands in chunk 7 and the predicate in chunk 8, and neither half embeds close to the question. Confirm by printing the top chunks for three failing queries and reading them aloud: if a chunk cannot be understood on its own, neither can its embedding. A second probe: re-ask a failing question using the document's exact wording — if it still misses, chunking (not the embedding model) is the prime suspect.
Re-chunk along the document's own structure — headings, paragraphs, list items — targeting a few hundred words per chunk, and add 10-20% overlap between neighbours so boundary-straddling answers survive the cut. Keep tables with their headers. Re-embed the corpus (chunking changes mean re-indexing, always) and re-run the same failing questions. Scores for correct chunks should rise clearly above the pack; the mid-sentence fragments should disappear.
After every indexing run, eyeball ten random chunks — each one should read as a sensible standalone passage. Keep a small fixed set of question-answer pairs and run it after any chunking or model change, exactly like the eval sets from module 3.3. Prefer structure-aware splitting from day one; fixed character counts are a prototype shortcut, not a production setting.
Q: "Your RAG bot keeps missing answers that are definitely in the corpus. Where do you look first?" Strong answer: rank chunking above the model — describe the muddy-average signature (flat mediocre scores) and the boundary-straddle signature (mid-sentence fragments), give the read-ten-chunks-aloud probe, and close with the fix: structure-aware chunks of a few hundred words with overlap, then re-embed and re-test against a fixed question set.