Semantic search that worked last month now returns irrelevant chunks for every
query. No exception, no warning — the scores print in a believable range
(0.3 to 0.6) and the code runs green. In the louder variant, the search crashes
with a shape error such as ValueError: shapes (5000,768) and (384,) not aligned
the moment the query path calls a different model than the one that built the index.
A fresher builds a document search or RAG pipeline. The chunk index was embedded
weeks ago with one model (say nomic-embed-text through Ollama). Later the query
code is "improved" — a new model name in one config line, a switch from Ollama to
sentence-transformers, or a copied snippet that quietly uses a different default.
Nobody re-embeds the corpus, because nothing looks broken. Linked module: 3.4
(Embeddings & vector search).
Every embedding model defines its own coordinate system. Dimension 217 in model A has no relationship to dimension 217 in model B, so a cosine between a model-A query vector and a model-B chunk vector is noise, not a similarity. When the two models share a dimension count, the noise arrives silently; when they differ, you get the shape error instead — the lucky outcome, because it cannot be ignored. Confirm the mismatch in two steps. First, compare dimension counts: print the index vector length and the query vector length. Second, run the self-match test: embed one sentence that already exists in the index through the query path and search for it. Healthy pipelines return that exact chunk at a score close to 1.0; a mixed-model pipeline returns strangers.
Pick one model and make the whole pipeline agree. Re-embed the entire corpus with
the model the query path uses — there is no conversion shortcut between spaces,
and padding or slicing vectors to force matching shapes only converts the crash
back into silent garbage. Store the model name and version next to the index (a
meta.json beside the vectors works), and add a startup assertion that compares
the stored name against the query-path model, failing loudly on mismatch. Rerun
the self-match test before calling the incident closed.
Treat "change the embedding model" and "rebuild the index" as one action — never two tickets. Write the model name into index metadata on every indexing run and assert against it on every service start. Keep the self-match test in your smoke tests: one known sentence, searched for itself, must score near 1.0. Any score drift after a deploy means the two sides of the pipeline stopped agreeing.
Q: "After a teammate upgraded the embedding model, retrieval quality collapsed but no errors appeared. Walk me through your diagnosis." Strong answer: state that vectors from different models live in unrelated spaces, so query and index must share one model; name the silent case (same dimensions) versus the loud case (shape error); give the self-match test as proof; finish with the fix — re-embed everything, then pin the model name in index metadata with a startup assertion.