What is embedding drift in AI search?
Search results get worse for months after a routine model upgrade, and nothing in the logs says why.
Embedding drift
embeddings drift
Embedding drift is the mismatch that appears when the embedding model behind a search system changes. Vectors from the old model no longer sit in the same space as vectors from the new one, so a similarity score between them stops meaning what it used to.
Every embedding model maps text into its own space of coordinates. Two documents land close together if the model judges them similar in meaning. That space is specific to the model that built it, not a fixed map that every model shares.
Swap the model and you get a new space with different coordinates. Old vectors, computed under the retired model, still sit in a vector database and still return a similarity score against a new query. That score is now comparing two different coordinate systems as if they were one, and nothing in the pipeline flags this as an error.
The failure has no error message
A broken API call fails loudly and gets fixed the same day. Embedding drift does the opposite. Queries still run, results still come back, and every response looks like a normal answer.
The real effect is quieter. Relevant documents rank lower. Retrieval starts pulling in passages that are near the query in the old space but not in the new one. A RAG system built on this index answers questions using the wrong context, and the language model writes a fluent answer from bad source material. Nobody sees a stack trace, so the drop gets blamed on the model, the prompt, or the users typing worse questions.
It shows up any time the embedding model changes underneath a stored index. A vendor deprecates a model version. A team upgrades to a newer model for better benchmarks. A self-hosted model gets fine-tuned and redeployed. All three leave the old vectors stranded.
- 01Mixing vectors from two model versions in one index is the most common cause, not a full model swap.
- 02A newer, better-scoring embedding model can still make your specific retrieval worse until the whole index catches up.
- 03The symptom reads as a prompting problem, so teams tune prompts for weeks before anyone checks the embeddings.
- StoreDocuments embedded, model version A.
- UpgradeProvider ships model version B.
- MixNew writes use B, old rows stay A.
- QueryOne search space, two coordinate systems.
- DecayRelevance drops with no error logged.
The break happens at the mix step, days or months before anyone notices the decay.
Common questions
01Do I need to re-embed my documents after a model upgrade?
Yes, if the model actually changed its output space, which most version upgrades do. Check the vendor's release notes for a compatibility statement with old embeddings. If it is not stated, assume it is not, and re-embed the full corpus rather than mixing old and new rows.
02Can I mix embeddings from two different models in one index?
No. Similarity search assumes every vector came from the same model. Comparing across two models returns a score that does not mean what your ranking logic thinks it means. Keep one model version per index, or run two indexes and merge results explicitly.
03How do I know embedding drift is causing my retrieval problem?
Check whether relevance dropped sharply after a model or provider change, rather than fading over time. Can you date the drop to a deployment where the index was never fully re-embedded? Test that first, before touching prompts or chunk sizes.
04Does a smaller, cheaper embedding model still cause drift risk?
Yes. Drift is about a change in the coordinate space, not about model size or cost. Switching to a cheaper model is still a model swap, and the same re-embed rule applies regardless of which direction the change goes.
