What is semantic search?
A support assistant built on semantic search alone can answer a policy question with total confidence, then get the policy number wrong.
Semantic search
Vector search
Semantic search ranks results by meaning rather than by matching words. It turns text into embeddings, numbers that stand in for meaning, then returns the documents whose embeddings sit closest to the query's.
Traditional search matches strings. Type "cancel subscription" and it looks for those words in the index. If the document says "end my plan" instead, a keyword search can miss it, even though the meaning is the same.
Semantic search skips the word match. An embedding model turns each piece of text into a list of numbers. Similar meanings land near each other in that space. "Cancel subscription" and "end my plan" sit close together, so the search finds both.
Documents get embedded once, ahead of time, and stored in a vector database. A query gets embedded the moment someone asks it. The search ranks stored vectors by how close they sit to the query vector, usually with a distance measure called cosine similarity.
It reads meaning well and identifiers poorly
Semantic search is the reason a retrieval-augmented generation system can answer a paraphrased question correctly. Ask about "getting a refund" and it finds the passage titled "return policy", with no shared words between the two.
The same property works against you when the query contains an identifier. A policy number, an order ID or a part code carries almost none of its meaning in the embedding. Change one digit and the vector barely moves, so the system can retrieve a neighboring record instead of the exact one.
We treat that as a design constraint, not a bug to tune away. When source documents mix free text with codes, contracts, part numbers, account references, we add a keyword search alongside the vector one from the start. Waiting until a wrong answer ships costs more.
- 01Finds paraphrases and synonyms that share no words with the query.
- 02Loses precision on exact identifiers, codes and short names.
- 03Ranking depends on the embedding model, so a model change can shift results.
- DocumentsEmbedded once, ahead of time.
- Vector storeEmbeddings held for lookup.
- QueryEmbedded at search time.
- CompareCloseness in vector space.
- Ranked resultsNearest meanings first.
The query never touches the document text directly. It only ever compares vectors.
Common questions
01Is semantic search the same as vector search?
Yes, the two terms describe the same technique: ranking results by embedding similarity rather than by word match. "Vector search" names the mechanism, and "semantic search" names what it achieves.
02Does semantic search replace keyword search?
Not in systems that need exact matches. Semantic search finds paraphrases and related meaning well, but it loses precision on identifiers and short codes. Most production retrieval systems run both and merge the results, an approach called hybrid search.
03What database do I need for semantic search?
You need a vector database, or a general database with a vector extension, to store embeddings and search them by similarity. Postgres with the pgvector extension is a common choice; dedicated vector databases are another.
04Why does semantic search sometimes return the wrong document with high confidence?
The similarity score reflects distance in embedding space, not correctness. A near-miss on an identifier can score almost as high as the exact match. The system has no separate check that the identifier is actually right.
Related
- embedding →The numeric representation semantic search compares.
- vector database →Where those embeddings get stored and searched.
- hybrid search →Pairing semantic search with keyword matching to keep exact identifiers accurate.
- RAG development →Where semantic search fits inside a retrieval pipeline we build and run.
