Comparison
Semantic search vs keyword search
Embeddings find meaning and miss exact terms; keyword search does the reverse. Why hybrid search beats either, and what each one fails at.
The short answer
Use both. Semantic search retrieves by meaning and reliably misses exact identifiers — product codes, error strings, proper nouns. Keyword search does the reverse. Hybrid search merges the two result sets and is usually the largest retrieval quality improvement available after fixing chunking, at no inference cost.
At a glance
| Semantic search | Keyword search | |
|---|---|---|
| Finds paraphrase | Yes | No |
| Finds exact identifiers | Unreliably | Yes |
| Setup cost | Embedding the corpus | An index |
| Per-query cost | An embedding call | Nothing |
| Storage | Vectors — kilobytes per chunk | An inverted index — much smaller |
| Handles typos | Reasonably | Poorly without fuzzy matching |
| Explainability | Low — a similarity score | High — which terms matched |
When to choose which
Choose Semantic search when
- Users ask questions in their own wordsSomeone asking about "getting money back" should find the refunds policy. Keyword search returns nothing, because there is no shared vocabulary.
- The corpus uses different language from the queriesTechnical documentation queried by non-technical users is the classic case. The gap between how a thing is written and how it is asked about is exactly what embeddings bridge.
Choose Keyword search when
- Exact strings matterError codes, SKUs, version numbers, function names. Embeddings treat these as near-meaningless tokens and rank them poorly.
- You need to explain the resultKeyword matching tells you which terms matched. A cosine similarity of 0.71 tells you nothing you can show a user or debug against.
- Cost and latency are tightKeyword search needs no embedding call at query time and no vector storage. On very high query volume that difference is real.
What it costs either way
Keyword search is close to free at query time. Semantic search costs an embedding call per query, which is small individually and adds up at volume — at high traffic it becomes the dominant line of the retrieval bill.
Storage differs by an order of magnitude. An inverted index is small; vectors at 1,536 dimensions in float32 are about six kilobytes per chunk before metadata and index overhead.
Both are trivial next to the generation call that consumes the retrieved chunks, which is why retrieval quality matters more than retrieval cost.
The mistake people make
Assuming embeddings replaced keyword search
Semantic search arrived with enough capability that many teams dropped keyword matching entirely, then spent months confused about why searching for an exact error code returned nothing useful. The two methods fail in opposite directions, which is precisely why combining them works so well. Hybrid search is not a compromise; it is the correct answer.
How to decide
- 1Start with hybrid if your stack supports it. It is rarely the wrong choice.
- 2If you must pick one, look at your queries. Mostly natural language questions means semantic; mostly identifiers and exact terms means keyword.
- 3Add a reranker over the merged candidate set — that is where most of the remaining quality is.
- 4Measure with your own failing queries rather than a benchmark. Retrieval quality is highly corpus-specific.
Price it yourself
Frequently asked questions
- How do I combine the two result sets?
- Reciprocal rank fusion is the common approach and needs no tuning: score each document by the reciprocal of its rank in each list and sum. Weighted score merging works too but requires calibrating scales that are not comparable by default.
- Does hybrid search cost more?
- Marginally — you run two retrievals instead of one, and both are cheap relative to the generation call. The quality gain almost always exceeds the cost.