What is Hybrid Search?
Hybrid search combines keyword search (such as BM25) with vector similarity search and fuses the two result lists, so a retrieval system finds both exact terms — IDs, error codes, product names — and semantically related passages that share no words with the query.
At a glance
- Runs a keyword (sparse/BM25) query and a vector (dense) query, then merges the results.
- Keyword search catches exact tokens: SKUs, error codes, names, acronyms, version numbers.
- Vector search catches meaning: paraphrases and synonyms that share no words with the query.
- Reciprocal Rank Fusion (RRF) is the usual, tuning-free way to merge the two lists.
- It is the single highest-leverage upgrade over pure vector retrieval in most RAG systems.
Why pure vector search isn't enough
Embedding-based retrieval is very good at meaning and very bad at literals. Ask for "error ORA-01555" or "policy SKU 4471-B" and a dense vector search will happily return passages about similar-sounding errors and adjacent products, because those strings carry almost no semantic signal. Classic keyword scoring — BM25 — nails exactly those cases.
The inverse is also true: keyword search misses "how do I stop my job from dying halfway through?" matching a document titled "Handling long-running query timeouts." Neither method dominates, so production systems run both.
How fusion works
Each retriever returns its own ranked list. The simplest robust merge is Reciprocal Rank Fusion: score each document by the sum of 1/(k + rank) across the lists it appears in (k is usually 60). RRF needs no score normalisation and no weight tuning, which is why most vector databases implement it natively. Weighted score fusion — normalising BM25 and cosine scores, then blending with an alpha — gives more control if you have evaluation data to tune against.
Hybrid search widens the candidate net; it does not put the best passage first. The standard production pipeline is therefore hybrid retrieval of 50–100 candidates, then a reranker to cut that down to the 3–8 chunks that actually go into the prompt, with metadata filters applied so results respect the user's permissions.
Frequently asked questions
What is the difference between hybrid search and semantic search?
Semantic (vector) search matches by meaning only. Hybrid search runs semantic search alongside a keyword method like BM25 and fuses the results, so exact identifiers and rare terms are retrieved as reliably as paraphrases.
What is Reciprocal Rank Fusion?
RRF merges two ranked lists by scoring each document as the sum of 1/(60 + its rank) in each list. It needs no score normalisation or weight tuning, which makes it the default fusion method in most hybrid search implementations.
Do I still need reranking if I use hybrid search?
Usually yes. Hybrid search improves recall — the right passage is somewhere in the candidate set — while a cross-encoder reranker improves precision by putting it in the top few slots that actually fit in the prompt.
Does hybrid search slow retrieval down?
Marginally. The two queries run in parallel and fusion is trivial arithmetic, so the added latency is typically tens of milliseconds — far less than the reranking or generation steps that follow.
Putting Hybrid Search to work?
We help data & AI teams design and ship this in production. Tell us what you're building and we'll point you at the shortest path.
Talk to our team