Why vector search misses exact terms, and what hybrid search fixed (measured)

You search your own docs for an error code and the AI hands you a different one. You ask for the v2.3 release notes and v3.2 shows up. The search is not broken in the usual sense: it is doing exactly what a meaning-based search does.

We fixed this in our in-browser knowledge bases with hybrid search: BM25 keywords plus vectors, fused with reciprocal-rank fusion. This post covers how it works, the numbers we measured, a database gotcha, and a second bug that made small collections return nothing.

This is part 2. Part 1, We rebuilt RAG in a browser tab, tells the whole rebuild, starting with the bug where every vector was empty.

Why vectors blur exact strings

To a sentence-embedding model, near-identical strings mean near-identical things. That’s exactly what you don’t want for error codes, SKUs, versions, names and IDs. Meaning is the right tool for “how do I get my money back?” and the wrong one for ERR_4521.

The setup, all on the user’s machine:

• Embeddings: all-MiniLM-L6-v2 (384-d), running in a Web Worker
• Store: SurrealDB 2.x compiled to WebAssembly, on IndexedDB (how that works: WebAssembly, explained)
• Indexes per chunk table: HNSW (cosine) for vectors, and a BM25 full-text index with a simple analyzer (lowercase, ascii, English snowball stemming)

How the hybrid query works

The query runs two searches and fuses their ranked lists:

query ─┬─ embed ───────────────► vector list (fetchK nearest)
       └─ keyword terms ───────► BM25 list   (fetchK best)
                                     │
                     reciprocal-rank fusion (k = 60)
                                     │
                           (optional rerank)
                                     │
                                   top k

Here’s that subtlety. Because the fused score is rank-based, you can’t put a “minimum relevance” cut-off on it.

So we apply the floor to each hit’s cosine similarity (computed from the stored vector for keyword-only hits). A hit also counts as relevant by itself if it’s a real keyword match: BM25 > 0 and more than half of the query terms present. That way an exact code survives even when its embedding scores below the cut-off.

The SurrealDB gotcha: @@ is AND-only

SurrealDB 2.x’s full-text match operator only matches rows that contain all the tokens, and there’s no OR form. A 6-word question would match almost nothing.

The workaround: give each term its own match reference and sum the scores.

SELECT id, text, metadata, vector,
  (math::max([search::score(1) ?? 0, 0])
 + math::max([search::score(2) ?? 0, 0])
 + math::max([search::score(3) ?? 0, 0])) AS bm25
FROM type::table($table)
WHERE (text @1@ $t1 OR text @2@ $t2 OR text @3@ $t3)
  AND knowledge_id = $knowledgeId
ORDER BY bm25 DESC
LIMIT $k;

Summing is legitimate, because BM25 is itself a sum over terms. A missing term adds 0. A term that appears in almost every row can score below 0, so each score is clamped at 0. A nice side effect: a term with several tokens (ERR_4521, v2.3, e-mail) still needs all of its own tokens, which keeps codes precise.

The numbers

Our test set: 20 questions over 27 documents, deliberately full of near-duplicate error codes and version families. 12 questions ask for an exact term, 8 are paraphrases. With the real MiniLM model:

Turn the dial to see what each step bought and what it cost, using the recall@1 figures from the table:

What we took from it:

• Hybrid never did worse than vector, and it fixed exact-term misses at rank 1. It’s the default now.
• The cross-encoder rerank (ms-marco-MiniLM-L-6-v2, on device) fixed the last exact-term miss but lost a paraphrase. It’s an opt-in switch (“Rerank on this device”), not the default.
• At @3 everything is already perfect on this set. Rank 1 matters most when only the top few passages fit in a small model’s context.

To be clear, this is 20 questions, not a benchmark. We kept it because it encodes the failures our users actually hit, and it runs in CI with a synthetic embedder. That synthetic embedder is much weaker (vector 0.35 → hybrid 0.90 at @1), which makes it a good canary when the keyword path regresses.

The second bug: filtered KNN that returns nothing

All knowledge bases that use the same embedding model share one table, filtered by knowledge_id. In SurrealDB 2.x, the bare KNN operator <|k|> on an HNSW index runs a top-k over the whole table and filters afterwards.

So imagine a 60-chunk knowledge base next to a 2,940-chunk one in the same table:

Ask for the top 4, and the global top-4 can be all from the big one. Filter to the small one, and you get 0 hits.

The fix has three parts:

1. Filter during the walk. The <|K,ef|> form applies the knowledge_id condition while it traverses the HNSW graph.
2. Over-fetch. fetchK = min(max(3k, k+8), 200), with the search effort ef scaled to it (64 to 800).
3. An exact fallback. If the approximate query errors or returns fewer than k rows, run an exact filter-first scan: WHERE knowledge_id = $kb, cosine similarity, ORDER BY, LIMIT k.

We measured both paths on 3,000 × 384-d vectors: the approximate query takes about 14 ms, the exact scan 160 to 556 ms depending on knowledge base size.

So the fallback is a safety net, not the main path. A low ef (40) had poor recall on the big knowledge base, hence the scaling.

Test this in your own stack: create a tiny collection next to a big one in the same index, and query the tiny one. Many vector stores post-filter by default.

Takeaways

These questions apply to any RAG stack, whatever store you use. If your models run on device as ours do, running an LLM in the browser explains the runtimes.

Have you measured a reranker that hurt? I’d like to know if paraphrases were the casualty for you too.

The knowledge bases described here are part of AWFlow, a browser extension for workflows and AI agents that runs locally.