We rebuilt RAG in a browser tab. Our first bug: every vector was empty.

You add your documents to an AI assistant, it says they are indexed, and then it answers as if it had never seen them. Most people conclude the AI is dumb. Often the pipeline behind it is quietly returning less than it should.

That is the story of our own knowledge bases. They live entirely in the browser: documents are chunked and embedded on your machine, stored in a database that runs inside the tab, and searched from chats, agents and workflows. Nothing goes to a server unless you choose a cloud model.

We recently rebuilt the whole thing: 28 issues covering storage, ingestion, search, citations and privacy. The audit before it started with a bad surprise.

This post is the list of what we changed, one lesson per section. RAG rarely fails loudly. It fails by returning less, and each lesson below is a place where it did.

Indexing that tells the truth

First, what sits under the hood. Each layer runs on your machine.

If the database-in-a-tab part sounds unlikely, WebAssembly, explained covers how a browser tab became a small computer.

1. Count what you actually wrote

The vector field was declared TYPE array in a SCHEMAFULL table, and SurrealDB saved every vector as []. The Indexer node reported success every time. Users indexed documents, got a green badge, and then searched nothing.

The fix for the empty vectors was one type (array<number>). The real fix was making the writer honest:

• Every write returns added / updated / failed / removed.
• A partial failure says “N of M passages could not be saved”.
• Writing nothing is an error, not a success with zero.
• The canvas badge reads “24 added · 3 updated”, not “Success”.

Lesson: an indexer that can’t fail can’t be trusted. Count real rows with real vectors, and treat zero as a failure.

2. Re-adding a document should never duplicate it

Every chunk gets a deterministic id: chunkId(kb, source, index). Writes are upserts. So:

• Re-adding the same page replaces it in place.
• If the new version is shorter, the extra chunks (index ≥ new count) are deleted.
• If the text hash hasn’t changed, the source is skipped.

Each chunk carries its provenance: source id, hash, type, title, URL or file name, chunk index and count, page number for PDFs, and the workflow that indexed it.

Lesson: if your ingestion isn’t idempotent, every re-run makes retrieval worse, because duplicates crowd out everything else in the top-k.

3. Vectors need a birth certificate

Vectors from different embedding models live in different spaces. Comparing them gives confident nonsense. We also had a migration in the past: the old default was TF.js Universal Sentence Encoder (512-d), the new one is MiniLM (384-d).

So each knowledge base records the model that first filled it, and each model has its own table, because a vector index has one fixed dimension. Every reader and writer opens a knowledge base through one function that routes to that model and that table.

Before, the chat assistant had its own routing and read the legacy table, so knowledge bases built with MiniLM returned 0 passages in chat.

Switching models is a re-embed: copy every chunk into the target model’s table, catch up chunks added in the meantime, then switch with one catalogue update. Any failure rolls back, and searches keep reading the old table until the switch. It’s safe to interrupt.

Lesson: store the model next to the vectors, and make “which model?” impossible to get wrong.

Search that misses, and why

Once the vectors were real, the next failures were about what search returns, or doesn’t.

4. Vector search can’t find ERR_4521

Embeddings are great at meaning and bad at exact strings. ERR_4521 and ERR_4512, v2.3 and v3.2: to a sentence model, these are nearly the same thing.

We added BM25 keyword search next to vector search and fused the two ranked lists with reciprocal-rank fusion. On our test set (20 questions over 27 documents full of near-duplicate error codes and versions), with the real MiniLM model:

Hybrid never did worse than vector, so it’s the default. The cross-encoder rerank fixed the last exact-term misses but hurt paraphrases, so it’s opt-in.

It’s a small test set, and I won’t pretend otherwise. The point is the method: build the eval from the failures your users hit, and keep it. The full details, including a SurrealDB full-text gotcha, are in part 2: Why vector search misses exact terms, and what hybrid fixed.

5. Filtered vector search can return nothing

Most apps keep many collections in one vector table and filter by collection. We found that in our engine, the plain KNN operator on an HNSW index does a top-k of the whole table first and filters afterwards.

A small knowledge base that shares a table with a big one can get zero hits: its chunks simply never make the global top-k.

The fix: a query form that applies the filter during the graph walk, over-fetching (fetchK = min(max(3k, k+8), 200)), plus an exact filter-first fallback when the approximate query returns too few rows. Measured on 3,000 × 384-d vectors: about 14 ms approximate vs 160–556 ms exact, so the fallback is a safety net, not the main path.

Lesson: test retrieval on a small collection next to a big one. That’s where filtering bugs hide.

6. Follow-ups retrieve nothing

User: What does the contract say about termination? User: And the notice period?

“And the notice period?” has almost nothing to search for. Now, when a message is short and anaphoric (≤ 12 words with “it/that/and…”, or ≤ 3 words), it’s rewritten into a standalone query from the last 4 turns.

There’s an 8 s timeout, and on failure it falls back to the original text.

Lesson: retrieval sees one message. Conversations don’t work that way.

Explain, protect, and share

The last four lessons are about what the user sees, what the cloud never sees, and what happens to the data.

7. When retrieval finds nothing, say why

“I couldn’t find anything in your documents” is the most frustrating sentence in RAG. Now every knowledge base gets one step in the reply:

• Docs · 3 passages
• · 2 weaker matches
• · nothing relevant, with the best score against the cut-off (0.35 by default)
• · empty
• · skipped · on this device only, naming the cloud model

The steps come with two actions: Use weaker matches for this reply and Test this search, which opens a search panel with scores and the cut-off line. Citations open a passage card showing the knowledge base, chunk, page and score, with a link to the source.

Lesson: show the score and the threshold, and the user can fix it in one click.

8. “Private” has to be a gate, not a setting

Each knowledge base is Device only or Cloud OK. A device-only one never goes into a prompt for a cloud model, and that’s enforced on every path: chats, agents, team threads, routines and workflow nodes. “Auto” or unknown models count as cloud. The assistant skips the knowledge base and says so. Workflow nodes fail by default (or skip, if you choose).

Retrieved passages are always fenced as “reference data, not instructions”. The only model-driven write (add_to_knowledge_base) asks before every call, and teams don’t get it.

Lesson: if privacy depends on each feature remembering a flag, one of them will forget. Put the check in the one function every path uses.

9. One single-threaded engine, one lock

The knowledge base database and the app’s local database are two IndexedDB databases on one SurrealDB WebAssembly module, which isn’t re-entrant. When two calls interleaved, it crashed with errors like “FnOnce called more than once” and “memory access out of bounds”. The knowledge base page went blank.

The fix: every engine call, from both databases, goes through one promise-chain lock. And each JS context now shares one connection, instead of one engine per knowledge base (about +10 MB each).

Lesson: WebAssembly modules are often single-threaded under the hood. If two subsystems share one, they share its limits too.

10. Make the data portable

A knowledge base exports to a .awkb file: gzip JSON with the model reference and every chunk with its vector as float32 base64, about 4× smaller than JSON numbers. Importing never re-embeds and always creates a new knowledge base. Every file is validated as untrusted input (size caps, schema, version).

Lesson: if the data lives on the user’s device, export is part of the core design, not a nice-to-have.

What’s still limited, and your checklist

Some things don’t work yet, and I’d rather you hear it here:

• Web URLs are fetched with plain fetch, so sign-in or CORS-blocked sites fail. Adding the open tab works (extension).
• Project-file search is vector-only, not hybrid yet.
• No sync between the web app and the extension (different origins).
• Knowledge base text sits in IndexedDB unencrypted, like the rest of local mode.
• Extension-only flows (right-click save, a real re-embed on a big knowledge base) are still being tested by hand.

If you run the embedding model on device as we do, running an LLM in the browser explains the runtimes behind it. And whatever your stack, these are the questions I’d ask of any RAG pipeline:

What’s the sneakiest RAG bug you’ve shipped? Ours was the empty vectors.

The knowledge bases described here are part of AWFlow, a browser extension for workflows and AI agents that runs locally.