I built a search layer over my own notes so I could text my assistant a question and get an answer out of my vault instead of a guess. An indexer walks the notes, splits them on headings, embeds each chunk with a local model, and writes the vectors into SQLite. 247 files, 1,455 chunks. Ask a question, embed the question, pull the nearest chunks, hand them to a model with instructions to cite its sources.

It worked immediately, which should have made me suspicious.

Then I asked it for my Rotella plan — a golf practice note I’d written months earlier — and it came back with nothing useful. The file was indexed. I checked. It was in the database, chunks and embeddings and all, and the search would not surface it.

The tell was that a longer question found it instantly. Ask about the mental-game approach in that book, in a full sentence, and the right file came back first every time. Ask for “rotella plan” and it vanished.

Embeddings are good at aboutness and bad at rare tokens

That gap is the whole diagnosis. A semantic embedding encodes what a passage is about. Two words carrying one unusual proper noun barely constitute an aboutness — there’s almost no context for the model to place, so the query vector lands somewhere vague, and vague is equidistant from everything. A 500-token chunk about golf psychology and a 500-token chunk about network hardware are both, from there, sort of far away.

This is not a flaw you fix by swapping embedding models. Every semantic retriever has this shape. Short, rare, high-signal tokens are exactly the thing a keyword index is good at, and exactly the thing a vector index is worst at.

So: add a keyword index. The pleasing part is that I didn’t need anything new to do it — SQLite ships FTS5 compiled in. Full-text search with BM25 ranking, no new package, no service, no daemon. It was already sitting inside the database I was already using.

The interesting problem is fusion, not retrieval

Running two retrievers is easy. Combining them is where people go wrong, including me for about an hour.

The naive move is to blend the scores. It doesn’t work, and it’s worth understanding why: cosine distance and BM25 aren’t measuring the same thing on the same scale. Cosine sits in a bounded range; BM25 is unbounded and its magnitude depends on corpus statistics — term rarity, document length, how many documents you happened to match. There’s no principled constant that converts one into the other.

The usual patch is to min-max normalize each result set before blending. That’s worse than it looks, because now every score depends on the set it arrived in. Add one more mediocre result to the bottom of a list and you’ve rescaled everything above it. Your top result’s score changes because of a document nobody will ever read.

Reciprocal Rank Fusion sidesteps all of it by throwing the scores away and keeping only the order. Each retriever contributes 1 / (k + rank) for every document it returns, summed across retrievers:

K_RRF = 60  # Cormack et al., 2009

def _rrf_fuse(*result_lists, k_rrf=K_RRF, final_k=8):
    """Each input list is chunk_ids, best-first."""
    scores = {}
    for results in result_lists:
        for rank, chunk_id in enumerate(results, start=1):
            scores[chunk_id] = scores.get(chunk_id, 0.0) + 1.0 / (k_rrf + rank)
    ranked = sorted(scores.items(), key=lambda x: -x[1])
    return [cid for cid, _ in ranked[:final_k]]

That is the entire fusion layer. Ten lines, no configuration, no tuning constants I have to maintain.

Three properties earn it its place. Only ordinal information crosses the boundary, so there’s nothing to normalize and nothing to calibrate. The constant k — 60 is the value from the original paper and I’ve never had cause to move it — damps the top of each list, so being one retriever’s #1 is a strong signal but not an overwhelming one; a document both retrievers merely like can and should beat a document one of them loves. And a retriever that returns nothing contributes nothing, automatically. No branch, no fallback path, no “if hybrid enabled” config. An empty list is just a list that adds zero.

That last property is why I refused to build a hybrid-vs-vector-only switch. RRF degrades to single-retriever ranking on its own. A toggle would only have given me a second code path to test and a setting to get wrong.

The trick that actually fixed my query

Here’s the part I’d have missed if I’d only indexed chunk text.

The FTS5 table indexes three columns: the chunk text, its heading path, and a filename slug derived from the note’s path — separators replaced with spaces so the path becomes searchable tokens:

def filename_slug(path: str) -> str:
    stem = path[:-3] if path.endswith(".md") else path
    return re.sub(r"[/\-_]+", " ", stem).strip()

So personal/golf/rotella-application-plan.md becomes personal golf rotella application plan.

This is why the failing query works now. The proper noun I was searching for lived in the filename, and the body of the chunk mostly discussed the ideas without repeating the name. Semantic search couldn’t reach it because the body isn’t about the word; keyword search over the body alone wouldn’t have found it either. Indexing the path is what closed the gap — and it costs one derived column.

Additive by design

The change is purely additive, which is why shipping it was uneventful:

  • One virtual table and one trigger. The delete trigger mirrors the existing one for embeddings, so when a file is removed the foreign-key cascade tears down its chunks and both side tables stay in sync. No orphans, no cleanup job.
  • A one-time backfill guarded on count > 0. The first indexer run after deploy populates the keyword index from chunks that already exist; every run after that is a no-op. Idempotent by construction, so I never had to think about running it twice.
  • No re-embedding. The existing vectors were untouched. Adding an entire second retrieval strategy cost zero embedding calls.
  • Rollback is one file. Revert the query module and the indexer keeps writing keyword rows harmlessly into a table nobody reads. If the table itself ever became a problem, drop it — the schema is purely additive.

Storage for 1,455 chunks: about 400 KB. Each retriever returns 20 candidates, fusion picks the top 8 for the model. Test count went from 36 to 62. Query latency was unchanged — fusion is a dictionary and a sort over 40 integers, which is not where your time goes when there’s a language model at the end of the pipeline.

Three bugs that reported success

Three things bit me around this work, and all three share a shape: the system said everything was fine.

An indexer run finished with errors=7 — seven large files whose oversized sections silently truncate at embed time rather than failing loudly. The run “succeeded.” The count was right there in the log line and I’d been reading past it for a day, because a summary line that ends in a number you’ve decided is normal is a summary line you’ve stopped reading.

Worse: a subprocess I depended on was exiting with a nonzero code and writing its actual error message to stdout instead of stderr. The calling code checked the return code, took the failure branch, and reported a clean zero result. There was nothing to see — just a quiet “found 0 items” indistinguishable from an honest empty result. I only found it by patching the call site to log the return code, stdout, and stderr all three, at which point a permissions error I’d been staring past for hours printed itself in one line.

And a set of scheduled jobs were recording last_status: ok while carrying a last_delivery_error explaining that no delivery target could be resolved. The jobs ran. They completed. They sent nothing, successfully, for weeks. A status field that reports whether the code finished, rather than whether the work happened, is a status field that will lie to you eventually.

What I’d tell someone building the same thing

On this corpus, semantic search failed in a specific and repeatable direction: short queries carrying rare tokens. If a RAG setup mostly works but sometimes cannot find an obvious file, that is one failure mode worth testing. Prompt work upstream cannot recover a document the retriever never handed to the model.

What worked here was not a stronger single retriever. It was a second retriever with different failure modes, combined without score calibration. On my benchmarks, two ordinary rankers fused by rank beat the carefully tuned single ranker and have stayed useful as the corpus changed. I expect there are better combinations; this is the simplest one I have found so far.