Semantic search for the price of a git push

I should be clear about my motives. This is not a post about frugality as a virtue. I am simply cheap, and I wanted a search box on a static site that understands meaning. Those two wants are usually in conflict: meaning tends to imply an embedding model, which implies a GPU or at least a process that stays up, which implies a monthly line item I will eventually resent. What follows is how a specific piece of retrieval research lets you have the meaning without the process, and what it honestly costs instead.

01The things I refused to pay for

This site is plain HTML pushed to GitHub Pages. No build step, no framework, no server, and I intended to keep it that way. That rules out the ordinary answer to "make search understand meaning", which is to run an encoder on every query. It also rules out the fashionable answer, which is to ship the encoder to the browser as WebAssembly: the document encoder I ended up using is a 268 MB safetensors file, and asking a reader to download it so they can find a PDF is not a trade I will make on their behalf.

But keyword search would not do either. The corpus is my papers, slides, projects and CV, and it is written in the register of people who say wilderness search and rescue when they mean finding someone lost in the woods. A reader types the second. BM25 matches the first. They never meet.

So the requirement was: retrieval that bridges vocabulary, on a static host, with the browser downloading something closer to a stylesheet than a model. The trick that makes this possible is that SPLADE is asymmetric, and one family of SPLADE models pushes the asymmetry all the way to its conclusion.

02SPLADE in one breath

SPLADE [1] takes a BERT-style masked language model and uses the head it was pre-trained with, the one that predicts a vocabulary token at every position, for something else. For an input of n tokens the MLM head produces an n × |V| matrix of logits, where |V| is the 30,522-entry WordPiece vocabulary. SPLADE squashes it into one vector per passage:

w_j = max_i log(1 + log(1 + ReLU(ℓ_ij)))

The ReLU keeps only positive evidence, the max over positions keeps the strongest occurrence of each vocabulary term, and the double logarithm is the v3 saturation the OpenSearch models use [3]; the original paper used a single log. Training adds a sparsity regulariser, so the vector that comes out has a few hundred non-zero entries rather than thirty thousand. Each non-zero entry has a name, because each dimension is a token. A passage about UAVs may come out with a weight on drone even though the word never appears. That is the "expansion" in Sparse Lexical and Expansion, and it is what BM25 cannot do.

In the standard, symmetric setup both the query and the document go through the model. Scoring is then a dot product of two sparse vectors, which an inverted index does well. The quality is competitive with dense retrieval on the BEIR suite [2], and unlike dense retrieval you can read the vector and see why a result matched.

03The asymmetry, which is the whole trick

Here is the observation that pays for everything. Vocabulary bridging only has to happen once, on one side. If the document side is expanded aggressively enough at index time, the query side can be almost nothing: tokenize, look up a weight per token, done. Models trained this way are called inference-free, because no neural network runs when a query arrives.

There are two schools. The naver splade-v3-doc line gives every query token the weight 1.0, a bag of words. The OpenSearch line [4][3] gives each query token a fixed, learned IDF-style weight, shipped with the model as a text file, and trains the document encoder to expect exactly those weights. I used opensearch-neural-sparse-encoding-doc-v3-distill: 67M parameters, Apache-2.0, an average NDCG@10 of 0.517 on the model card's thirteen BEIR sets. The weights matter more than they sound. From the table this site actually ships:

With bag-of-words weights, the and drones would count the same and a passage that mentions the a lot would win. With these weights the query side already knows what is worth matching, in general, before it has seen a single document. It knows nothing about this corpus; that is the price, and I come back to it in §10.

04A vector you can read

Dense embeddings ask you to trust them. A sparse one shows its working, and I think that is reason enough to prefer it for a personal site where the point is partly to explain the retrieval. Here is the abstract of the SAREnv paper as the document encoder sees it: 160 activated terms, 88 of which appear in the text and 72 of which the model added.

Across the whole corpus the picture is the same. Of 31,556 postings in the index, 59.5% are terms not literally present in their passage. The 203 passages between them touch 5,757 of the 30,522 dimensions. So the "sparse" vector is sparse in the sense that matters, at about 0.5% density per passage, but it is far from a bag of the words on the page.

05The dot product, with the working shown

Retrieval is now a sparse dot product between a query vector that cost nothing and a document vector that was paid for weeks ago. Take the query from the top of this post.

Now the honest counter-example, because this is where inference-free stops being magic and starts being a design choice. Type heart attack into this site and the top result is a section of a U-space planning paper, with a score of 1.01, entirely from an expansion term heart the model put on a passage about UAV corridors. Type cardiac arrest and you get nothing at all. In a symmetric SPLADE the query encoder would have expanded cardiac towards heart. Here the query side has no encoder, so the bridge has to have been built from the document side, and no document on this site is about hearts. Inference-free retrieval can only be as clever as the document encoder's guesses about future queries.

06The factory: GitHub Actions

The heavy pan of Figure 1 is a workflow. Any push that touches an indexed page, a deck, a paper text or the build script itself triggers search-index.yml, which installs the CPU wheel of torch, pulls the model from the Hub, encodes every passage and commits the result back to the branch. GitHub-hosted runners are free for public repositories, and the whole thing takes between two and three and a half minutes on the runs I have on record, most of it pip install.

# the part that does the work, trimmed
- name: Install dependencies
  run: |
    pip install torch --index-url https://download.pytorch.org/whl/cpu   # not the CUDA wheel, no GPU here
    pip install -r tools/requirements.txt
- uses: actions/cache@v4                 # ~/.cache/huggingface, so the 268 MB is fetched once
- run: python tools/build_search_index.py --out search
- run: python tools/check_search_index.py     # browser tokenizer == HuggingFace tokenizer, or fail
- run: |
    git add search/index.json search/qweights.u16.bin search/vocab.txt search/corpus.json
    git diff --cached --quiet || { git commit -m "Rebuild the SPLADE search index"; git push;
                                   gh workflow run pages.yml --ref "$GITHUB_REF_NAME"; }

Four details earned their place the hard way.

1. Determinism. Multi-threaded CPU reductions sum in a nondeterministic order, which moves the odd weight by a hundredth and reshuffles ties. Invisible on the page, but it made every rebuild a fresh diff, so the "index unchanged, nothing to commit" branch could never fire. torch.set_num_threads(1) costs a few seconds on a corpus this size and buys byte-identical output on the same runner image. Not across images: the rebuild this very post triggered landed on a fresh runner with newer wheels, and twelve of the 203 passages moved by one quantisation step, one posting appeared, and the diff was a single line of JSON. The numbers in this post are read from that rebuilt index.
2. The token gotcha. A push made with the workflow's own GITHUB_TOKEN deliberately does not trigger other workflows, so the commit carrying the new index never started the Pages deploy. Every rebuilt index was sitting undeployed until my next human push. The job now dispatches the deploy itself.
3. The check step. The browser has its own WordPiece implementation, and a tokenizer that disagrees with the one the model was trained with fails silently: results are merely, quietly wrong. So the build tokenizes a fixture of awkward strings (grøntved, ESP-NOW, TrajAllocPy, café résumé naïve) both ways and scores the same queries both ways, and the job fails if either disagrees beyond 10⁻⁴.
4. Truncation. The encoder reads 512 tokens. Paper sections are chunked to about 1,800 characters between sentences, so nothing the model is scored on was cut off unseen.

07The shop floor: GitHub Pages

The light pan is three files and a script. Nothing is fetched until the reader focuses the search box, so a visitor who never searches pays nothing for the feature.

That is a 930× reduction in what the reader downloads, for the same document vectors. The quantisation is deliberate and its error is bounded: document weights are stored as round(100·w) in one byte, so the absolute error is at most 0.005. The smallest weight kept is 0.05 (anything below is dropped, then the top 160 terms are kept), so the worst relative error on the least important term is 10%, and on the terms that decide a ranking, which sit around 0.5 to 0.8, it is under 1%. Query weights are stored at 1/1000 and the error there is negligible. The CI check above compares the browser's scores to an unquantised reference, so the bound is not just argued, it is tested on every build.

Scoring is brute force: for every passage, walk its postings and multiply where the query has a weight. There are 31,556 postings in total. In Node on a laptop-class CPU that is 0.7 ms per query, and building the engine from the fetched files is 11 ms, both measured with the same splade.js the browser runs. An inverted index would make each query cheaper and the code less legible, and at this scale legibility wins; the point of the search page is to show the arithmetic, not to hide it.

08The one part that talks to a server

Retrieval is the whole of the search page and the whole of the front page until someone presses Answer this. Then, and only then, the four best passages are handed to a language model that writes two to four sentences with bare [n] citations. This is the part that costs money in most RAG systems, and it is the part where being cheap took the most engineering.

The problem is not the model. Mistral's Experiment tier is free and is enough for a personal site by a wide margin. The problem is that a static site cannot hold a secret. Every byte GitHub Pages serves is public, so an API key in the JavaScript is an API key on the internet. Something has to hold the key, and it has to be free too.

That something is a Cloudflare Worker on its own hostname, on the Workers Free plan. It does one job: the browser sends it passage ids, never text, it resolves them against a copy of the corpus fetched from this site, and it forwards the question and those passages to Mistral with the key attached. Because it only accepts ids, it can only ever generate from this site's own corpus: you cannot paste in a document and have it summarised on my key.

The design principle is that this stays free structurally rather than carefully. Workers Free has no overage billing. The Cloudflare account has no payment method. The Mistral organisation is on the Experiment tier, whose limits are per second, per minute and per month and live in the admin console rather than the docs. So every limit in the system, when hit, produces an error, and every error the browser sees means it leaves the retrieved passage on screen and stops offering to generate. The caps inside the worker make that degradation rare and quiet. They do not prevent a bill, because there is no bill to prevent.

Two constraints on the model's output are worth stealing. It is told to cite as bare [1], [2] and never to write a URL; the page resolves those numbers against the ordered id list it sent, so a hallucinated link is not representable, it just stays as text. And the passages come with their full text, not their 240-character snippets, because I once shipped the snippets and had the model confidently answer from four opening sentences.

09The receipt, itemised

Every free tier has a limit that would bind first if this site were popular. Here they are, with what actually happens at the limit, as documented by each provider at the time of writing.

The 10 ms CPU budget is the tightest one and it shaped the code. Parsing the 317 KB corpus measures 0.66 ms, streaming the answer back is done by handing Mistral's response body straight through with a clone() for the cache rather than transforming it, and the per-IP counters are the Cache API rather than KV because KV allows a thousand writes a day and a busy afternoon would spend them on accounting.

10Where the cheapness stops

Being rigorous about cost means being rigorous about where the argument fails.

Scale

The index is 2.1 KB per passage uncompressed and 0.6 KB gzipped, and brute-force scoring is linear in postings. Extrapolating from the measured figures:

Those are estimates from a 203-passage measurement and should be read as such; the download size is the reliable column, the timing assumes the loop stays cache-friendly, which it will not. The honest reading is that this architecture is right for a corpus you could read in an afternoon and wrong for one you could not. Around a few thousand passages the per-term posting-file variant from §7 becomes worth writing. Somewhere past that, Lucene-based engines do this natively and OpenSearch ships this exact model for the purpose [4], which is the not-free path.

What the query side cannot know

The static weights are a learned IDF over the training corpora, mostly MS MARCO-shaped web text. They know drone is far more discriminative than system in general, and the table says so: 8.71 against 3.43. They do not know that on this site the two words are equally ordinary, with drone in the vectors of 38% of the passages and system in 36%. A corpus-specific IDF would fix that, and I could compute one in CI, but it would break the contract the document encoder was trained under, and I have not measured whether the trade is worth it on 203 passages. That is a real experiment I have not run, so I will not claim the answer.

Known items

WordPiece shreds proper nouns: ICUAS 2024 becomes ic ##ua ##s 202 ##4, and Grøntved becomes gr ##ø ##nt ##ved. The fragments happen to be rare enough here that the right thing still surfaces, but that is luck, not design. Meaning is what this is for. Finding a specific known thing is what a metadata filter is for, and the search page has those.

English only, and the other kind of cost

The model is English. Danish queries tokenize into fragments that match by accident. And the free tiers are not free in the sense of costing nobody anything: a free API tier is typically paid for in the prompts you send, so read the current terms before you route anything sensitive through one. The questions readers ask this site are about my papers, which are public, and the passages are already on the internet. That is the case where the trade is fine. It is not every case.

11Steal it

Everything is in the repository and none of it is clever enough to deserve a package:

• tools/build_search_index.py: extract passages from HTML with the standard library, encode them, quantise, write the three files. The v3 activation is the double logarithm; do not "simplify" it to the v1 formula.
• search/splade.js: WordPiece, the dot product, the loader. The search itself is six lines inside search().
• .github/workflows/search-index.yml: the factory, including the two lines that dispatch the deploy.
• worker/: the key holder and its gates, with the grounding prompt in a file of its own because the prompt is the interesting part and hiding it would only hide it from people acting in good faith.

And go and type something into the search. Pick a result, and the page shows you the query decomposed, the terms that scored, and the whole 30,522-dimensional vector as a strip of mostly nothing. Then try cardiac arrest, and enjoy the silence. It is the most instructive result on the site.

12Up next, if the receipt stays at zero

Three things I want to try, in rising order of how likely they are to cost money.

Inference-free visual SPLADE

The document encoder does not have to read text. V-SPLADE [7] puts a vision-language model on the document side, reads the page as an image, and still leaves the query side as a tokenizer and a weight table. Caption-gated supervision teaches it which vocabulary terms a picture should light up, and the paper reports it more than doubling recall over a same-scale dense model on an 18.7M-document corpus. The vocabulary is larger, but the shape of the query table is identical, and splade.js was written not to care about the vocabulary size for exactly this reason. My slides are already images with structure, and the papers have figures nobody can search. The open question is whether a VLM encoding a few hundred page images fits inside a free CPU runner's six-hour job limit. I expect it does, slowly, and I intend to time it rather than guess.

A vocabulary of my own

Grøntved becomes gr ##ø ##nt ##ved because the vocabulary was cut for English web text in 2018. Yu, Mallia and Petri [8] show that pre-training the encoder on a corpus-specific vocabulary improves learned sparse retrieval by up to 12% and, because the posting lists get shorter, cuts latency by up to half. For this site the interesting version is a Danish-and-robotics vocabulary. It means training the document encoder again, which is the first thing on this list that a CPU runner might refuse to do for free. A 67M model, in pieces, across several six-hour jobs, with the checkpoint committed between them, is either a clever use of a free tier or an abuse of one, and I have not decided which.

SPLARE, and the end of readable dimensions

The dimensions of a SPLADE vector are words. That is what made §4 possible. SPLARE [9] gives that up: it trains a sparse autoencoder on an LLM's hidden states and retrieves over the autoencoder's latent features instead of the vocabulary, which buys a representation that is multilingual and holds up out of domain. The latents are interpretable in the mechanistic sense, but they no longer have a WordPiece spelling, so the figures in this post would need a legend generated by a model rather than read off a file. It is also not inference-free: the query goes through the LLM too. Whether a static table from tokens to latents can stand in for that, the way an IDF table stands in for the query encoder here, is a research question rather than a weekend, and the honest teaser is that I do not know. It is the one on this list most likely to end with a GPU invoice, which is how I will know it was worth it.

SourcesReading list

1. T. Formal, B. Piwowarski, S. Clinchant. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. SIGIR 2021. arXiv:2107.05720
2. T. Formal, C. Lassance, B. Piwowarski, S. Clinchant. SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval. 2021. arXiv:2109.10086
3. X. Shen, Z. Geng, Y. Yang. Exploring ℓ₀ Sparsification for Inference-free Sparse Retrievers. SIGIR 2025. The paper behind the v3 models. arXiv:2504.14839
4. Z. Geng, Y. Wang, D. Ru, Y. Yang. Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrievers. 2024. Where the IDF-weighted query side comes from. arXiv:2411.04403
5. OpenSearch Project. opensearch-neural-sparse-encoding-doc-v3-distill, model card with the BEIR table and the reference encoding code. Apache-2.0.
6. K. Shivendu. Inference-Free SPLADE: Full Quality, 13× Faster Queries. 2026. A proper benchmark of inference-free versus full SPLADE on BEIR SciFact across two engines; the post that nudged me to write this one up, and the place to go if you want the numbers rather than the receipt.
7. G.-H. Cho, Y. Lee, K. Jeong, S. Lee, S. Han, H. Déjean, S. Clinchant, S.-w. Hwang. Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search. 2026. V-SPLADE. arXiv:2605.30917
8. P. Yu, A. Mallia, M. Petri. Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies. ECIR 2024. arXiv:2401.06703
9. T. Formal, M. Louis, H. Déjean, S. Clinchant. Learning Retrieval Models with Sparse Autoencoders. ICLR 2026. SPLARE. arXiv:2603.13277
10. Provider limits, as read on 8 September 2026: GitHub Pages, GitHub Actions billing, Workers pricing, Mistral usage limits.