How it works
Five stages, all running on free infrastructure. Scroll through them: each one is real code in the repository, linked below its description.
No git clone, no per-file API calls.
Fetch the whole repo in one request
grepless downloads the default branch as a single gzip tarball and streams it through a small hand-written tar reader. The first header, a pax global record, carries the exact commit SHA, so every link it gives you is a permalink.
- 1 HTTP request per repo
- 60 MB compressed cap
- commit SHA from the pax header
source → lib/github.ts
GET codeload.github.com/…/tar.gz/HEAD
repo.tar.gz
pax: comment=
0d59458a0a58…
Dependencies and build output would drown the signal.
Throw away everything that isn’t code
Before a file is even read into memory, its path is checked against ignore rules for dependencies, build output, lockfiles, generated files and binaries. Survivors are sniffed for null bytes and minified one-liners.
- node_modules · vendor · dist · .next
- lockfiles, *.min.js, *.map, *.d.ts
- 200 KB per-file cap
source → lib/languages.ts
checkPath() → checkContent()
the sieve: 30+ ignore rules, 200 KB cap, binary + minified sniffing
Chunks are functions, not arbitrary line windows.
Split along the syntax tree
tree-sitter parses 13 languages. The walker keeps each function, method and class as one chunk, merges tiny neighbours (imports, constants) and splits oversized nodes along their children. Every chunk remembers its qualified name, like Ky.#retryFromError.
- 13 grammars in WASM
- ≤ 1,500 chars per chunk
- qualified symbol names
source → lib/chunker.ts
tree-sitter → chunks
- ◆program
- └import_statement
- └class_declaration · Client
- └method · retry
- └method · refresh
- └export · createClient
chunk 1
retry
chunk 2
refresh
chunk 3
createClient
gte-small runs inside the serverless function.
Embed every chunk, in-house
Each chunk, prefixed with its path and humanised symbol name, becomes a 384-dimension unit vector from an int8-quantised ONNX model. The browser drives three workers that claim batches with FOR UPDATE SKIP LOCKED, so no request runs long and a closed tab only pauses the job.
- 384 dims · cosine
- ≈ 12 chunks/s on the free tier
- 0 API keys
source → lib/embedder.ts · lib/indexer.ts
gte-small · mean-pooled · normalised
[0.021, -0.113, 0.087, …]
Hybrid retrieval, fused without tuning score scales.
Rank by meaning and by name
Your question is embedded with the same model. Postgres returns the 40 nearest chunks from an HNSW index and the 40 best full-text matches, where symbol and path words outweigh body words. Reciprocal Rank Fusion merges the two lists, and each hit explains which list found it and why.
- pgvector HNSW (iterative scan)
- weighted tsvector (A/D)
- RRF k = 60, keyword ×0.6
source → lib/search.ts
score = 1/(60+rank_sem) + 0.6 · 1/(60+rank_kw)
HNSW · cosine
1. #calculateDelay
2. calcRetryTiming
3. raceBodyRead
4. #calcRetryDelay
GIN · ts_rank_cd
1. #calcRetryDelay
2. calcRetryTiming
3. retryFromError
4. retry-timing.md
Three moving parts, all on free tiers.
drives the job
Your browser
- POST /api/repos
- 3 embed workers
- search · explain
Mumbai · bom1
Vercel functions
- prepare: tar → AST chunks
- embed: gte-small ONNX
- search: hybrid SQL
Mumbai · ap-south-1
Supabase Postgres
- chunks.embedding vector(384)
- HNSW + GIN indexes
- RLS · least-privilege role
384
dimensions per chunk
13
tree-sitter grammars
~50 ms
warm search latency
0
API keys needed