vannevar

Local RAG search for markdown files. Combines SQLite FTS5 keyword search with HNSW vector search, fused via Reciprocal Rank Fusion for high-quality hybrid retrieval. Runs entirely offline with ONNX or native llama.cpp embeddings.

Named after Vannevar Bush, who envisioned the Memex — a device for storing, linking, and retrieving personal knowledge.

Features

• Hybrid search — BM25 keyword search (SQLite FTS5) + semantic vector search (HNSW) fused with RRF (k=60)
• Field-aware lexical search — body, title, path, headings, collection context, and bounded generic frontmatter receive separate BM25 weights and explain traces
• Quality modes — query batch-reranks bounded candidates with a local MiniLM cross-encoder; --fast skips it when latency matters
• Automatic query expansion — plain hybrid queries gain conservative lexical, semantic, and HyDE variants locally; no API or generative model required
• Structured queries — qmd-style intent:, lex:, vec:, and hyde: variants are fused with explicit retrieval paths
• Metal acceleration — new Apple-Silicon indexes use llama.cpp/Metal automatically when its optional runtime is installed; ONNX remains the portable fallback
• Fully local — no API keys or hosted inference; selected models download once into a local cache
• Markdown-aware chunking — respects heading hierarchy, keeps code blocks and tables atomic, applies overlap
• Vault collections — named roots, relative scan globs, inherited path context, and collection-scoped retrieval
• Sync-safe indexing — cross-document embedding batches, durable checkpoints, and optional polling watch mode for iCloud-style rename bursts
• Evaluation-ready — versioned relevance fixtures, paired bootstrap intervals, holdout slices, and competitive QMD gates
• Bun native — built on bun:sqlite, Bun.file, Bun.CryptoHasher
• Root-aware indexes — one DB can safely hold multiple indexed directories with overlapping filenames

Install

Requires Bun v1.3+.

bun install

The default embedding profile is EmbeddingGemma 300M q8. New indexes resolve --embedding-backend auto to llama.cpp/Metal on Apple Silicon when the optional runtime is installed and otherwise use EmbeddingGemma ONNX. Existing indexes retain their recorded backend; pre-backend indexes are treated as ONNX for compatibility.

Usage

Index documents

# Index a directory of markdown files
bun src/cli.ts index ./docs

# Index a single file
bun src/cli.ts index ./notes/readme.md

# Custom options
bun src/cli.ts index ./docs --chunk-size 512 --overlap 100 --ext .md,.txt

# Select an embedding profile or an experimental Matryoshka dimension for a new index
bun src/cli.ts index ./docs --db .vannevar-qwen --model qwen3
bun src/cli.ts index ./docs --db .vannevar-256 --model embeddinggemma-256

# Force a portable ONNX index or a native llama.cpp index
bun src/cli.ts index ./docs --db .vannevar-onnx --embedding-backend onnx
bun src/cli.ts index ./docs --db .vannevar-metal --embedding-backend llama

Search

# Hybrid search (default, best quality)
bun src/cli.ts query "how to configure authentication"

# Skip reranking for a lower-latency hybrid query
bun src/cli.ts query "how to configure authentication" --fast

# Plain hybrid queries expand locally by default; disable it for exact diagnostics
bun src/cli.ts query "where is the current product strategy" --no-expand

# Keyword search only (fast, BM25)
bun src/cli.ts search "authentication config"

# Vector/semantic search only
bun src/cli.ts vsearch "setting up user login"

# Interactive fzf-style typeahead search (FTS is the fast default)
bun src/cli.ts tui
bun src/cli.ts tui authentication --limit 15

# Use the persistent vector or hybrid runtime while typing
bun src/cli.ts tui --mode hybrid --fast

# With options
bun src/cli.ts query "auth setup" --limit 5 --mode hybrid --json
bun src/cli.ts query "auth setup" --files --min-score 0.3
bun src/cli.ts search "user:admin -deprecated" --md --line-numbers

# Typed variants: lex only hits FTS; vec/hyde only use embeddings
bun src/cli.ts query $'intent: Locate design details\nlex: indexing pipeline\nvec: batched markdown embedding implementation' --json --explain

# Keep result lists diverse and bound reranking work
bun src/cli.ts query "embedding pipeline" --candidate-limit 40 --rerank-limit 8 --max-per-document 1

# Attach neighboring evidence without changing the ranked child chunk
bun src/cli.ts query "embedding pipeline" --context-radius 1 --json

# Experimental one-round pseudo-relevance feedback (off by default)
bun src/cli.ts query "specialized terminology" --prf --explain --json

query enables reranking by default for vector and hybrid modes. Use --fast or --no-rerank to skip it; --rerank explicitly enables it for vsearch. Plain hybrid queries also expand by default into compact lexical, semantic, and HyDE retrieval variants. Full search omits HyDE because the cross-encoder recovered the same measured quality without its extra embedding pass. The original query remains highest weighted, strong exact FTS matches skip expansion, and user-authored structured queries remain unchanged. Use --no-expand for exact diagnostics or --expand to opt a structured query in.

tui opens an interactive terminal search with live results after each short pause in typing. It keeps one search runtime open for the session, defaults to FTS for responsive typeahead, and supports the same --collection, --min-score, and retrieval options as the search commands. Use --mode hybrid --fast for live semantic retrieval without local reranking. Arrow keys select results, Enter prints the selected path:start-end reference, and Ctrl-C or Escape exits.

candidate-limit controls retrieval recall. rerank-limit separately bounds the more expensive MiniLM cross-encoder; candidates outside the rerank window retain their retrieval order. Confidence routing can skip exact, well-supported matches, and otherwise scores at most 8 query-focused 128-token candidate windows by default. Use --force-rerank for fixed-window diagnostics.

Reranking defaults to portable ONNX. A converted static Core ML model and the helper under experiments/coreml can be selected explicitly with --reranker-backend coreml --coreml-model <path> --coreml-helper <path> on macOS. It is intentionally not selected automatically because warm kernel gains have not yet offset the measured cold-start cost.

Embedding backends

Backend identity is part of the index fingerprint and is stored in vannevar_meta. Search always reopens the backend that created the vectors. Switching between ONNX and llama.cpp requires a new --db path; Vannevar rejects an in-place switch instead of mixing vector spaces.

The native backend currently supports the EmbeddingGemma profile. Its GGUF model is cached under ~/.cache/vannevar/models, or under VANNEVAR_MODEL_CACHE when set. VANNEVAR_LLAMA_CONTEXTS can explicitly select 1-16 parallel native embedding contexts. Use doctor --json to see the persisted profile, backend, fingerprint, and vector health.

Collections and vault context

Collections make a single database practical for several vaults while keeping search and embedding context scoped to the right source.

# Register an Obsidian vault (the glob is relative to the vault root)
bun src/cli.ts collection add ~/Library/Mobile\ Documents/iCloud~md~obsidian/Documents/agents --name agents --pattern '**/*.{md,mdx}'

# Context at the vault root and a more specific path; both are embedded for matching files
bun src/cli.ts context set agents . "Private agent notes and operating procedures"
bun src/cli.ts context set agents projects/vannevar "Vannevar design, benchmarks, and retrieval work"

# Index once, then keep a synced vault updated with deliberate polling
bun src/cli.ts collection index agents
bun src/cli.ts collection index agents --watch --watch-interval 2000

# Persist progress every 5,000 new chunks during a large first-time index
bun src/cli.ts collection index agents --checkpoint-chunks 5000

# Search only this vault, or several named vaults
bun src/cli.ts query "retrieval decisions" --collection agents
bun src/cli.ts query "release process" --collection agents,work --fast

Changing a collection context changes that collection's index fingerprint, so the next index scan re-embeds affected files. collection remove <name> removes both its SQLite records and HNSW vectors.

Other commands

# Retrieve a specific document or chunk
bun src/cli.ts get setup.md
bun src/cli.ts get /absolute/path/to/docs/setup.md:25 --limit 80
bun src/cli.ts get '#<chunk-or-document-id-prefix>'

# Retrieve several indexed documents
bun src/cli.ts multi-get "guides/*.md" --json

# Show index statistics
bun src/cli.ts stats
bun src/cli.ts doctor

# Clear index artifacts (vannevar.db, WAL/SHM, vectors/)
bun src/cli.ts clear

Options

| Option | Description | Default | |--------|-------------|---------| | --db <path> | Database directory | .vannevar/ | | --chunk-size <n> | Max chunk size in chars | 3600 | | --batch-size <n> | Embeddings per inference batch; tune to available memory | 128 llama, 32 ONNX | | --overlap <n> | Overlap between chunks in chars | 540 | | --ext <exts> | File extensions (comma-separated) | .md,.mdx,.markdown | | --model <profile> | embeddinggemma (768d), embeddinggemma-512, embeddinggemma-256, embeddinggemma-128, nomic, or qwen3 | embeddinggemma | | --embedding-backend <name> | auto, onnx, or llama; recorded per index | auto | | --limit <n> | Max search results | 10 | | --mode <mode> | Search mode: fts, vector, hybrid | hybrid | | --min-score <n> | Minimum score from 0 to 1 | 0 | | --collection <name[,name]> | Restrict search; selects one collection while indexing | | | --candidate-limit <n> | Candidates fused before reranking | 40 | | --fetch-limit <n> | Candidates fetched from each retrieval pass | 50 | | --rerank-limit <n> | Maximum fused candidates scored by MiniLM | 8 | | --reranker-backend <name> | onnx or explicit experimental coreml | onnx | | --reranker-model <id> | Hugging Face ONNX sequence-classifier ID | MiniLM-L6 | | --reranker-dtype <name> | q4, q8, or fp32 | q8 | | --reranker-max-length <n> | Cross-encoder token budget | 128 | | --rerank-blend <name> | rank-protected, linear, or reranker-only | rank-protected | | --rerank-blend-alpha <n> | Cross-encoder weight for linear blending | 0.5 | | --fusion-model <path> | Experimental versioned learned-fusion model JSON | | | --force-rerank | Disable confidence routing and score the full rerank window | | | --max-per-document <n> | Chunks allowed from one document | 1 | | --context-radius <n> | Attach 0-5 neighboring chunks without changing rank | 0 | | --prf | Experimental one-round conservative pseudo-relevance feedback | off | | --fast / --no-rerank | Skip local reranking | | | --rerank | Enable local reranking for vector search | | | --expand / --no-expand | Enable or disable automatic local expansion | plain hybrid queries expand | | --explain | Include RRF and reranker trace in JSON output | | | --pattern <glob> | Relative scan glob for index/collection | collection pattern | | --checkpoint-chunks <n> | Flush vectors and commit completed files during indexing | 5000 | | --watch | Rescan a directory after the initial index | | | --watch-interval <ms> | Poll interval (minimum 250 ms) | 2000 | | --json | Output results as JSON | | | --files | Output compact file references | | | --md | Output Markdown | | | --csv | Output CSV | | | --xml | Output XML | | | --line-numbers | Add line numbers to returned chunk text | |

Directory indexes store each document as root path + relative path, so two indexed directories can both contain README.md without colliding. get accepts a unique relative path, an absolute path, or a # document/chunk ID prefix.

Retrieval benchmarks

Keep hand-curated relevance cases in a versioned JSON fixture. Each case has a query (plain or structured) and the expected indexed relative path(s):

{
  "version": 1,
  "cases": [
    {
      "name": "find the retrieval entry point",
      "query": "intent: Find hybrid retrieval\nlex: hybrid search\nvec: RRF fusion implementation",
      "expected": ["src/hybrid-search.ts"],
      "collections": ["source"]
    }
  ]
}

Run it against an existing index, using --fast when comparing retrieval alone:

bun run bench --fixture benchmarks/example.json --db .vannevar --fast

The JSON report includes per-case rank, reciprocal rank, MRR, and recall@limit.

Graded retrieval evaluation

The corpus-neutral v2 evaluator supports graded qrels, dev/validation/holdout families, intent and difficulty slices, randomized repeated trials, stage and routing telemetry, paired significance tests, and machine-readable gates:

bun run eval:retrieval --fixture benchmarks/retrieval-comparison-v2.example.json \
  --db .vannevar --collection docs --corpus-root ./docs \
  --repetitions 5 --output retrieval.json --markdown retrieval.md

bun run eval:retrieval:compare --baseline baseline.json --candidate retrieval.json \
  --config full --max-quality-loss 0.005 --min-p50-improvement 0.25

Use eval:retrieval:pool to combine top results from several reports into a blind human-judgment file and a separate identity key. This makes engine and configuration identity unavailable to the grader:

bun run eval:retrieval:pool --fixture fixture.json --corpus-root ./docs \
  --report baseline.json --report candidate.json --depth 20 \
  --output judgments.json --key judgments-key.json

eval:retrieval:adjudicate merges blinded judge files into a provenance-bearing fixture. eval:reranker:freeze stores identical candidate pools; eval:reranker compares ONNX models, token budgets, and blend strategies over those pools. eval:reranker:public runs compact rerankers over a BEIR dataset such as SciFact to guard against selecting a private-fixture-only winner. eval:fts-weights and eval:fusion:train keep development-only tuning separate from validation and holdout reporting.

For cold indexing, the automated harness runs Vannevar and QMD sequentially from one corpus path and records the snapshot, wall time, peak RSS, index bytes, health, and no-change rescans. A QMD process can exit successfully at its session limit with embeddings still pending, so the harness checks status, resumes up to three times, and fails the report if either index remains incomplete. It also fails on corpus mutation. The harness keeps every generated index under one reported temporary artifact directory for inspection:

bun run eval:indexing --corpus-root ./docs --repetitions 3 \
  --embedding-backend llama --output indexing.json

Compare against qmd

benchmarks/qmd-comparison.example.json uses qmd's native fixture schema, so the same human-judged cases run through both engines' BM25, vector, hybrid, and full (reranked) backends. It reports qmd-compatible precision, recall@1/3/5, MRR, binary nDCG@10, cold-inclusive and steady-state latency, ranked paths, Vannevar-minus-qmd deltas, and paired bootstrap 95% intervals.

bun run eval:qmd --fixture benchmarks/qmd-comparison.example.json \
  --db .vannevar --collection my-vault --qmd-collection my-vault \
  --gate-backend hybrid --gate-split holdout --output comparison.json

The fixture's expected paths are human relevance judgments, not search output. They must use qmd's virtual path form: each path segment has whitespace and punctuation replaced by hyphens (for example, notes/Current Plan.md becomes notes/Current-Plan.md). The runner maps Vannevar's original filesystem paths to that form before scoring. For a fair run, index the same source snapshot in both engines, use matching collections, and keep the default candidate limit of 40.

For a useful regression set:

• Put a source revision or manifest ID in corpus_snapshot, then rebuild both indexes before an evaluation run. Do not compare a fresh index to a stale one.
• Create queries from actual retrieval tasks and stratify them: exact names/paths, semantic questions, current-status questions, cross-domain questions, and aliases.
• Mark each query "split": "dev" or "split": "holdout". Tune only on development queries and use --gate-split holdout for the claimed result.
• Pool the top 10 paths from every system/configuration, judge the pooled set by hand, and record every relevant path in expected_files. Keep separate development and holdout fixtures so tuning does not leak into the reported score.
• Read the per-query rankings as well as aggregate metrics. Compare no-rerank and reranked modes separately; expansion and reranking can improve recall while harming top-rank precision or tail latency.

The report keeps all observations in latency_ms and excludes each backend's first observation in steady_state_latency_ms. Competitive latency gates use the latter so lazy model initialization is visible without dominating repeat-query performance. Pass --corpus-root <path> to recompute the sorted Markdown path/content hash and fail before inference when the fixture snapshot is stale. A completed report can be checked again without model work:

bun run eval:qmd --report comparison.json --gate-backend semantic --gate-split holdout

# Compare both inference backends on identical prepared document batches
bun run bench:embeddings --batch-size 32 --iterations 3 --text-chars 2000

Architecture

                    ┌──────────────┐
                    │   CLI (cli)  │
                    └──────┬───────┘
                           │
                    ┌──────┴───────┐
                    │   Indexer    │
                    └──┬───────┬──┘
                       │       │
              ┌────────┴─┐   ┌┴──────────┐
              │ Chunker  │   │ Embeddings │
              └────────┬─┘   └┬──────────┘
                       │      │
              ┌────────┴──────┴──────────┐
              │      Hybrid Search       │
              │  (RRF Fusion, k=60)      │
              └────┬──────────────┬──────┘
                   │              │
           ┌───────┴───┐   ┌─────┴──────┐
           │   Store   │   │VectorStore │
           │(SQLite+FTS│   │ (verso-db) │
           │  Drizzle) │   │   HNSW     │
           └───────────┘   └────────────┘

Components

| Module | Description | |--------|-------------| | src/cli.ts | CLI entry point with command routing | | src/indexer.ts | Two-phase indexing pipeline with generic frontmatter context, cross-document embedding batches, and polling watch mode | | src/chunker.ts | Markdown-aware chunking with heading hierarchy, code/table preservation, overlap | | src/embeddings.ts | Portable ONNX embeddings via @huggingface/transformers | | src/llama-embeddings.ts | Native GGUF embeddings with parallel llama.cpp contexts and Metal auto-detection | | src/embedding-runtime.ts | Persisted-backend dispatch shared by indexing and search | | src/hybrid-search.ts | Typed query variants, static or learned fusion, confidence routing, optional PRF, diversity, and reranking | | src/query-expansion.ts | Deterministic local lexical, semantic, and HyDE query variants | | src/benchmark.ts | Versioned relevance fixture runner and MRR/recall scoring | | src/store.ts | SQLite store with Drizzle ORM + FTS5 virtual table | | src/vector-store.ts | verso-db HNSW vector index wrapper | | src/schema.ts | Drizzle ORM schema definitions | | src/types.ts | Shared TypeScript interfaces |

How hybrid search works

1. FTS5 runs a strict literal AND pass, then a safe content-word OR fallback when needed
2. Vector search embeds the query with the same model and finds nearest neighbors via HNSW cosine similarity
3. RRF fusion merges both ranked lists: score = weight / (k + rank) with k=60, deduplicating chunks that appear in both
4. MiniLM optionally reranks a bounded top window while preserving the retrieval tail
5. Results are normalized so the top result has score 1.0

Development

# Run tests
bun run test

# Run all tests (including embedding tests — slower, downloads model)
bun run test:all

# Run with coverage
bun run test:coverage

# Lint
bun run lint

# Type check
bun run typecheck

Tech stack

• Runtime: Bun
• Database: SQLite via bun:sqlite + Drizzle ORM
• Full-text search: SQLite FTS5 with porter stemming
• Vector index: verso-db (HNSW)
• Embeddings: EmbeddingGemma GGUF through llama.cpp/Metal, plus EmbeddingGemma, Nomic, and Qwen3 ONNX profiles via @huggingface/transformers
• Linter: Biome

License

MIT