md-search

Semantic search for markdown files. Creates a portable, git-committable vector index using an integrated local model by default or any OpenAI-compatible embedding API.

Why? Find relevant content in documentation, notes, or knowledge bases without exact keyword matching. Results work for both humans (readable output) and AI agents (JSON/JSONL).

Installation

Requires Bun runtime. Local embeddings and reranking run in-process with ONNX Runtime; no Ollama, LM Studio, Python, or separately installed model server is required.

# Run directly (no install)
bunx md-search search "your query"

# Install globally
bun add -g md-search

Bun may report blocked lifecycle scripts for transitive ONNX dependencies during installation. They are not required for md-search's supported CPU local-embedding path; no --trust step is needed for normal use.

Quick Start

# Index and search - no config file or API key needed
# First use downloads the pinned ~24 MB local model to ~/.cache/md-search/models.
md-search index ./docs
md-search search "how to configure logging"

Project-based setup with config file:

Create .md-search.yaml:

api:
  provider: openai
  base_url: http://localhost:11434/v1  # Ollama, or any OpenAI-compatible API
  model: nomic-embed-text
  dimensions: 768
sources:
  docs: ./docs

For a remote/OpenAI-compatible provider, configure it explicitly (this keeps existing remote setups supported):

api:
  provider: openai
  base_url: https://api.openai.com/v1
  model: text-embedding-3-small
  dimensions: 1536

Then set the API key in .md-search.local.yaml (gitignored):

api:
  api_key: sk-...

See CONFIGURATION.md for complete configuration reference including local model caching/offline use, environment variables, credential precedence, and provider-specific setup.

Usage

Index:

md-search index              # Index all sources
md-search index ./docs       # Index specific directory
md-search index --force      # Re-index everything

Search:

md-search search "how to configure logging"
md-search search -k 5 "authentication flow"    # Top 5 results
md-search search -k 50 -l 10 "database queries" # Search 50, return 10
md-search search -o json "api endpoints"       # JSON for agents
md-search search --rerank "complex question"   # Local cross-encoder reranking by default

Search and list filters accept one value per option and may be repeated:

md-search search --source docs --source notes --tag api "authentication"
md-search list --filter "status=published" --filter "type=guide"

For compatibility with compact multi-value usage, sources and tags also accept comma-delimited values:

md-search search --source docs,notes --tag api,reference "authentication"

Whitespace-separated values such as --source docs notes are no longer accepted. Use repeated options or a comma-delimited source/tag value so the required search query is never interpreted as a filter value. Frontmatter filters must be repeated; commas inside a filter value are preserved literally.

Output Formats

Text (default):

docs:guides/auth.md:45-62 0.847 # Authentication
Preview of matching content...

JSON: Full result objects for programmatic use (-o json)

JSONL: One result per line for piping (-o jsonl)

Advanced Configuration

Integrated local embeddings (default): no configuration is needed. The immutable Xenova/all-MiniLM-L6-v2@751bff37182d3f1213fa05d7196b954e230abad9 q8, 384D profile downloads once to $XDG_CACHE_HOME/md-search/models (or ~/.cache/md-search/models). It was accepted against the declared SciFact comparison thresholds: nDCG@10 0.6520 (91.84% of the compact-candidate best), Recall@10 0.7947 (95.29%), 65.81 warm CPU inputs/s, and a 23.69 MB cache. Those results cover whole scientific documents, not md-search Markdown chunks. To use an existing cache offline:

api:
  local:
    allow_download: false

The index records its provider/model/revision identity once in SQLite metadata. A legacy or different embedding space must be rebuilt with md-search index --force before new vectors are written.

Integrated local reranking: with the default/local embedding setup, --rerank needs no service or API key. md-search downloads the pinned q8 CPU cross-encoder Xenova/ms-marco-MiniLM-L-6-v2@a09144355adeed5f58c8ed011d209bf8ee5a1fec (about 23 MiB) to the same cache root. It scores one query/passage pair at a time, truncating combined input to 512 tokens, and is English-focused. Existing configurations using remote embeddings continue to inherit the remote reranker unless reranker.provider: local is explicit. The upstream cross-encoder model is Apache-2.0; the Transformers.js conversion supplies the ONNX weights. To use an existing cache offline:

reranker:
  provider: local
  local:
    # cache_dir: ~/.cache/md-search/models
    allow_download: false

Remote reranking remains supported for OpenAI-compatible APIs, LM Studio, Ollama-compatible servers, and Azure:

# .md-search.local.yaml
reranker:
  provider: openai
  base_url: https://api.openai.com/v1
  model: gpt-4o-mini
  api_key: sk-...

Ignore Patterns (exclude files from indexing):

# .md-search.yaml
ignore_patterns:
  - "_*"      # Drafts (default)
  - ".*"      # Hidden files (default)
  - "archive/*"

Azure OpenAI:

# .md-search.yaml
api:
  provider: azure
  base_url: https://my-resource.openai.azure.com/openai
  model: text-embedding-ada-002
  azure_deployment: text-embedding-ada-002
  azure_api_version: "2024-02-01"

Migrating from 0.1.x

A fresh installation defaults to the integrated local MiniLM model. Existing configs that explicitly set remote API settings retain OpenAI-compatible behavior. Rebuild a legacy or changed embedding index with md-search index --force.

Version 0.2.0 renames CLI flags for consistency:

| Old Flag | New Flag | Commands | |----------|----------|----------| | -f, --format | -o, --output | search, links, debug, status | | -F, --filter | -f, --filter | search |

The -f flag on the index command (for --force) is unchanged.