codebase-indexer

    /\_/\   codebase-indexer
   ( •.•)  ─────────────────
 ⊂/ 🌰 \⊃  Semantic code search
  /|   |\   powered by AI

A semantic codebase indexer that chunks your source code, generates embeddings via Ollama, and stores them in Qdrant for natural-language code search. Ships as an MCP server for Claude Code and Codex integration, and also works as a standalone CLI.

Features

• Semantic search — find code by meaning, not just keywords ("retry logic with exponential backoff")
• 18 languages with symbol-aware chunking (functions, classes, interfaces, etc.)
• 70+ file extensions recognized
• Incremental indexing — only re-indexes changed files (MD5 content hashing)
• MCP server — plug directly into Claude Code or Codex as a tool provider
• File watcher — auto-reindex on save (500ms debounce)
• Interactive CLI — squirrel mascot, spinners, colored output
• Zero config — sensible defaults, works out of the box with npx

Quick Start

One-Line Install

The fastest way to get started — checks prerequisites, sets up infrastructure, indexes your project, and configures integrations interactively:

bash <(curl -fsSL https://raw.githubusercontent.com/ygtdgn/codebase-indexer/main/install.sh)

Or equivalently:

curl -fsSL https://raw.githubusercontent.com/ygtdgn/codebase-indexer/main/install.sh | bash

Manual Setup

Prerequisites

• Node.js >= 18
• Docker (for Qdrant)
• Ollama running locally

1. Set up infrastructure

npx codebase-indexer init

This starts a Qdrant container via Docker Compose, verifies your Ollama connection, and pulls the embedding model if needed.

2. Index your project

npx codebase-indexer index ./your-project

3. Search

npx codebase-indexer search "authentication middleware"

4. Use with Claude Code or Codex

# Set up Claude Code integration (writes CLAUDE.md + .mcp.json)
npx codebase-indexer index ./your-project --setup-claude

# Set up Codex integration (writes AGENTS.md + .codex/config.toml)
npx codebase-indexer index ./your-project --setup-codex

# Set up both interactively
npx codebase-indexer index ./your-project --setup

# Global MCP setup (user-level config files)
npx codebase-indexer index ./your-project --setup --setup-globally

CLI Reference

Global Options

| Option | Default | Description | |--------|---------|-------------| | --ollama-url <url> | http://localhost:11434 | Ollama API URL | | --qdrant-url <url> | http://localhost:6333 | Qdrant API URL | | --model <name> | qwen3-embedding:0.6b | Embedding model name | | --dim <number> | 512 | Embedding vector dimension | | --dir <path> | . | Directory to index/watch | | --collection <name> | codebase | Qdrant collection name | | --no-watch | — | Disable file watching in MCP mode |

Commands

init

Set up Qdrant (Docker) and check Ollama connection.

codebase-indexer init

• Creates ~/.codebase-indexer/docker-compose.yml
• Starts Qdrant container (qdrant/qdrant:latest)
• Waits for health check (30s timeout)
• Verifies Ollama and auto-pulls the embedding model

index [directory]

Index a directory for semantic search.

codebase-indexer index ./my-project [options]

| Option | Description | |--------|-------------| | --setup | Interactively choose setup targets (Claude/Codex) | | --setup-claude | Write CLAUDE.md + .mcp.json for Claude Code | | --setup-codex | Write AGENTS.md + .codex/config.toml for Codex | | --setup-globally | Write MCP config to user-level files instead of project-local | | --force | Re-index all files, ignoring cached hashes |

search <query>

Semantic search across the indexed codebase.

codebase-indexer search "database connection pooling" -k 5
codebase-indexer search "error handling" -l typescript

| Option | Default | Description | |--------|---------|-------------| | -k, --top-k <number> | 10 | Number of results | | -l, --language <lang> | — | Filter by language |

status

Check health of Ollama and Qdrant, show index statistics.

codebase-indexer status

config

Interactively edit persistent settings.

codebase-indexer config

Opens an interactive menu to edit Ollama URL, Qdrant URL, embedding model, dimension, and collection name. Settings are saved to ~/.codebase-indexer/config.json and loaded by all commands automatically.

Default (no subcommand)

Start the MCP server over stdio.

codebase-indexer --dir ./my-project

MCP Server

When run without a subcommand (or via an MCP config), codebase-indexer starts as an MCP server using stdio JSON-RPC transport. It exposes five tools:

| Tool | Description | |------|-------------| | search_code | Semantic search with optional language and file_path_prefix filters | | index_file | Index or re-index a single file | | index_directory | Incrementally index an entire directory | | get_index_status | Health check and index statistics | | delete_file | Remove a file from the index |

Example .mcp.json

{
  "mcpServers": {
    "codebase-indexer": {
      "command": "npx",
      "args": ["codebase-indexer", "--dir", "/absolute/path/to/project"]
    }
  }
}

Example usage in Claude Code

search_code({ query: "retry logic with exponential backoff", top_k: 5 })
search_code({ query: "error handling", language: "typescript", file_path_prefix: "src/api/" })
index_file({ path: "src/new-module.ts" })
index_directory({})
get_index_status({})
delete_file({ path: "src/old-module.ts" })

Configuration

Settings are resolved in this order (last wins):

Hardcoded defaults → ~/.codebase-indexer/config.json → Environment variables → CLI flags

Environment Variables

| Variable | Maps to | |----------|---------| | OLLAMA_URL | --ollama-url | | QDRANT_URL | --qdrant-url | | EMBEDDING_MODEL | --model | | EMBEDDING_DIM | --dim | | COLLECTION_NAME | --collection |

Persistent Config

Run codebase-indexer config to interactively set values, or manually create ~/.codebase-indexer/config.json:

{
  "ollamaUrl": "http://localhost:11434",
  "qdrantUrl": "http://localhost:6333",
  "model": "qwen3-embedding:0.6b",
  "embeddingDim": 512,
  "collectionName": "codebase"
}

Collection Name Auto-Derivation

When using the default collection name (codebase) and indexing a specific directory, the collection name is automatically derived from the directory name:

codebase-indexer index ./my-cool-project
# → collection: "codebase-my-cool-project"

Architecture

index.ts (CLI entry, commander.js)
  → cli/commands.ts (command handlers)
    → core/indexer.ts (orchestrator)
      → core/chunker.ts (symbol-based splitting, sliding window fallback)
      → core/embedder.ts (Ollama /api/embed client, MRL truncation + L2 normalize)
      → core/vectorstore.ts (Qdrant client, cosine similarity search)
  → mcp/server.ts (MCP stdio transport, 5 tools)
  → watcher/watcher.ts (chokidar, 500ms debounce, feeds into indexer)

Pipeline

Discover files → Chunk code → Embed via Ollama → Store in Qdrant

1. File discovery — uses git ls-files (fast, respects .gitignore) with glob fallback. Filters by 54 code extensions, skips lock files, respects size limits (1 MB default).

2. Chunking — dual strategy per file:

   • Symbol-based: regex patterns detect functions, classes, interfaces, etc. for 18 languages
   • Sliding window fallback: used when symbols cover <50% of the file. Default 1500 chars with 200-char overlap.

3. Embedding — batches of 16 chunks sent to Ollama's /api/embed endpoint. Vectors are MRL-truncated to the target dimension and L2-normalized. Retry with exponential backoff (3 attempts).

4. Storage — chunks upserted to Qdrant with deterministic IDs (MD5(path:startLine:endLine)). Payload indices on file_path, language, and chunk_type for filtered search. Cosine similarity.

5. Incremental indexing — each file's content hash is stored in the Qdrant payload. On re-index, unchanged files are skipped entirely.

Supported Languages (Symbol Detection)

| Language | Detected Symbols | |----------|-----------------| | TypeScript | functions, classes, interfaces, types, enums, arrow functions | | JavaScript | functions, classes, arrow functions, module.exports | | Python | functions, classes | | Go | functions, types (struct, interface) | | Rust | functions, structs, enums, traits, impl blocks | | Java | classes, interfaces, methods | | Kotlin | functions, classes, interfaces, objects | | Ruby | functions, classes, modules | | PHP | functions, classes, interfaces, traits | | Swift | functions, classes, structs, protocols, enums | | C# | methods, classes, interfaces | | Scala | functions, classes, traits, objects | | C | functions, structs, typedefs | | C++ | functions, classes, structs, namespaces, templates | | Elixir | functions, private functions, modules | | Haskell | type signatures, data types, classes, instances | | Dart | functions, classes, mixins | | Zig | functions, const structs |

All other recognized file types fall back to sliding window chunking.

File Watcher

In MCP mode, the file watcher is enabled by default:

• Watches for file add, change, and unlink events
• 500ms debounce before processing
• 300ms write-finish stability detection
• Batch processing with retry (max 3 attempts, exponential backoff)
• Automatically re-indexes modified files and removes deleted ones

Disable with --no-watch.

Development

git clone https://github.com/ygtdgn/codebase-indexer.git
cd codebase-indexer
npm install
npm run dev     # tsx watch mode (auto-rebuild on save)

Build

npm run build   # tsc → dist/

Test

npm test            # run once (vitest)
npm run test:watch  # watch mode

Test coverage includes chunker (symbol detection, sliding window), embedder (truncation, normalization), file utilities (extension mapping, discovery), and hash functions (MD5, chunk IDs).

Project Structure

src/
├── index.ts              # CLI entry point (commander.js)
├── config/
│   └── config.ts         # Config interface, defaults, env vars, persistent config
├── cli/
│   ├── commands.ts       # Command handlers (init, index, search, status, config)
│   ├── mascot.ts         # Squirrel ASCII art with gradient colors
│   └── ui.ts             # Spinners, progress formatting, result display
├── core/
│   ├── indexer.ts        # Central orchestrator
│   ├── chunker.ts        # Symbol-based + sliding window chunking
│   ├── embedder.ts       # Ollama embedding client
│   └── vectorstore.ts    # Qdrant CRUD operations
├── mcp/
│   └── server.ts         # MCP stdio server (5 tools)
├── watcher/
│   └── watcher.ts        # File change detection + auto-reindex
├── utils/
│   ├── files.ts          # File discovery, language detection
│   ├── hash.ts           # MD5, chunk IDs, index hashing
│   └── logger.ts         # stderr logging (warn, error)
└── __tests__/
    ├── chunker.test.ts
    ├── embedder.test.ts
    ├── files.test.ts
    └── hash.test.ts

License

MIT