RepoRAG

Semantic code search CLI for Jira tasks. Indexes your repository by AST blocks (functions, classes, interfaces), finds relevant code via vector search + cross-encoder reranking, and outputs a repomix-compatible XML with only the files & blocks you need to feed into Kimi / Claude / GPT.

Features

• Lazy repomix integration — generates repomix-output.xml only if missing or stale
• AST-level indexing — parses TypeScript/JavaScript into functions, classes, interfaces, variables
• Smart chunking — large functions (> 6000 chars) are split by control flow (if/for/while/switch/try)
• Cross-file dependency graph — resolves import { foo } from './bar' and call references to pull in related files automatically
• Cached embeddings — SHA256-based cache in ./cache/; re-indexing only computes new/changed blocks
• Cross-encoder reranking — ms-marco-MiniLM-L-6-v2 re-ranks top vector hits for better precision
• Vector DB — LanceDB (local, serverless) with IVF-PQ indexes
• Repomix XML output — standard format that Kimi/Claude understand natively

Architecture

Repository
    │
    ▼
repomix API (lazy XML)
    │
    ├── searchFiles(dir)      → filePaths[]
    ├── collectFiles(paths)   → raw files
    ├── processFiles(raw)     → cleaned content
    └── TokenCounter          → token stats
    │
    ▼
TypeScript AST parser
    │
    ├── functions / classes / interfaces / types / enums / variables
    ├── import graph (cross-file deps)
    └── chunking (big functions → sub-blocks)
    │
    ▼
transformers.js (all-MiniLM-L6-v2)
    │
    ├── block embeddings  → LanceDB table "blocks"
    ├── file embeddings   → LanceDB table "files"
    └── SHA256 cache      → ./cache/*.json
    │
    ▼
Query: "add Google OAuth2 authentication"
    │
    ├── vector search (top-40 blocks)
    ├── cross-encoder reranking (top-15)
    ├── import-graph expansion (1-hop related files)
    └── repomix XML assembly

Installation

npm install

Dependencies (auto-installed via npm install):

• @huggingface/transformers
• @lancedb/lancedb
• repomix
• typescript

Usage

1. Index a repository

node rag.js index <directory> [--force]

Examples:

# First time — creates repomix-output.xml + builds LanceDB index
node rag.js index ./my-project

# Force full re-index (regenerate XML + rebuild DB)
node rag.js index ./my-project --force

What happens:

1. Checks if repomix-output.xml exists and is fresh (compares mtime with .git/HEAD). If stale or missing, runs repomix via its programmatic API.
2. Uses searchFiles → collectFiles → processFiles to get clean file contents.
3. Parses every .ts/.tsx/.js/.jsx file into AST blocks.
4. Builds an import/call-reference graph.
5. Computes embeddings (cached by SHA256).
6. Stores everything in ./repo-rag-db/ (LanceDB).

2. Search by Jira task

node rag.js search "<your task description>" [--top-k N] [--files N] [--output path.xml]

Examples:

# Basic search
node rag.js search "add Google OAuth2 authentication with JWT callback"

# Custom output file
node rag.js search "refactor database connection pool" --output jira-442.xml

# More results
node rag.js search "implement WebSocket notifications" --top-k 60 --files 12

What happens:

1. Embeds your query with all-MiniLM-L6-v2.
2. Vector search in LanceDB (blocks + files tables).
3. Reranks top-40 blocks with ms-marco-MiniLM-L-6-v2 cross-encoder.
4. Expands 1-hop via import graph (pulls in type definitions, utilities, etc.).
5. Assembles a repomix-compatible XML with only relevant blocks.

Output Format

The generated XML follows the repomix standard so Kimi, Claude, and other AI assistants parse it natively:

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Task: add Google OAuth2 authentication with JWT callback
  Engine: repomix API → AST → LanceDB → cross-encoder reranking → import graph
-->
<file_summary>
This file contains a semantically retrieved subset of the codebase ...
</file_summary>

<<directory_structure>
  src/auth/providers.ts
  src/utils/jwt.ts
  src/models/user.ts
</directory_structure>

<<files>
<file path="src/auth/providers.ts">
// ===== imports =====
import { OAuth2Client } from 'google-auth-library';
import { User } from '../models/user';
import { signJwt } from '../utils/jwt';

// === function: createGoogleStrategy (combined: 0.891) ===
export function createGoogleStrategy(config: OAuthConfig) {
  return new OAuth2Client(config.clientId, config.clientSecret, config.redirectUri);
}

// ... [340 chars omitted] ...

// === class: GoogleAuthProvider (combined: 0.854) ===
export class GoogleAuthProvider {
  private client: OAuth2Client;
  constructor(config: AuthConfig) { ... }
  async authenticate(code: string): Promise<<AuthResult> { ... }
}
</file>

<file path="src/utils/jwt.ts">
// ===== imports =====
import jwt from 'jsonwebtoken';

// === function: signJwt [RELATED] (dist: 0.234) ===
export function signJwt(payload: object, expiresIn = '7d'): string {
  return jwt.sign(payload, process.env.JWT_SECRET!, { expiresIn });
}
</file>
</files>

Markers explained:

• [RELATED] — file pulled in via import graph (not a direct semantic hit, but required for compilation context)
• [chunk N of M] — a large function was split by control flow
• [... X chars omitted] ... — unrelated code between blocks was truncated

Prompt Template for Kimi

Copy-paste this together with the XML:

Task from Jira: <paste your task here>

Below is a semantically retrieved subset of our codebase. 
Only relevant functions, classes, and their imports are included.
Implement the changes preserving existing style, patterns, and types.
Do not modify unrelated code.

Directory Structure

.
├── rag.js              # Main CLI (index + search)
├── repo-rag-db/        # LanceDB vector database
│   ├── meta.json       # AST metadata + import graph
│   └── ...
├── cache/              # SHA256 → embedding cache
│   ├── a1b2c3d4.json
│   └── ...
└── repomix-output.xml  # Generated lazily by repomix API

Requirements

• Node.js ≥ 18
• RAM — ~300-500 MB for transformer models (first run downloads ~100 MB)
• CPU — inference runs on CPU via ONNX Runtime; WebGPU/CUDA optional for speed

How It Works Under the Hood

| Stage | Tech | What it does | |-------|------|--------------| | File discovery | repomix API (searchFiles) | Respects .gitignore, .ignore, default ignore patterns | | Pre-processing | repomix (processFiles) | Removes comments / empty lines (configurable), counts tokens | | AST parsing | typescript compiler API | Extracts top-level declarations with start/end positions | | Chunking | Custom control-flow splitter | Splits functions > 6000 chars by if/for/while/switch/try | | Embeddings | @huggingface/transformers | Xenova/all-MiniLM-L6-v2 (384-dim) | | Cache | SHA256 JSON files | Avoids recomputing unchanged blocks on re-index | | Vector DB | @lancedb/lancedb | IVF-PQ indexes for fast approximate search | | Reranking | Xenova/ms-marco-MiniLM-L-6-v2 | Cross-encoder scores query↔passage relevance | | Graph | Import + reference resolver | Maps from './foo' and function calls to file dependencies | | Output | Custom XML generator | Repomix-compatible format with truncation markers |

License

MIT