ace-indexing

Intelligent project analysis and indexing system for AI agents. Fast incremental indexing with shard-based storage, LRU caching, and comprehensive code analysis.

Features

📦 Checkpoint System

• Shard-based storage - Efficient data partitioning for scalability
• LRU caching - Fast access to frequently used checkpoints with TTL support
• Incremental updates - Only save changes, not the entire state
• Throttled flushing - Batched writes for better performance (default: 5000ms)
• On-disk change detection - Detect external file modifications
• Manifest tracking - Version management and metadata persistence

🔍 Project Analyzer

• File summaries - Exports, imports, keywords, descriptions for each file
• Symbol indexing - Function, class, interface, type definitions with locations
• Dependency graph - Import/importedBy relationships between files
• Smart search - Intelligent search based on summaries and symbols
• Multi-language support - TypeScript, JavaScript, Python, Go, Rust, and more

🛠 Developer Experience

• Full TypeScript support - Strict mode with complete type definitions
• CLI tool - Command-line interface for project analysis
• ESM native - Modern ES module support

Requirements

• Node.js >= 22.0.0
• Package Manager: npm, pnpm, yarn, or bun

Installation

# Using bun (recommended)
bun add ace-indexing

# Using npm
npm install ace-indexing

# Using pnpm
pnpm add ace-indexing

# Using yarn
yarn add ace-indexing

Global CLI Installation

# Install globally to use the `ace` command
npm install -g ace-indexing

Interactive Setup (Recommended)

# Interactive wizard - select tools and options
ace init

# Initialize for a specific project (adds AGENTS.md workflow)
ace init /path/to/project

# Non-interactive with defaults
ace init --yes

# Non-interactive with specific options
ace init . --tools claude,codex --link --yes

What ace init Does

1. Installs Skill - Copies skill to ~/.claude/skills (and/or ~/.codex/skills)
2. Configures AGENTS.md - Adds workflow instructions so AI automatically uses ace
3. Sets up Config - Adds ace-indexing configuration block to AGENTS.md

Customize what gets indexed

# Only index TS/JS, skip dist & coverage, ignore *.snap, cap file size at 512KB
ace analyze . \
  --ext .ts,.tsx,.js,.jsx \
  --ignore-dir dist,coverage \
  --ignore-pattern *.snap \
  --max-size 524288

# Skip symbol/dependency extraction (faster, summaries only)
ace analyze . --no-symbols --no-deps

Or define defaults in your project's AGENTS.md so every agent run uses them automatically:

```ace-indexing
{
  "extensions": [".ts", ".tsx", ".js", ".jsx"],
  "ignoreDirs": ["dist", "coverage"],
  "ignorePatterns": ["*.snap"],
  "maxFileSize": 524288,
  "extractSymbols": true,
  "buildDependencies": true
}
```

Quick Start

import {
  createIndexingSystem,
  createFilePath,
  createDocument,
} from 'ace-indexing';

// Create indexing system
const { manager, initialize, dispose } = createIndexingSystem({
  baseDir: '~/.my-app/indexing', // Optional: custom storage location
  flushIntervalMs: 5000, // Optional: flush interval (default: 5000ms)
  maxCachedShards: 10, // Optional: max cached shards (default: 10)
});

// Initialize
await initialize();

// Create a file path
const filePath = createFilePath({
  rootPath: '/home/user/project', // Optional
  relPath: 'src/index.ts',
});

// Add a checkpoint
await manager.addCheckpoint(
  { conversationId: 'conv-1', path: filePath },
  {
    sourceToolCallRequestId: 'tool-1',
    timestamp: Date.now(),
    conversationId: 'conv-1',
    document: createDocument(filePath, 'original code', 'modified code'),
  }
);

// Get latest checkpoint
const checkpoint = await manager.getLatestCheckpoint({
  conversationId: 'conv-1',
  path: filePath,
});

// Get all checkpoints
const checkpoints = await manager.getCheckpoints({
  conversationId: 'conv-1',
  path: filePath,
});

// Clean up when done
await dispose();

API

createIndexingSystem(options)

Creates an indexing system with the specified options.

interface IndexingSystemOptions {
  baseDir?: string; // Storage directory (default: ~/.ace-indexing)
  flushIntervalMs?: number; // Flush interval in ms (default: 5000)
  maxCachedShards?: number; // Max cached shards (default: 10)
  logger?: Logger; // Custom logger
  clientWorkspaces?: ClientWorkspaces; // For on-disk change detection
}

Returns:

• manager: ShardManager instance
• storage: FileShardStorage instance
• initialize(): Initialize the system
• dispose(): Clean up resources

ShardManager

Main class for managing checkpoints.

// Add a checkpoint
await manager.addCheckpoint(keyData, checkpoint);

// Get latest checkpoint
const checkpoint = await manager.getLatestCheckpoint(keyData);

// Get all checkpoints
const checkpoints = await manager.getCheckpoints(keyData);

// Get checkpoints with filter
const filtered = await manager.getCheckpoints(keyData, {
  afterTimestamp: 1234567890,
});

// Check if key exists
const exists = await manager.hasKey(keyData);

// Update a checkpoint
await manager.updateCheckpoint(keyData, checkpoint);

// Remove checkpoints
await manager.removeCheckpoint(keyData);
await manager.removeCheckpoint(keyData, { sourceToolCallRequestId: 'tool-1' });

// Force flush to storage
await manager.flush();

// Clear all data
await manager.clear();

LRUCache

LRU cache with TTL support.

import { LRUCache } from 'ace-indexing';

const cache = new LRUCache<string, object>({
  max: 1000, // Max entries
  ttl: 1000 * 60 * 5, // 5 minutes
  updateAgeOnGet: true, // Update timestamp on access
});

cache.set('key', { data: 'value' });
const value = cache.get('key');
cache.delete('key');
cache.clear();

Storage Backends

import { FileShardStorage, MemoryShardStorage } from 'ace-indexing/storage';

// File-based storage (default)
const fileStorage = new FileShardStorage('~/.my-app/indexing');

// In-memory storage (for testing)
const memoryStorage = new MemoryShardStorage();

Project Analyzer

The ProjectAnalyzer provides intelligent code analysis capabilities.

Basic Usage

import { ProjectAnalyzer } from 'ace-indexing';

const analyzer = new ProjectAnalyzer({
  rootDir: '/path/to/project',
  // Optional configuration
  ignoreDirs: ['node_modules', '.git', 'dist'],
  extensions: ['.ts', '.js', '.py'],
  maxFileSize: 1024 * 1024, // 1MB
});

const analysis = await analyzer.analyze();

// Access results
console.log(analysis.structure);     // Project structure stats
console.log(analysis.summaries);     // File summaries
console.log(analysis.symbols);       // Symbol index
console.log(analysis.dependencies);  // Dependency graph

Search

// Search for files matching a query
const results = await analyzer.search('authentication', {
  limit: 10,
  type: 'function', // Optional: filter by symbol type
});

// Results include file path, match score, and reason
results.forEach(r => console.log(`${r.file}: ${r.score} - ${r.reason}`));

Symbol Lookup

// Find symbol definitions
const symbols = await analyzer.findSymbol('UserService');

// Returns symbol info with location
symbols.forEach(s => {
  console.log(`${s.name} (${s.type}) - ${s.file}:${s.line}`);
});

Dependency Analysis

// Get dependencies for a file
const deps = await analyzer.getDependencies('src/services/auth.ts');

// deps.imports - files this file imports
// deps.importedBy - files that import this file
// deps.impactAnalysis - what files would be affected by changes

CLI Commands

# Analyze a project
ace analyze ./my-project

# Search for code
ace search ./my-project "auth" --limit 10

# Get file summaries (compact format for AI agents)
ace summaries ./my-project --compact

# Find symbol definitions
ace symbol ./my-project UserService

# Analyze dependencies
ace deps ./my-project src/index.ts

Supported Languages

| Language | Symbol Extraction | Dependency Analysis | |----------|-------------------|---------------------| | TypeScript/JavaScript | Full | Full (import/require) | | Python | Full | Full (import/from) | | Go | Full | Basic | | Rust | Full | Basic | | Java | Basic | - | | C#/C/C++ | Basic | - |

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Public API                                │
│  createIndexingSystem() | ProjectAnalyzer                    │
└──────────────────────────┬──────────────────────────────────┘
                           │
           ┌───────────────┴───────────────┐
           ▼                               ▼
┌─────────────────────────┐     ┌─────────────────────────────┐
│     ShardManager        │     │     ProjectAnalyzer          │
│  - Shard routing        │     │  - File scanning             │
│  - LRU caching          │     │  - Symbol extraction         │
│  - Throttled flushing   │     │  - Dependency analysis       │
│  - Manifest management  │     │  - Smart search              │
└───────────┬─────────────┘     └─────────────────────────────┘
            │
            ▼
┌─────────────────────────┐
│       ShardData         │
│  - Checkpoint storage   │
│  - Dirty tracking       │
│  - (De)Serialization    │
└───────────┬─────────────┘
            │
            ▼
┌─────────────────────────────────────────────────────────────┐
│                    ShardStorage                              │
│  FileShardStorage (file system) | MemoryShardStorage (test)  │
└─────────────────────────────────────────────────────────────┘

Core Modules

| Module | Responsibility | |--------|---------------| | ShardManager | Shard routing, caching, throttled flushing, manifest tracking | | ShardData | Single shard data structure, dirty tracking, serialization | | LRUCache | LRU cache with TTL support for checkpoints and documents | | ProjectAnalyzer | Code analysis, symbol indexing, dependency graph | | extractors | Multi-language symbol and dependency extraction |

Storage Format

~/.ace-indexing/
├── manifest.json           # Shard metadata
├── shards/
│   ├── conversation-xxx.json
│   └── conversation-yyy.json
└── assets/
    └── checkpoint-documents/
        └── <conversation-id>/
            └── document-<path>-<timestamp>-<id>.json

Development

# Clone the repository
git clone https://github.com/kingsword09/ace-indexing.git
cd ace-indexing

# Install dependencies
bun install

# Build
bun run build

# Run tests
bun run test

# Type check
bun run typecheck

# Lint
bun run lint

# Format code
bun run format

Available Scripts

| Script | Description | |--------|-------------| | build | Build ESM + TypeScript declarations | | dev | Development mode with file watching | | typecheck | TypeScript type checking | | test | Run tests in watch mode | | test:run | Run tests once | | test:coverage | Generate coverage report | | lint | Run oxlint with type awareness | | lint:fix | Auto-fix lint issues | | format | Format code with oxfmt | | format:check | Check code formatting |

License

Apache-2.0