@webneat/codewiki

Core library for parsing, analyzing, and documenting TypeScript codebases with dependency tracking and topological ordering.

Overview

@webneat/codewiki is a TypeScript codebase analysis engine that builds a comprehensive knowledge graph of your code. It extracts symbols, files, directories, and dependencies, then provides topological ordering to enable AI tools to generate contextual documentation with maximum context for each entity.

Features

• 🔍 TypeScript AST Parsing - Extract symbols, imports, exports, and file structure
• 📊 Dependency Analysis - Build complete dependency graphs between files, symbols, and external packages
• 🗄️ SQLite Storage - Fast, portable database with Drizzle ORM for codebase metadata
• 📝 Description Management - Store and manage contextual documentation with smart ordering
• ⚡ Incremental Updates - Git-based change detection for fast re-indexing
• 🎯 Topological Ordering - Intelligent entity ordering to enable optimal AI context
• 🔗 Relationship Tracking - Complete import/export relationships and internal symbol links

Installation

pnpm add @webneat/codewiki

Quick Start

import { create, load } from '@webneat/codewiki'

// Create a new wiki instance
const wiki = await create({
  db_path: './codewiki.db',
  project_path: './my-typescript-project',
  description: 'My awesome TypeScript project'
})

// Or load an existing wiki database
const wiki = await load('./codewiki.db')

// Update the wiki with latest changes
await wiki.update()

// Get counts of entities in different states
const counts = wiki.counts()
console.log(`Ready: ${counts.symbols.ready} symbols, ${counts.files.ready} files`)

// Get next entities to describe (in topological order)
const toDescribe = wiki.to_describe(10)
if (toDescribe.type === 'symbols') {
  console.log(`Next symbols to describe: ${toDescribe.items.map(s => s.name).join(', ')}`)
}

Core API

CodeWiki Interface

The main interface provides access to all entity types:

interface CodeWiki {
  readonly config: Config
  update(): Promise<void>                    // Re-index the project
  counts(): WikiCounts                       // Get entity counts by state
  to_describe(limit: number): EntitiesToDescribe // Get next entities for AI description
  
  files: FilesWiki           // File operations
  symbols: SymbolsWiki       // Symbol/function/class operations  
  directories: DirectoriesWiki // Directory operations
  dependencies: DependenciesWiki // External dependency operations
}

Entity Operations

Each entity type provides a consistent API:

// Search and retrieve
const files = wiki.files.search({ where: { state: 'ready' }, limit: 10 })
const file = wiki.files.get(fileId)

// Set AI-generated descriptions
wiki.files.set_description(fileId, 'This file contains utility functions...')

// Query relationships
const importedFiles = wiki.files.imported_by_file(fileId)
const usingSymbols = wiki.symbols.using_dependency(dependencyId)

Entity Types

Files

• States: to_scan → to_describe → ready
• Tracks: File metadata, imports, exports, symbols
• Relationships: Import relationships with other files and external dependencies

Symbols

• States: to_link → to_describe → ready
• Tracks: Functions, classes, variables, types
• Relationships: Internal symbol references and external dependency usage

Directories

• States: to_describe → ready
• Tracks: Directory structure and contents
• Relationships: Parent-child relationships and contained files

External Dependencies

• States: to_describe → ready
• Tracks: npm packages, built-in modules
• Relationships: Usage by files and symbols

Topological Description Order

The to_describe() method returns entities in an intelligent order that maximizes AI context:

1. External Dependencies (no dependencies)
   • Built-in packages first, then by usage count
2. Symbols (depend on other symbols and dependencies)
   • Fewest missing dependencies first, then by complexity
3. Files (contain symbols and have imports)
   • Fewest missing descriptions first, then by file size
4. Directories (contain files and subdirectories)
   • Fewest missing content first, then by total size

This ensures that when describing an entity, all its dependencies are already described.

CLI Usage

The package includes a comprehensive CLI:

# Initialize and index a project
codewiki update

# Show current status
codewiki status

# Search entities
codewiki files search --where.state=ready
codewiki symbols search --where.name="myFunction"

# Set descriptions
codewiki symbols set-description 123 "This function does X..."

# Query relationships
codewiki files imported-by-file 456
codewiki dependencies used-by-file 456

Database Schema

Uses SQLite with Drizzle ORM. Key tables:

• files - File metadata and state
• symbols - Function/class/variable definitions
• directories - Directory structure
• external_dependencies - npm packages and built-ins
• imports - File-to-file import relationships
• symbol_*_links - Symbol dependency relationships

Configuration

type Config = {
  db_path: string              // SQLite database file path
  project_path: string         // Root directory of TypeScript project
  description?: string         // Optional project description
  package_json_path?: string   // Optional package.json location
  node_modules_paths?: string[] // Optional node_modules directories
}

Development

# Install dependencies
pnpm install

# Build the library
pnpm build

# Run tests
pnpm test

# Type checking
pnpm typecheck

# Database operations
pnpm db:generate  # Generate migrations
pnpm db:push      # Push schema changes
pnpm db:studio    # Open Drizzle Studio

Architecture

This is the core library that powers:

• codewiki-cli - Command-line interface and MCP server
• codewiki-vscode - VSCode extension (planned)

The library is designed to be:

• Incremental - Only processes changed files
• Performant - Uses SQLite and efficient AST parsing
• Extensible - Clean separation between parsing, storage, and API layers
• Type-safe - Full TypeScript support throughout

License

MIT © Amine Ben hammou