TokenCave

Semantic cache layer for codebases. Scans TypeScript repos and stores compact representations — function signatures, structural skeletons, and LLM summaries — in a local SQLite database. AI agents (Claude Code, custom agents, scripts) query the cache instead of reading raw source, typically consuming 60–90% fewer tokens per task.

Without TokenCave:  Claude reads engine.ts   →  ~1,800 tokens
With TokenCave:     Claude reads cache entry  →    ~96 tokens   (95% savings)

Table of contents

• Install
• Running from source
• API key setup
• Quick start
• Commands
• Use with Claude Code
• Programmatic API
• Token savings reference
• Data storage

Install

npm install -g token-crave

Verify:

token-crave --version

Running from source

Clone the repo and install dependencies:

git clone https://github.com/rnavdeep/tokencave.git
cd tokencave
npm install

Run without building (uses tsx to execute TypeScript directly):

npm run dev -- scan /path/to/your-repo
npm run dev -- context /path/to/your-repo src/foo.ts
npm run dev -- visualize /path/to/your-repo

Build and run the compiled output:

npm run build
node dist/cli/index.js scan /path/to/your-repo

Type-check only (no emit):

npm run typecheck

API key setup

TokenCave calls the Anthropic API to generate per-file summaries and architecture docs. You need an API key.

Get an API key

1. Go to console.anthropic.com
2. Sign in → API Keys → Create Key
3. Copy the key (starts with sk-ant-…)

Set the key

Option 1 — shell environment (quickest):

export ANTHROPIC_API_KEY=sk-ant-your-key-here

Add that line to ~/.zshrc or ~/.bashrc to make it permanent:

echo 'export ANTHROPIC_API_KEY=sk-ant-your-key-here' >> ~/.zshrc
source ~/.zshrc

Option 2 — .env file in your project root:

echo 'ANTHROPIC_API_KEY=sk-ant-your-key-here' > .env

TokenCave picks it up automatically when the key is present in the environment.

Scan without an API key: pass --no-summaries to skip LLM calls entirely. Signatures and skeletons are extracted purely from the AST — no API cost.

Quick start

# 1. Set your API key (if not already done)
export ANTHROPIC_API_KEY=sk-ant-...

# 2. Go to any TypeScript repo
cd /path/to/your-repo

# 3. Scan it — builds the cache (~30 s for a 25-file repo)
token-crave scan .

# 4. Install the Claude Code hook so reads are intercepted automatically
token-crave hook install

# 5. Start a Claude Code session — it now uses the cache for every file read
claude

That's it. Claude Code will read cached representations instead of raw source from this point on. Re-run token-crave scan . --force after large changes to refresh the cache.

Commands

token-crave scan <path>

Scan a repo and build (or refresh) the semantic cache.

token-crave scan .
token-crave scan /path/to/other-repo

| Flag | Default | Description | |---|---|---| | --no-summaries | off | Skip LLM summaries — AST-only, no API cost | | --force | off | Reprocess every file even if hash unchanged | | --extensions <exts> | .ts,.tsx | Comma-separated list, e.g. --extensions .ts,.tsx,.js | | --max-file-size <bytes> | none | Skip files larger than N bytes | | --provider anthropic\|zai | anthropic | LLM provider for summaries |

# Full scan with summaries (uses Anthropic API)
token-crave scan . --force

# Fast scan, no API calls
token-crave scan . --no-summaries

token-crave context <path> <files...>

Fetch compact cached representations for one or more files and print them.

token-crave context . src/api/auth.ts
token-crave context . src/api/auth.ts src/models/user.ts

| Flag | Description | |---|---| | --json | Output as JSON array (pipe to jq, scripts, agents) | | --expand-deps | Also include signatures of each file's direct imports |

# Human-readable
token-crave context . src/semantic/engine.ts

# Machine-readable JSON
token-crave context . src/semantic/engine.ts --json

# Include dependency signatures (one call, no round-trips)
token-crave context . src/semantic/engine.ts --expand-deps --json

Each JSON entry:

{
  "path": "/abs/path/to/file.ts",
  "language": "typescript",
  "mode": "summary",
  "content": "This file implements...",
  "tokenCount": 96,
  "stale": false,
  "dependencies": [...]
}

mode is one of summary | skeleton | signatures | raw — TokenCave picks the best available representation automatically.

token-crave visualize <path>

Open a browser UI to browse the cache for a repo.

token-crave visualize .
# → http://localhost:5555

| Flag | Default | Description | |---|---|---| | --port <n> | 5555 | Port to listen on |

token-crave hook install

Install a Claude Code PreToolUse hook that automatically intercepts file reads and serves cached representations. See Use with Claude Code for the full walkthrough.

token-crave hook install           # project-level (.claude/settings.json)
token-crave hook install --global  # global (~/.claude/settings.json)

token-crave hook uninstall

token-crave hook uninstall
token-crave hook uninstall --global

token-crave hook status

token-crave hook status
token-crave hook status --global

Use with Claude Code

Option A — Hook (automatic, recommended)

One command wires up a PreToolUse hook that intercepts every Read tool call Claude makes. When the target file is in a fresh cache, the hook blocks the raw read and returns the compact representation instead. Nothing else to configure.

# Inside the repo you want to protect
token-crave scan .
token-crave hook install

What Claude Code sees when it tries to read src/semantic/engine.ts:

[TokenCave: src/semantic/engine.ts — summary, 96 tok (raw ~1800 tok, 88% savings)]

This file implements the core semantic analysis engine that processes source
files to extract and cache their code representations…

Pass-through rules — the hook does nothing when:

• The file is not in the TokenCave cache
• The file has been modified since the last scan (stale)
• The file is outside a git repo

Re-scan to refresh stale files: token-crave scan . --force

Option B — CLAUDE.md instruction (manual)

Add this block to your repo's CLAUDE.md if you prefer Claude to explicitly decide when to use the cache:

## Semantic cache (TokenCave)

This repo has a TokenCave cache. Before reading raw source files, get compact
context with:

    token-crave context . <file1> [file2 ...] --json

Each result has `mode` (summary/skeleton/signatures/raw), `content`, `tokenCount`,
and `stale`. Use this instead of Read for any TypeScript file.

Re-scan after large changes:  token-crave scan . --force

Programmatic API

Install as a library dependency:

npm install token-crave

import { getContext } from 'token-crave/api/context';

// Best cached representation per file
const results = await getContext('/path/to/repo', [
  '/path/to/repo/src/api/auth.ts',
  '/path/to/repo/src/models/user.ts',
]);

// With dependency expansion
const results = await getContext('/path/to/repo', [
  '/path/to/repo/src/semantic/engine.ts',
], { expandDependencies: true });

for (const file of results) {
  console.log(file.mode, file.tokenCount, 'tokens');
  for (const dep of file.dependencies ?? []) {
    console.log('  dep:', dep.resolvedFromSpecifier, dep.tokenCount, 'tok');
  }
}

FileContext

interface FileContext {
  path: string;
  language: string;
  mode: 'summary' | 'skeleton' | 'signatures' | 'raw';
  content: string;
  tokenCount: number;
  stale: boolean;
  dependencies?: DependencyContext[];
}

interface DependencyContext {
  path: string;
  language: string;
  mode: 'signatures' | 'skeleton' | 'summary';
  content: string;
  tokenCount: number;
  resolvedFromSpecifier: string;
}

getContext returns raw mode (full source) as a fallback when no cached representation exists or when the file is stale.

Token savings reference

Typical numbers on a mid-size TypeScript codebase:

| Representation | Avg tokens/file | vs raw source | |---|---|---| | summary | ~100 | ~75% fewer | | skeleton | ~200 | ~50% fewer | | signatures | ~30 | ~90% fewer | | raw source | ~400 | baseline |

With expandDependencies: true (one getContext call for a file with 8 imports):

| What's fetched | Tokens | |---|---| | 1 × file summary | ~100 | | 8 × import signatures | ~240 | | Total | ~340 | | Reading all 9 files raw | ~3,600 | | Saving | ~90% |

Data storage

• Cache is at ~/.token-crave/<repo-slug>/index.db (SQLite, never committed to your repo)
• One database per repo, keyed by the absolute path slug
• Safe to delete at any time — token-crave scan . rebuilds it from source
• The cache stores: file hashes, token counts, signatures, skeletons, summaries, inter-file dependency edges, and a generated architecture doc