ctx-optimize

Gather a codebase — and its world — into one local knowledge store an AI agent answers from. Deterministic. No LLM API. No DB. Gather once, refresh cheaply, never go everywhere every time.

Your coding agent burns its context window on grep-and-read: to answer one question it greps, opens files, chases callers, re-reads. ctx-optimize turns a repo — plus, via adapters, database schemas, messaging topics, log shapes, documents — into a queryable graph stored as plain files in a central per-module store, and your agent (Claude Code, Codex, Devin — any skill-capable harness) answers from the store in a single call. The binary is deterministic — no LLM, no DB, and network only when you ask: update (releases), grammar build (zig, downloaded once), and whatever your own remote scripts do. The only intelligence in the system is the agent you already run.

Status: v0.4. On npm (@muthuishere/ctx-optimize) with prebuilt binaries for macOS / Linux / Windows; CI green; benchmarks reproducible (see Proof). Working today: code extraction for 12 embedded languages (Go, Python, JS, TS/TSX, Java, C, C++, C#, Rust, Zig, SQL — tree-sitter compiled to WASM, zero setup) plus drop-in grammar packs for any other language (kotlin/swift/dart ship in grammars/), markdown docs, the universal adapter door, query/path/explain/affected/hubs, symbol cards (card X: signature + doc + callers/callees, no file read), change-plan (one composed answer for "I'm about to change X": signature + callers + blast radius + which tests to run), the deterministic wiki (regenerated on every add) with a community-detected "Subsystems" map, the save-result/reflect learning loop, merge/export (json/dot/graphml/csv/obsidian), scripted remote push/pull, and multi-module monorepo support (scan / init --scan / parallel fan-out add / navigator + federated queries). New in v0.4 (breaking): the remote is your script — the binary ships no transport of its own; remote push/pull run the commands you declare in the committed config (remote init and the built-in file:///s3:// lanes are gone — see Sharing), up (the one onboarding verb: pull, gather, or refresh — whatever the state needs), sync + adapters run (fast lane / slow lane around adapter scripts), and update (self-updates the binary and every installed surface — sha256-verified, user-invoked only). New in v0.3: framework routes (FastAPI/Flask/Express/NestJS/Angular/ React Router/Vue + OpenAPI/Drupal/Ingress YAML — route nodes linked to their handlers, so affected <handler> surfaces the URL that binds it), the manifest lane (package.json/pom.xml/csproj+sln/go.mod/gradle dependencies + K8s topology as graph — one dep: node federates across build tools and modules), git-history co-change edges ("these files change together", from git log alone), a first-class React dashboard (serve: onboard/repos/viewer/settings/changes, all audited), and the pack doctrine — routes and manifests are extensible with drop-in JSON packs (routes add / manifests add, name or GitHub URL) exactly like grammar packs. Exact call edges (x/tools + LSP) are next — see openspec/.

Site, demos, benchmarks: https://muthuishere.github.io/ctx-optimize-site/ — landing page, unedited demos, and the full proof write-up. Everything below is reproducible; see Proof.

Install

npm (recommended — thin JS launcher resolves a prebuilt platform binary via optionalDependencies; no postinstall script, no download):

npm install -g @muthuishere/ctx-optimize

Go:

go install github.com/muthuishere/ctx-optimize/cmd/ctx-optimize@latest

Then install the agent surface — skills + hooks for every agent CLI it detects (Claude Code, Codex, Copilot, Devin):

ctx-optimize install

Later, one command updates the whole tool:

ctx-optimize update           # the binary itself (npm installs via npm; standalone
                              # binaries from GitHub Releases, sha256-verified
                              # against checksums.txt, swapped atomically; dev
                              # builds left alone), then skills + hooks + the
                              # global rule from the new binary — an exact replace
ctx-optimize update --check   # report only, touch nothing

The network call happens only when YOU run it — the binary never checks for updates in the background. ctx-optimize uninstall removes everything install wrote; stores and committed repo pointers stay.

Usage

One verb is the whole getting-started story — bare repo, fresh clone, teammate machine, CI, stale store, doesn't matter:

ctx-optimize up

up looks at the state and does the right thing: no config → bootstraps it (monorepos via scan; curate .ctxoptimize/config.json after) and gathers; committed config with a remote.pull and no local store → pulls the team's prebuilt graph (falls back to gathering, loudly); no remote → gathers; stale vs git HEAD → fast re-gather; fresh → no-op. Idempotent — run it whenever.

# author-side, when you want control instead of `up`'s defaults:
# scaffold .ctxoptimize/ (config, adapter + transport samples,
# remote.example.md), review monorepo module lists, pick pointer targets
ctx-optimize init

# gather / refresh explicitly (up calls these lanes for you)
ctx-optimize add .

# ask the store — complete, citable hits under a token budget
ctx-optimize query "where is the refund flow" --json

# about to change something? ONE composed call: signature + callers +
# blast radius + WHICH TESTS TO RUN + co-change history
ctx-optimize change-plan "RefundService"

# fast lane / slow lane: re-gather code without running adapter scripts;
# run adapters (DB dumps, doc converters) on demand — all, or one by name
ctx-optimize sync
ctx-optimize adapters run

# feed ANY system through the universal adapter door (strictly validated)
./my-postgres-adapter | ctx-optimize add --json -

# combine module stores into one view; dump for other tools
ctx-optimize merge api worker billing --into everything
ctx-optimize export --format dot --out graph.dot

# see it: local dashboard (embedded single file, zero external requests)
ctx-optimize serve          # → http://127.0.0.1:4747 — graph, search, details

ctx-optimize status --json

• The store is plain files (ndjson/json/md) — diffable, portable, at ~/ctxoptimize/<repo-name>/. The only thing in your repo is the committable .ctxoptimize/ directory.
• Sharing is your script. remote push / remote pull run the commands declared in the committed config — the binary ships no transport of its own. Queries always run on the local folder. See Sharing.

Sharing — the remote is your script

The binary never moves bytes to a host it chose. remote push / remote pull run the commands you declare in .ctxoptimize/config.json — any shell line (js, py, sh, or inline):

{
  "remote": {
    "push": "node .ctxoptimize/push.js",
    "pull": "node .ctxoptimize/pull.js"
  }
}

Your command gets the store context in env — CTX_STORE_DIR (the local store tree; pull pre-creates it), CTX_STORE_KEY, CTX_SCOPE_PREFIX (module scope), CTX_DIRECTION (push/pull — one script can serve both) — and a non-zero exit fails the verb. Same trust model as adapters and npm scripts.

init scaffolds a complete zero-dependency git lane as inert samples: a private git repo hosts every store (artifacts are sorted ndjson, so git diffs and merges them cleanly). Arming it:

gh repo create your-org/ctx-stores --private          # once per team
mv .ctxoptimize/push.js.sample .ctxoptimize/push.js
mv .ctxoptimize/pull.js.sample .ctxoptimize/pull.js
# set STORE_REPO_URL in both, add the "remote" block to config.json, commit
ctx-optimize remote push

A teammate who clones the repo runs ctx-optimize up — done. S3/R2/MinIO is a small aws-CLI script over the same env contract; GCS, artifactory, rsync-over-ssh, anything: write the script that copies CTX_STORE_DIR to and from your host and declare it. Recipes live in the scaffolded .ctxoptimize/remote.example.md. Secrets stay env-var NAMES that the shell expands at run time — never in config or scripts, never printed.

Upgrading from v0.3: remote init and the built-in file:///s3:// transports are gone. A legacy URL-shaped config still loads but is inert — push/pull print the migration pointer.

Multi-module — monorepos get one graph per module, plus a navigator

One giant graph for a 300-module monorepo helps nobody: people work in one module at a time, and an agent that loads the whole repo's graph pays for 299 modules it isn't asking about. ctx-optimize builds one store per module and a small navigator that routes questions instead:

# find every project in the tree — read-only, prints the exact config it would write
ctx-optimize scan                # markers: go.mod/go.work, package.json, gradle,
                                 # maven, Cargo.toml, pyproject… (--depth N, default 5)

# write ALL found modules into the committed config — generated once, then the
# list is yours: edit, add, prune (.ctxoptimize/config.json modules[])
ctx-optimize init --scan --yes

# gather: one worker per module, in parallel; stores mirror the repo tree
ctx-optimize add .               # → ~/ctxoptimize/<repo>/<module-path>/, each with
                                 #   its own graph + wiki  [--jobs N]

Measured on apache/beam: 310 modules discovered at depth 8, all gathered in 14.5s at ~9× CPU, zero failures — including maven modules nested inside other modules' resource trees.

The root store holds a navigator, not a merged giant graph: modules.json + navigator.md — every module's path, node/edge counts, top hub symbols, and README one-liner — plus a unified wiki front page linking into each module's own wiki. Query scope then follows your cwd:

cd sdks/java/transform-service
ctx-optimize query "expansion service"  # answers from THIS module's graph, labeled;
                                        # zero hits auto-escalate repo-wide (--root forces)
cd -                                    # back at the repo root:
ctx-optimize query "kafka read"         # navigator ranks modules, federates across the
                                        # best matches  [--modules all|a,b]
ctx-optimize card SomeSymbol            # not in your module? answered from the owning
                                        # module, labeled "[not in X — found in Y]"

merge <mod>... --into <name> stays opt-in for when you actually want one combined graph. (graphify's monorepo story is manual per-directory builds — no discovery, no parallel gather, no navigator.)

Proof — reproducible, not our word

Two kinds of evidence, both runnable.

Speed vs graphify (raw data in benchmarks/): a 12k-file corpus gathered in 0.67s vs 8.88s, queries ~4× faster, a smaller store. Methodology on the site.

What an agent actually saves. A headless harness lets the same model answer a set of questions three ways over OpenRouter — plain shell, ctx-optimize, and graphify — and reports the provider's own token/cost accounting (usage.include=true), not our estimate. Last public CI run on gorilla/mux (a small, well-named repo — plain grep's best case, i.e. the hardest terrain for a graph to win on):

| comparison | result | |---|---| | ctx-optimize vs plain shell | −31% cost · −64% tool calls · −36% tokens | | ctx-optimize vs graphify | ~half the tokens & tool calls | | graphify vs plain shell | +22% tokens — its query returns a raw node dump that costs more than grep |

ctx-optimize answers most questions in a single query/card call; both arms answered correctly with file:line citations (a cheaper wrong answer is a loss, not a saving).

Run it yourself — no source needed, it uses the published CLI:

npm i -g @muthuishere/ctx-optimize      # the store CLI
pipx install graphifyy                  # the competitor (arm c; optional)
export OPENROUTER_API_KEY=sk-or-...      # read from env only, never logged
bash proof/agent/run-bench.sh           # defaults: gorilla/mux, openai/gpt-4o-mini

Or fork and click Run workflow — .github/workflows/benchmark.yml runs it headless on a clean runner and publishes the table to the job summary. Harness + full write-up: https://muthuishere.github.io/ctx-optimize-site/proof/agent/

.ctxoptimize/ — config that travels with the repo

.ctxoptimize/
  config.json          name + remote commands (+ modules[] in a monorepo)
  adapters/            drop scripts here — every .js/.py/.sh runs on `add`
  push.js / pull.js    your transport scripts (init writes an inert *.sample pair)
  remote.example.md    transport recipes: git lane, s3 lane, custom

config.json:

{
  "name": "my-module",
  "remote": {
    "push": "node .ctxoptimize/push.js",
    "pull": "node .ctxoptimize/pull.js"
  }
}

Commit the directory — it is safe by construction:

• name picks the store folder under ~/ctxoptimize/ (default: repo basename).
• remote declares the push/pull commands — plain shell lines the binary runs as-is (cwd = repo root). Secrets stay env-var NAMES in scripts and config alike; the shell expands them at run time — values are never written or printed.
• Adapters are files: dropping kafka.js into .ctxoptimize/adapters/ is the whole registration (.js/.mjs → node, .py → python3, .sh → sh; other extensions inert — init seeds an example.js.sample template). Each script prints batch JSON to stdout; ctx-optimize add runs the built-in extractors and every adapter through the fail-closed door. Adapters can be arbitrarily slow (DB dumps, doc converters), so they get their own lanes: sync re-gathers the repo you're in and skips them (safe — replace is producer-scoped, adapter nodes stay put), adapters run [name] re-runs all or one on demand, add --no-adapters is the fast lane spelled long. One add refreshes the whole world; a fresh clone needs zero setup — ctx-optimize up.

Grammar packs — add any language without recompiling

A language is just a grammar + a node-type mapping. The 12 embedded ones cover the mainstream; anything else is a pack: <name>.wasm + <name>.json dropped into ~/ctxoptimize/grammars/ (machine-wide) or .ctxoptimize/grammars/ (travels with the repo). Next add picks it up; pack extensions override embedded ones. kotlin, swift and dart ship as packs in grammars/ — copy the pair in to enable.

Build your own from ANY tree-sitter grammar with one command — no toolchain to install (zig auto-downloads once, sha256-verified; grammar fetched as a tarball, no git):

ctx-optimize languages add kotlin        # known names resolve to the right repo/branch/exts
ctx-optimize languages add https://github.com/tree-sitter-grammars/tree-sitter-lua
# → ~/ctxoptimize/grammars/<name>.wasm + <name>.json (mapping auto-suggested
#   from the grammar's node-types.json — review it, then `add` just works)
ctx-optimize languages list              # embedded + packs + addable names
ctx-optimize languages remove <name>

Adapters — the open door

Everything external is an adapter emitting one JSON schema into ctx-optimize add --json -: nodes (id, label, kind, file_type, source, location) and edges (source, target, relation, confidence ∈ EXTRACTED|INFERRED|AMBIGUOUS). The door validates strictly and tags provenance per producer. Your agent can write a new adapter on demand — point it at any system with the schema and it gathers it. Make it permanent by dropping the script into .ctxoptimize/adapters/ — every future add runs it.

Design

Evidence-first: every product decision traces to a measured spike (openspec/changes/2026-07-11-graphify-gaps/spikes.md) — including honest benchmarks against a real agent baseline (not corpus-stuffing strawmen), the terrain law (graph value is inverse to a codebase's greppability), and the symbol-card finding (agents' reads are pointer-chases a complete answer eliminates). Extensibility is a verified differentiator, not a slogan: a source audit of graphify (2026-07-11) found its languages, data-source lanes and exporters are all fork-required static registries (only its remote hooks are user-pluggable); here languages are drop-in packs, adapters are dropped scripts, and the batch door takes any producer. Vision: docs/VISION.md. Standing critique: docs/CRITIQUE.md.

Lineage

With all due respect to graphify — a project we learned a great deal from — there is a direct line between it and this tool: graphify's central graph store and its pluggable remote push/pull hooks (the one part of graphify an end user can extend without forking) were contributed upstream by this project's author (graphify #1751 / #1752; git-verifiable). ctx-optimize is that same idea carried through the whole product: the store, the languages, the adapters, and the sync are all open seams by design — nothing here requires a fork to extend.

License

MIT © 2026 Muthukumaran Navaneethakrishnan

Made by muthuishere.