Toji

Toji is a Pi extension workspace with three packages:

• toji-mem: local code memory, search, graph traversal, and benchmarks
• toji-kompak: branch-local hybrid compaction and persistent goal execution
• toji-agents: specialized, inspectable child agents forked from the current Pi session

toji-kompak preserves Pi's immutable session tree and compaction boundaries while adding inspectable structured checkpoints, file freshness, verification state, and bounded /goal continuation. It stores no external session snapshots.

What toji-mem does

toji-mem indexes a repository into SQLite and answers common agent questions:

• where is this symbol?
• what file should I read?
• what related files/symbols are nearby?
• what might be affected if I change this symbol?

It uses:

• SQLite FTS5 for fuzzy search over files, symbols, docstrings, and paths
• B-tree indexes for exact file and symbol lookups
• Tree-sitter parsers for symbols, imports, and calls
• graph edges for blast-radius and related-file traversal

What toji-agents does

toji-agents lets the active Pi model delegate work to six fixed specialists. Every invocation creates a real child session from the current parent leaf, runs a separate Pi process, streams its activity in the parent tool row, and returns the child's final response.

• Doran validates backend code, tests, and edge cases.
• Oner scouts and summarizes codebases.
• Faker creates strict technical plans.
• Peyz handles frontend design and implementation.
• Keria designs cloud architecture from requirements and costs.
• Rekkles performs graph-assisted critical-path and performance reviews.

Full child transcripts remain available through the session path shown in the expanded tool result. Doran and Peyz now default to Opus 5 with low thinking and fall back first to their former Fable 5 model. Rekkles runs on Opus 5 and calls the deterministic toji_review graph helper before reviewing critical frontend/backend paths, bottlenecks, and dead code. All six specialists can use that helper; it does not spawn another agent.

Install from Pi

After publishing to npm, install Toji like this:

pi install npm:@mr-jones123/toji

Try it for one Pi run without installing:

pi -e npm:@mr-jones123/toji

Local setup

npm install

Use with Pi

If installed with pi install, start Pi normally and all three extensions are discovered.

For local development, start Pi with one extension loaded:

pi -e ./packages/toji-mem/src/index.ts
pi -e ./packages/toji-kompak/src/index.ts
pi -e ./packages/toji-agents/src/index.ts

toji-mem commands

/toji-index .
/toji-query indexProject
/toji-overview .
/toji-graph index project
/toji-blast indexProject
/toji-bench .

| command | purpose | |---|---| | /toji-index [path] | index a project into Toji memory | | /toji-query <query> | search indexed files, symbols, and standards | | /toji-overview [path] | show compact project overview, expandable in Pi | | /toji-graph <intent> | find related files/symbols from a natural-language intent | | /toji-blast <symbol> | traverse likely impact radius for a symbol | | /toji-bench [path] | run the operational benchmark from inside Pi |

toji-agents

Ask naturally and the parent model calls toji_agent:

Ask Oner to scout the authentication flow.
Work with Faker on a plan for multi-tenant billing.
Have Doran validate the backend changes.
Ask Peyz to redesign this dashboard.
Have Keria compare AWS and Cloudflare architectures.
Ask Rekkles to review the critical frontend and backend performance paths.
Query graph-selected review context directly with toji_review.

toji_agent accepts an agent name and concrete task; toji_review accepts a review scope and returns deterministic graph context. Child output is capped at Pi's 50KB tool limit; the full child session remains on disk.

Reproduce the operational benchmark

The benchmark measures Toji core directly, without LLM/session overhead:

• cold index
• hot re-index
• query p50
• project overview p50
• blast-radius p50
• indexed files/symbols/edges

1. Benchmark Toji itself

cd packages/toji-mem
npm run bench -- --repo . --json

2. Benchmark Flask

From the repository root:

mkdir -p benchmarks/repos
git clone --depth 1 https://github.com/pallets/flask.git benchmarks/repos/flask
cd packages/toji-mem
npm run bench -- --repo ../../benchmarks/repos/flask --json

If Flask is already cloned:

git -C benchmarks/repos/flask pull --ff-only
cd packages/toji-mem
npm run bench -- --repo ../../benchmarks/repos/flask --json

Current smoke scores

| repo | cold index | hot index | query p50 | overview p50 | blast p50 | files | symbols | edges | |---|---:|---:|---:|---:|---:|---:|---:|---:| | packages/toji-mem | 679.02 ms | 5.18 ms | 0.68 ms | 0.75 ms | 1.71 ms | 20 | 98 | 647 | | flask | 1888.94 ms | 12.85 ms | 0.71 ms | 9.67 ms | 0.29 ms | 83 | 1620 | 7658 |

RepoBench retrieval scores

Toji also has a local RepoBench Python v1.1 adapter for cross-file retrieval quality. It materializes synthetic repos grouped by repo_name, indexes them, then queries with the gold identifier plus repo/current-file context.

| split | rows | files | Path Hit@1 | Path Hit@5 | Path MRR | Symbol Hit@1 | Symbol Hit@5 | Symbol MRR | p95 latency | |---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | cross_file_first | 8,026 | 17,081 | 0.576 | 0.600 | 0.587 | 0.540 | 0.551 | 0.545 | 0.784 ms | | cross_file_random | 7,610 | 16,750 | 0.513 | 0.544 | 0.526 | 0.467 | 0.478 | 0.471 | 1.014 ms |

Compared to the original name-only baseline, repo/import-aware reranking improves Path Hit@1 by about 13 points on both splits.

Benchmark notes

• CLI benchmark is the source of truth for stable operational numbers.
• /toji-bench uses the same benchmark engine from inside Pi.
• Cloned benchmark repos are scratch data and are not committed.
• Full RepoBench tables live in packages/toji-mem/README.md.