pi-cli-dynamic-tools

Pi extension that lets the LLM build and install CLI tools on the fly. Generated tools persist across sessions, but they are inactive by default until explicitly enabled for the current session.

When Pi needs a capability it doesn't have (web search, file conversion, API calls, etc.), it can call toolbox_create, write the code, validate the generated CLI, and register it as a Pi tool. Use toolbox_manage with action enable to activate a named generated tool for the current session.

How it works

User: "search the web for..."
         │
         ▼
Pi notices it has no web-search tool
         │
         ▼
Pi calls toolbox_create  ──► scaffolds Node.js script
                          ──► runs npm install (if deps needed)
                          ──► validates the CLI contract
                          ──► saves to registry (~/.pi/agent/toolbox/)
                          ──► generates a SKILL.md
                          ──► registers as an inactive Pi tool
         │
         ▼
Pi calls toolbox_manage action=enable name=web-search
         │
         ▼
Pi can now call web_search "your query"  ──► returns results
         │
         ▼
Pi answers the user ✓

Pi proposes the tool; creation requires your explicit confirmation before npm installation or generated code execution. Creation is unavailable in print/JSON modes; TUI and RPC confirmation are supported.

Compatibility

Verified against pi 0.85.1 (@earendil-works/pi-coding-agent), the latest npm release checked during this review. Uses the current typebox and @earendil-works/pi-ai imports. The legacy @mariozechner distribution is not a supported target.

Requires Node.js 22.6+ and a POSIX environment (macOS/Linux, or WSL); generated wrappers require /bin/sh. Peer ranges follow pi's package recommendations; they do not promise compatibility with every historical or future release.

Installation

# From this repository
npm ci
pi install .
# In a running pi session, use /reload

Tools registered by the extension

toolbox_create

Creates a new CLI tool from code the LLM writes.

The extension wraps your code in a skeleton that automatically handles:

• --help / -h — structured help with NAME, USAGE, DESCRIPTION, OPTIONS, EXAMPLES, EXIT CODES
• --version
• schema subcommand — machine-readable JSON schema
• doctor subcommand — self-diagnostics / connectivity checks
• --json — structured output mode
• Exit codes (0 success, 1 general error, 2 network, 3 bad args)

After creation the tool is validated against this contract. If validation fails, Pi reads the error and tries again. Created tools are registered but stay inactive by default; enable them with toolbox_manage when you want them available in the current session.

toolbox_manage

List, remove, run diagnostics on, or enable installed tools.

| Action | Description | |--------|-------------| | list | Show all installed tools | | enable <name> | Additively enable a generated tool for the current session | | remove <name> | Uninstall a tool and its skill | | doctor [name] | Run self-diagnostics on one or all tools |

/toolbox command

Slash command available in the Pi CLI for quick management:

/toolbox list
/toolbox enable web-search
/toolbox remove web-search
/toolbox doctor
/toolbox doctor web-search

File layout

By default, tools are stored under ~/.pi/agent/toolbox/. For tests or isolated runs, set PI_TOOLBOX_DIR to another directory before Pi loads the extension.

~/.pi/agent/toolbox/
├── registry/          # JSON entry per installed tool
├── installs/
│   └── web-search/    # Node.js package (index.mjs + package.json + node_modules)
├── bin/
│   └── web-search     # Shell wrapper → node installs/web-search/index.mjs
└── skills/
    └── web-search/
        └── SKILL.md   # Auto-generated skill picked up by Pi

Example: Pi builds a web-search tool

Just ask Pi to search the web. It will build the tool if it doesn't exist yet.

You: search the internet for "latest Node.js release"

Pi will:

1. Call toolbox_create with a name web-search, install something like node-fetch or use a search API.
2. Write the run() logic, doctor() check, help text, and schema.
3. Install, validate, and register the generated tool.
4. Enable it for the current session with toolbox_manage action enable.
5. Call web_search --query "latest Node.js release" and answer you.

On the next session the tool is still installed and registered, but remains inactive until enabled again. The generated skill is picked up so Pi knows when the tool may be useful.

Example: manually inspecting a created tool

After Pi creates web-search, you can interact with it directly:

# See the help
~/.pi/agent/toolbox/bin/web-search --help

# Check connectivity
~/.pi/agent/toolbox/bin/web-search doctor

# Run a search
~/.pi/agent/toolbox/bin/web-search --query "Node.js 22 release notes"

# Get structured JSON output
~/.pi/agent/toolbox/bin/web-search --query "Node.js 22" --json

Example: listing and removing tools

You: /toolbox list
Pi:  Installed: web-search, pdf-to-text, github-search

You: /toolbox doctor web-search
Pi:  web-search: OK

You: /toolbox remove web-search
Pi:  web-search removed

Or via Pi tools (the LLM can do this too):

You: remove the web-search tool from the toolbox
Pi:  [calls toolbox_manage action=remove name=web-search]
     "web-search" removed.

What gets validated before a tool is accepted

Every new tool must pass 5 checks before it's registered:

| Check | What it verifies | |-------|-----------------| | --help | Exits 0, output contains all required sections | | -h | Identical output to --help | | schema | Exits 0, returns valid JSON object | | --version | Exits 0, returns non-empty string | | doctor | Exits 0 (connectivity / prerequisites ok) |

If any check fails, the tool is deleted and Pi gets the failure details so it can fix and retry.

Persistence across sessions

• Tools are stored on disk — they survive restarts.
• At session_start, the extension scans the registry and re-registers every installed tool whose binary exists.
• Generated tools are kept inactive by default. toolbox_manage action enable uses Pi's dynamic tool loading API additively, preserving active tools owned by other extensions while adding the named generated tool for this session.
• Existing active-tool choices are preserved; management tools are registered normally, without forcibly re-enabling tools the user disabled.
• Removal deactivates the tool and revokes stale handlers. Name collisions with other extensions are rejected.
• Generated tool definitions omit active-only prompt metadata to avoid unnecessary system-prompt cache invalidation.
• Skills in ~/.pi/agent/toolbox/skills/ are automatically surfaced via resources_discover, so Pi knows about the tools and when to use them.

Security model

Generated tools and extensions can execute arbitrary code, install npm packages (including lifecycle scripts), access files and network resources, and run with the current user's permissions. This is not a sandbox. Validation itself executes generated code, including imports and doctor; review the proposed code and dependencies before confirming creation, not just before enabling the tool.

Execution is asynchronous and cancellable, with 30-second command, 15-second validation-check, and 60-second npm deadlines. Combined stdout/stderr capture is limited to 50 KiB; overflow terminates execution and reports an error. Displayed output is limited to 2000 lines; discarded output is not saved. Storage rejects invalid names, reserved built-in names, malformed registry entries and symlinks within the toolbox root. These checks do not constrain malicious generated code or provide isolation between OS processes.

Generated Pi tools accept exact argv arrays, for example {"args":["--query","hello world","--json"]}. Shell quoting/expansion is not performed. Older string-based tool calls must be updated to arrays.

New or removed skills are reflected after /reload (or the next session); tool registration and activation take effect immediately.

Development

npm ci
npm run check   # strict TypeScript check against pi 0.85.1 + integration tests
npm audit

Tests use a temporary PI_TOOLBOX_DIR, real schema dependencies, and pi's real extension loader; they do not require model credentials or touch your installed toolbox.

Architecture

| File | Role | |------|------| | src/index.ts | Extension entry point — registers tools, session hooks, slash command | | src/installer.ts | Scaffold → npm install → wrapper → validate → registry → skill | | src/skeleton.ts | Generates the Node.js script from the LLM-provided code | | src/validator.ts | Runs the 5 CLI contract checks | | src/skill-gen.ts | Generates the SKILL.md for each installed tool | | src/registry.ts | Read/write JSON entries in toolbox/registry/ | | src/types.ts | Shared types and directory constants | | src/process.ts | Cancellable subprocess execution with bounded capture and deadlines |