Pi Training Harness

This repository provides Pi-based focused harness profiles without replacing Pi's runtime. See Profiles for the complete list.

The custom profiles share Pi's model runtime, TUI, headless modes, compaction, resume, and native JSONL sessions. Their system prompts and tool contracts remain isolated from the pi harness.

Install

Requires Node.js 22.19 or newer.

npm install --global @qwen-code/qwen-harness@dev
qwen-harness --help

For repository development, install dependencies with npm install --ignore-scripts.

Development

Use one development command for every harness. Omitting --harness runs the pi profile:

npm run dev
npm run dev -- --harness bash-apply-patch -p "Fix the failing tests"

Run

qwen-harness
qwen-harness --harness bash-apply-patch -p "Fix the failing tests"

Pass any name from the Profiles table to --harness. The wrapper consumes only this optional argument; every other argument is passed to Pi unchanged.

To use the same command from a checkout, run npm run build && npm link once. Re-run npm run build after source changes; qwen-harness will continue to point at this checkout.

The compiled entrypoint calls Pi's public main() function. The pi harness adds no system prompt or new tool names; it keeps Pi's four built-in tools and applies the bounded bash capability. pi-ptc preserves Pi's normal prompt but activates the packaged programmatic-tool-calling capability: the model sees only run_code, while Pi's four default tools are callable from its JavaScript runtime. self-compact preserves Pi's normal prompt and built-in tool names while adding context compaction and applying the same bounded bash capability; external extension and skill discovery are disabled for these harnesses. The Arena profiles load only their explicit tool extensions. Long Horizon loads only its declared pi-subagents dependency by its resolved package path and restricts the Supervisor to its three orchestration tools. Extension discovery, built-in tools, and unrelated resource catalogs remain disabled by each profile's own contract. The tool-comparison profiles keep Pi's built-in implementations except for the bounded bash capability, apply an explicit model-visible tool allowlist, and load only the selected custom editor when one is required.

Pi settings

Pi and the harness tools use the same global file:

~/.pi/agent/settings.json

Long Horizon does not use project or global extension discovery. It resolves pi-subagents from the harness runtime's own installed package dependencies, so other installed extensions do not run in the Supervisor session. The process working directory remains the writable task workspace, so container workspaces such as /app do not need to contain the harness's node_modules directory.

Pi's native fields configure the main agent. The harnesses field configures models and services used inside tools:

{
  "defaultProvider": "openai",
  "defaultModel": "qwen3.5-plus",
  "defaultThinkingLevel": "high",
  "compaction": {
    "enabled": true,
    "reserveTokens": 16384,
    "keepRecentTokens": 20000
  },
  "retry": {
    "enabled": true,
    "maxRetries": 5,
    "baseDelayMs": 2000,
    "provider": {
      "maxRetries": 2,
      "maxRetryDelayMs": 60000
    }
  },
  "shellPath": "/bin/bash",
  "npmCommand": ["node", "/path/to/npm-cli.js"],
  "harnesses": {
    "toolServices": {
      "routify": {
        "apiKey": "routify-key",
        "webSearchUrl": "https://routify-pub.alibaba-inc.com/protocol/compatible-mode/res/v1/llm/context",
        "imageSearchUrl": "https://routify-pub.alibaba-inc.com/protocol/brave/res/v1/images/search",
        "webFetchUrl": "https://routify-pub.alibaba-inc.com/protocol/search/tools"
      },
      "dashScope": {
        "apiKey": "dashscope-key",
        "apiBase": "https://dashscope.aliyuncs.com/api/v1"
      },
      "pexels": {
        "apiKey": "pexels-key",
        "cacheTtlMs": 86400000,
        "cacheDir": "/tmp/pi-harness-pexels-cache"
      }
    },
    "tools": {
      "webSearch": {
        "model": "brave.llm-context"
      },
      "generateImage": {
        "model": "qwen-image-2.0-pro",
        "size": "2048*2048",
        "taskTimeoutMs": 300000,
        "taskPollIntervalMs": 1000
      },
      "generateSpeech": {
        "model": "qwen-audio-3.0-tts-flash",
        "voice": "longanhuan_v3.6"
      },
      "imageCaption": {
        "provider": "openai",
        "model": "qwen-vl-max",
        "systemPrompt": "Return an accurate image caption.",
        "prompt": "Describe this image in detail for another model."
      },
      "code": {
        "bashTimeoutSeconds": 300,
        "fullStackBuildTimeoutMs": 300000,
        "previewPort": 3000,
        "previewUrl": "http://127.0.0.1:3000"
      },
      "documents": {
        "sofficePath": "/path/to/soffice"
      }
    }
  }
}

Only configure fields required by the tools you enable. Defaults are applied when optional fields are absent. Unknown harnesses fields fail at startup instead of being silently ignored. A missing service credential produces a tool error only when that service-backed tool is called.

Execution capabilities reuse Pi's native settings instead of defining harness-specific duplicates: npmCommand controls npm invocation, shellPath selects the command shell, and Pi resolves that shell with its normal cross-platform fallback behavior. Routify, DashScope, and Pexels HTTP requests use retry.provider.maxRetries and retry.provider.maxRetryDelayMs; the top-level retry count remains Pi's agent-turn policy. The harnesses namespace contains only capability-specific settings that Pi does not own.

The optional bash capability wraps Pi's built-in bash definition and changes only its omitted timeout to 120 seconds; explicit timeouts are preserved. Profiles must compose the capability to install it. Every profile that already exposed bash now uses it: pi, self-compact, Agent Arena, Code Arena Full Stack, and all four tool- comparison profiles. pi-ptc reuses the same bash definition for its nested tool. Profiles that did not expose bash do not gain it.

Tool service and model configuration does not read the legacy ARENA_*, CODE_HARNESS_*, DASHSCOPE_*, ROUTIFY_*, or PEXELS_* variables. Subprocess tools still inherit the normal process environment so builds and user programs can access their own environment variables; the runtime also injects workspace-path variables for command compatibility.

The settings file can contain plaintext service credentials. Restrict it to the current user:

chmod 600 ~/.pi/agent/settings.json

Main model versus tool models

• defaultProvider, defaultModel, model selection CLI flags, thinking level, retry, and compaction are handled by Pi.
• npmCommand and the shell resolved from shellPath are consumed by execution capabilities. External HTTP clients use retry.provider.maxRetries and retry.provider.maxRetryDelayMs.
• harnesses.tools.webSearch, generateImage, and generateSpeech configure the corresponding external tool calls.
• harnesses.tools.imageCaption is optional. When the active Pi model cannot read images, read_file sends the image to this Pi-registered side model and returns its caption. The side model uses Pi's model registry and credentials; provider and model must resolve through the same Pi installation.

Profiles

| Profile | Distinct behavior | | --- | --- | | pi | Pi's default prompt and tool names, with the bounded bash capability | | pi-ptc | Pi's default prompt with only run_code model-visible; JavaScript can call nested read, bash, edit, and write | | self-compact | Isolated Pi plus the bounded bash, SelfCompact-style context status, rubric, and the compact_context tool | | agent-arena | General workspace agent tools, multimodal file reading, search, image and speech generation, and interactive ask_user | | code-arena-html | Single-file HTML tools with a smaller edit/search/media tool set | | code-arena-react | React/Vite file tools, package installation, build validation, research, media search, and image generation | | code-arena-full-stack | Next.js/PostgreSQL tools, bash, managed build/start validation, research, media search, and image generation | | long-horizon-mea | A supervisor session coordinating fresh Manager, Executor, and Auditor child Pi sessions through pi-subagents | | bash-only | Pi's default prompt and runtime with only the bounded bash active | | bash-apply-patch | bash plus the Codex-style apply_patch capability | | bash-str-replace-editor | bash plus the DSH-style str_replace_editor capability | | bash-file-search | Pi's built-in read, edit, write, and grep, the bounded bash, plus glob backed by Pi's native file finder |

Tool-interface comparison

The profiles under src/harnesses/tool-comparison/ form a controlled tool-interface matrix. They all retain Pi's native prompt, model runtime, settings, sessions, and workspace behavior while disabling external resource discovery. Their model-visible tool sets are:

| Experiment | Run with --harness | Active tools | | --- | --- | --- | | Bash only | bash-only | bash | | Codex editing | bash-apply-patch | bash, apply_patch | | Pi editing baseline | pi | read, bash, edit, write | | Programmatic tool calling | pi-ptc | run_code (nests Pi's four default tools) | | DSH editing | bash-str-replace-editor | bash, str_replace_editor | | File search and editing | bash-file-search | read, bash, edit, write, grep, glob |

Pi calls its native glob-based search tool find; the final experiment exposes that same implementation under the conventional glob name without changing its execution or result format.

Programmatic tool calling / code mode

createProgrammaticToolCallingCapability(cwd, options?) (also exported as createCodeModeCapability) registers one model-visible function tool:

run_code(code, timeout_ms?, max_output_chars?)

The pi-ptc profile disables direct built-in-tool exposure and nests Pi's default read, bash, edit, and write definitions under the JavaScript tools object. Scripts emit selected results with text() or forward either an image block or a nested tool result with image(). Each call runs in a fresh child Node process with an empty environment, a memory limit, and Node's permission model denying direct filesystem, child-process, and worker access. The runtime stops synchronous JavaScript loops and scripts that make no progress, but it does not impose a default wall-clock deadline while a nested tool or a bounded setTimeout() is active. Configure duration on the nested tool doing the work (for example, tools.bash({ command, timeout: 300 })). Use top-level timeout_ms only when the entire orchestration needs an additional independent hard deadline. When embedding shell parameter expansion in a JavaScript template literal, escape its dollar sign (for example, \${name}). No runtime dependency or copied Codex source is added.

Pi's public extension API can enumerate another extension's tool metadata, but it does not provide an execution handle for those tools. Consequently, this capability cannot safely auto-capture arbitrary peer-extension tools. A future profile that needs custom nested tools must pass their ToolDefinition objects explicitly through options.nestedTools (and, when cwd-bound, options.resolveNestedTools). The current pi-ptc profile needs no such bridge because its nested set consists of Pi's four public default tool definitions.

Reusable editing capabilities

Two standalone editing capabilities are available for profile composition. The comparison profiles activate them independently so their effects can be evaluated without changing another profile:

• createApplyPatchCapability(cwd) registers apply_patch, which accepts a Codex patch envelope in { "patch": "*** Begin Patch\n...\n*** End Patch" } and supports add, update, delete, and move hunks.
• createStrReplaceEditorCapability(cwd, options?) registers DSH-style str_replace_editor with view, create, str_replace, and insert commands. Its public paths are absolute and confined to cwd.

Both factories return the normal HarnessCapability contract, so another profile can pass them to runComposedHarness or merge them with composeHarnessCapabilities without copying their tools.

SelfCompact context compaction

The self-compact harness loads the reusable capability implemented under context-management/self-compact, inspired by Self-Compacting Language Model Agents. Before every model call, the capability injects an ephemeral <context_status> block with estimated tokens, context-window size, used percentage, and remaining tokens. It registers one model tool:

compact_context(reason, preserve?)

Before calling the tool, the model applies the paper-inspired rubric: the current work unit must be closed, the continuation state must be safely summarizable, concrete progress must exist since the last compaction, and the agent must not be stuck. Token pressure alone is not enough. The tool finishes its own tool turn, runs Pi's native compactor, and automatically starts a continuation turn from the compacted state. The capability cancels only threshold-triggered automatic compaction. Pi's overflow recovery and user-initiated /compact remain enabled. Keep Pi's native compaction.enabled setting set to true; the capability suppresses the threshold through the compaction lifecycle hook rather than disabling Pi's safety path.

Long-horizon orchestration

The long-horizon-mea profile activates a reusable orchestration capability. The capability gives the parent session ask_user and the explicitly loaded pi-subagents tools, then runs three namespaced package-owned agents in order:

long-horizon.manager → long-horizon.executor → long-horizon.auditor

Each role starts with fresh context. Manager has no workspace tools. Executor receives Pi's built-in read, edit, write, grep, find, and ls plus the bounded bash; Auditor receives the built-in read-only tools plus the same bash. Their packaged child runtime applies the shared capability; no role-specific child tool extension is loaded. The parent repeats the cycle until an Auditor independently reports complete, clean, contract-aligned evidence.

Role definitions ship under .pi/agents/long-horizon/. At startup, the capability resolves that catalog relative to the installed @qwen-code/qwen-harness package, independently of the task workspace. Future harnesses can add their own subdirectory and unique package namespace under .pi/agents/. The Long Horizon Supervisor also registers an allowedAgents capability ceiling, so it cannot launch roles belonging to another harness even though Pi discovers the shared packaged agent catalog. Parent and child sessions remain separate so Harbor can capture them as related trajectories instead of flattening distinct tool contracts into one conversation.

The source tree keeps every atomic capability under one capabilities/ boundary, grouped by how it participates in the harness:

.pi/
  agents/
    long-horizon/   namespaced Manager, Executor, and Auditor definitions
src/
  capabilities/
    types.ts          common contract for composable harness capabilities
    settings.ts       loads Pi-owned settings and composes capability-owned settings
    global/
      session_affinity.ts  capability enabled for every harness
    orchestration/
      mea/            prompt, Supervisor lifecycle, round tracking, and role contract
    context-management/
      self-compact/   model-visible status, compaction rubric, and continuation lifecycle
    tools/
      settings.ts     composes tool-owned settings
      ask-user/       Pi tool adapter for the question workflow
      context-management/  model-visible context management tools
      execution/      tools, the bash capability, and their local settings parser
      filesystem/     model-visible workspace file tools
      media/          media tools and their local settings parser
      programmatic-tool-calling/  run_code adapter and isolated JavaScript child runtime
      web/            web tools and their local settings parser
  utils/
    capability_settings.ts  composes settings consumed by reusable utilities
    ask-user/         interactive question workflow, independent of tool schemas
    filesystem/       document extraction and its settings parser
    image-caption/    side-model perception and its settings parser
    settings.ts       shared parsing primitives
    workspace.ts      workspace confinement and common result formatting
    clients/
      settings.ts     composes external-client settings
      media/          DashScope and Pexels clients plus their settings parser
      retry.ts        provider-style HTTP retry shared by external clients
      web/            Routify clients and settings parser
  runtime/
    harness-capabilities.ts  merges peer capability contributions
    harness-main.ts          starts isolated and composed harnesses
  harnesses/          thin profile composition and Pi startup
  entrypoint.ts       selects and starts one harness

Capability directories follow seven harness dimensions by primary responsibility:

• E — Execution environment: runtimes, sandboxes, microVMs, browser or computer-use environments, and OS permission boundaries.
• T — Tool interface and protocol: tool description, discovery, selection, invocation protocols, tool-augmented training, and session interfaces. The current tools/ directory belongs here.
• C — Context and memory management: short-term context, session state, persistent memory, long-horizon context, and context-drift mitigation.
• L — Lifecycle and orchestration: agent loops, multi-agent control flow, state transitions, and end-to-end task pipelines. The current orchestration/ directory belongs here.
• O — Observability and operations: traces, costs, failures, reliability signals, and operational control planes.
• V — Verification and evaluation: benchmarks, controlled execution, judgement, failure attribution, regression feedback, and deployment-time evaluation.
• G — Governance and security: permissions, lifecycle hooks, component hardening, constitutions, audit, and organizational policy.

New capabilities should live under the dimension that owns their primary behavior. Reusable implementation belongs under utils/, and provider-specific network access belongs under utils/clients/, instead of being presented as independently activatable capabilities. Dimension directories are added when they gain a concrete capability rather than being created empty in advance.

capabilities/tools/ contains the Pi-specific exposure layer: tool names, TypeBox parameter schemas, descriptions, and result contracts. utils/clients/ owns vendor-specific transport and retry behavior. The rest of utils/ contains reusable implementations that are not independently exposed to the model. For example, the ask_user tool delegates to the reusable interactive-question workflow, while read_file composes document extraction, image captioning, and workspace confinement. Plain-text reads and filesystem search delegate their result limits, truncation, and continuation notices to Pi instead of defining harness-specific caps. Each capability-owned settings parser lives beside that capability; the root capabilities/settings.ts only loads Pi settings and composes the three category-level settings objects. Adding a capability changes its local parser and category composition, not the root loader.

Dependencies point inward: harnesses select capabilities/tools, tools may compose utils and utils/clients, and those lower layers do not import tool adapters. Both fetch_page and web_fetch use Routify; fetch_page adds its own 8,000-character chunking and cache.

Harness capabilities are peer contributions. They may append system-prompt sections, declare tools and extension paths, register inline extensions, and prepare packaged resources. A harness may still provide one base system prompt; normal capability prompts append after it, and orchestration prompts append last. Long Horizon uses this final phase so an existing harness prompt remains available as task and domain requirements while the Supervisor protocol controls how those requirements are delegated. The standalone long-horizon-mea harness is only a thin preset around createLongHorizonCapability(); another harness can compose that capability with its own peer capabilities without copying the Manager, Executor, Auditor, prompt, or runtime setup.

Sessions and resume

Sessions use Pi's native storage under ~/.pi/agent/sessions/, organized by working directory. Pi writes its append-only JSONL containing messages, tool calls/results, usage, model changes, and compaction entries.

npm run agent-arena -- --resume
npm run code-arena-react -- --continue
npm run code-arena-html -- --session /absolute/path/to/session.jsonl
npm run long-horizon-mea -- --resume

Headless modes use the same session behavior:

npm run agent-arena -- -p "First turn"
npm run agent-arena -- --continue -p "Continue"
npm run code-arena-full-stack -- --mode json "Build the application"

Validate

npm run check
npm test