dynamic-context-pruning

A pi extension that automatically prunes stale, duplicate, and superseded tool-call content from long-running sessions, gated by a cache-aware net-benefit calculation so pruning only happens when it's actually worth the prompt-cache cost. Includes a manual /prune picker and a /context-pruning status/control command.

This is v1: deterministic, non-agentic strategies only (no model-invoked compression, no summarization). See Roadmap for what a v2 would add and why it hasn't been built yet.

Design

• Non-destructive. Nothing is ever deleted from the session file. A prune replaces a tool result's content (or a tool call's input arguments) with a short placeholder string; the original message stays in the transcript and can always be restored.
• Branch-aware. Prune decisions and restores are persisted as custom session entries (dynamic-context-pruning:decision, dynamic-context-pruning:stats, dynamic-context-pruning:restore) appended via pi.appendEntry. On session_start and session_tree, the extension folds the current branch's entries back into in-memory state, so forking/switching branches correctly recomputes which decisions are active for that branch.
• Tombstone-based restore. Restoring a pruned result doesn't delete the original decision entry — it appends a restore entry for that decision's idempotency key. The most recent entry (by chronological order) for a given key wins when replaying history, so a decision can be pruned, restored, and re-pruned any number of times.
• Recomputed every call, not on a timer. All three automatic strategies run fresh on every context event, over the current message array. Nothing is pruned "once and forgotten"; if a tool result would no longer qualify (e.g. it's now within the protected recency window), it isn't proposed again, but an already-applied, persisted decision still applies until explicitly restored.

Strategies

Three deterministic strategies propose prunes; each can be toggled independently.

1. Deduplication (dedupe)

If the same tool is called more than once with canonically identical arguments, every occurrence except the newest is proposed for pruning (its tool result content is replaced with a placeholder pointing at the newest call).

• Canonicalization: JSON keys are recursively sorted before hashing/comparing ({"b":1,"a":2} and {"a":2,"b":1} are the same key). Tool name comparison is case-insensitive.
• Only completed tool calls (an assistant toolCall block with a matching toolResult message) are considered; in-flight calls are never touched.
• The newest occurrence in a duplicate group is always kept in full.

2. Error-input purge (error-purge)

Once an errored tool call's input arguments are older than a configurable number of turns, they're proposed for pruning (the call's arguments are replaced, not the error message itself).

• Default: minTurnsOld = 4 — a call must be strictly older than 4 turns to qualify (eligible at 5+ turns elapsed, not at exactly 4).
• Only applies to tool calls whose result was isError: true.
• Configurable per session via strategy error-purge on|off and the strategies.errorPurge.minTurnsOld config field.

3. Superseded file operations (superseded-file-ops)

When the same file path is read and/or written more than once, older read/write/edit tool outputs for that path can become stale. This strategy is deliberately conservative — it only proposes a prune when it can prove the older output is stale, never on a guess:

1. A full-file read (no offset/limit) is superseded by a later full-file read of the same normalized path.
2. A partial read (has offset/limit) is not superseded by a later partial read unless the later read's range fully covers the earlier one. If range coverage can't be reliably determined from the arguments (e.g. the earlier read has no limit — reads to EOF — but the later read is bounded), the strategy skips it rather than guessing.
3. Any read of a path (full or partial) is superseded once a later, successful (non-error) write/edit to that path occurs — the old read is now known-stale regardless of range comparisons. An errored later write/edit does not count, since the file may be unchanged.
4. Older write/edit outputs for a path are superseded by a later, successful write/edit to that same path. An errored later write/edit never supersedes.

The newest operation for a given path (by message order) is never superseded, regardless of kind. Path extraction accepts path, file_path, and filePath argument names, matched only against pi's built-in read/write/edit tools; bash file writes are out of scope (this strategy does not parse shell commands).

Protections

Before any strategy is allowed to propose a prune, several protections apply:

• Protected tool names (case-insensitive, never pruned): todo, task, subagent, agent, skill, oracle, librarian, contrarian, code_reviewer, gnosis, triage_comments.
• Protected path globs (never pruned if any string argument matches): **/.env, **/.env.*, **/*.pem, **/*.key, **/id_rsa*, **/id_ed25519*, **/secrets/**, **/*.p12, **/*.pfx.
• Recency window: the most recent recentTurns conversational turns (default 4) are never touched by any automatic strategy, regardless of what they contain.

All three lists/values are configurable (see Configuration).

Savings accounting and the net-benefit gate

Every applied decision's estimated token savings are folded into cumulative, per-strategy stats (/context-pruning stats), persisted as dynamic-context-pruning:stats session entries.

Before a fresh, automatic prune proposal is actually applied, it must pass a cache-aware net-benefit gate. The intuition: prompt caches are prefix-matched, so pruning at message position p invalidates ("busts") the cached prefix from p onward, exactly once. That's a one-time cost, offset by a smaller recurring saving on every subsequent call (fewer tokens sent). The gate models this as a break-even calculation:

penalty          ≈ (1 - r) * tailTokensAfterEarliestChange
recurringSaving  ≈ r * tokensRemoved
breakEvenCalls   =  penalty / recurringSaving

Where r is gate.cachedPriceRatio — the fraction of full price still paid for cached tokens (default 0.1, i.e. cached tokens cost ~10% of fresh tokens). breakEvenCalls is how many subsequent LLM calls are needed to recoup the one-time cache-bust cost via the recurring saving. The gate accepts a batch of candidates only if breakEvenCalls <= gate.breakEvenThreshold (default 22, calibrated from the representative-corpus benchmark — see the calibration note under Configuration and the Roadmap evidence).

All candidates proposed in the same context call share one cache bust (since a prompt cache is a single linear prefix), so they're evaluated and accepted/rejected jointly against the earliest position any of them touches.

Manual prunes via /prune bypass the gate entirely — you already decided the result isn't worth keeping, so there's no threshold to check.

Gate modes (/context-pruning gate <mode>):

• on (default): reject batches above the threshold.
• off: bypass the gate entirely; no cost modelling, everything applies.
• always-apply: cost is still modelled (for stats/observability) but nothing is ever rejected.

Commands

/prune

Interactive picker (TUI/RPC) over every prunable tool result in the current branch:

/prune

• Lists active, pruned, and previously-restored tool results.
• Selecting an active result shows a cost preview (predicted cache-bust penalty vs. recurring saving) before you confirm the prune.
• Selecting a pruned result offers to restore it.
• Outside an interactive UI (no ctx.hasUI), prints a read-only report of prunable results instead.

/context-pruning

/context-pruning status
/context-pruning stats
/context-pruning on
/context-pruning off
/context-pruning toggle
/context-pruning strategy <dedupe|error-purge|superseded-file-ops> on|off
/context-pruning gate <on|off|always-apply>

• status: enabled state, gate mode/threshold/ratio, per-strategy on/off state, protection counts, last call's raw/effective/saved token snapshot, and current context usage.
• stats: cumulative tokens saved, overall and per strategy.
• on / off / toggle: enable or disable the extension entirely.
• strategy <name> on|off: toggle one strategy independently.
• gate <mode>: change the net-benefit gate mode at runtime.

Config changes made via /context-pruning are persisted to disk (see Configuration) and also applied in-memory immediately, even if the write fails.

Configuration

Config is stored at:

<pi agent dir>/extensions/dynamic-context-pruning.json

It's read once on session_start and rewritten whenever /context-pruning mutates it. A missing or unparseable file falls back to defaults; the file is never required to exist. Full shape, with defaults:

{
  "enabled": true,
  "protections": {
    "toolNames": ["todo", "task", "subagent", "agent", "skill", "oracle", "librarian", "contrarian", "code_reviewer", "gnosis", "triage_comments"],
    "pathGlobs": ["**/.env", "**/.env.*", "**/*.pem", "**/*.key", "**/id_rsa*", "**/id_ed25519*", "**/secrets/**", "**/*.p12", "**/*.pfx"],
    "recentTurns": 4
  },
  "thresholds": {
    "minCharsSaved": 200
  },
  "strategies": {
    "dedupe": { "enabled": true },
    "errorPurge": { "enabled": true, "minTurnsOld": 4 },
    "supersededFileOps": { "enabled": true }
  },
  "gate": {
    "mode": "on",
    "cachedPriceRatio": 0.1,
    "breakEvenThreshold": 22,
    "breakEvenThresholdByState": { "idle": 22, "mid_loop": 22 }
  }
}

Notes:

• thresholds.minCharsSaved is a minimum-size floor enforced against fresh automatic strategy proposals only, before the net-benefit gate runs: a text-only candidate whose raw character savings (original text chars minus placeholder chars) fall below this value is dropped up front and never reaches gate or batch consideration. A tool-result candidate containing one or more image blocks bypasses this raw-text floor (images do not receive synthetic character counts) and is evaluated by the existing token-based net-benefit gate. Manual prunes (/prune) and already-persisted/replayed decisions always bypass this floor, same as the gate. Set to 0 to disable filtering entirely. This is independent of the net-benefit gate's break-even math (gate.*), which is a separate, later check on whatever survives this floor.
• gate.breakEvenThresholdByState lets idle vs mid_loop agent states use different thresholds; both default to gate.breakEvenThreshold. Real mid-loop/idle detection is wired through the context event handler (pe-zy4s): the state is classified straight from the message payload — idle if the most recent relevant message is a user message (this is the first LLM call of the turn), mid_loop if it's an assistant message or a tool result (the agent is iterating mid-turn). The representative-corpus benchmark (see Roadmap) was re-derived against this same runtime-observable definition and found the state-split REVERSED from an earlier turn-end-based analysis: at r=0.1, idle candidates now carry essentially all the realized net benefit and mid_loop candidates carry essentially none. Since idle is not the worthless state under this definition, both states are kept at parity (22/22) rather than forcing a stricter default in either direction.
• gate.breakEvenThreshold's default of 22 is calibrated from the representative-corpus benchmark at cachedPriceRatio r=0.1 (aggressive prompt caching, the common case) — see Roadmap for the full evidence and its ratio-sensitivity caveats. It is provider/ratio- dependent: the optimal threshold rises as caching gets weaker (T=29 at r=0.25, T=54 at r=0.5, T=58 at r=0.9 on the same corpus).

Prompt-cache trade-off

Pruning trades a smaller stream of tokens sent on every future call against a one-time cache-bust cost, because prompt caches match on a strict prefix: any change to the message stream busts the cached prefix from that point onward for the next call. This is exactly why the net-benefit gate exists rather than pruning unconditionally — a prune that happens too late in a session (few calls left to amortize the one-time cost over) or removes too few tokens relative to the tail it invalidates can cost more than it saves. The gate's break-even math (above) is the mechanism that keeps pruning net-positive on average rather than accepting the cache trade-off unconditionally.

gate.cachedPriceRatio (r) should track your provider's real prompt-cache pricing:

• Anthropic prompt caching: cached reads ~0.1x base input price, plus a ~1.25x one-time cache-write premium → r ≈ 0.1.
• Google Gemini context caching: ~0.25x base input price, plus a separate storage/time fee → r ≈ 0.25.
• OpenAI automatic prompt caching: ~0.5x base input price → r ≈ 0.5.
• No caching available or enabled → r → 1: the cache-bust penalty vanishes, so pruning is a pure win regardless of threshold.

Provider pricing drifts — check current price sheets before trusting these ratios. The penalty math also assumes the cache would still be warm at the next call; if the provider's cache TTL (e.g. ~5 min default on Anthropic) elapses between calls, the bust costs nothing and pruning is again a pure win.

Benchmark harness

A standalone, read-only offline benchmark replays real (or fixture) pi session .jsonl files through the extension's exported strategy/pipeline/cost model helpers — it never writes back to a session file or changes runtime behavior.

node extensions/dynamic-context-pruning/scripts/benchmark.mjs [paths...] [options]

Options:

• --limit N: only process the first N session files found.
• --ratio R: cached-price ratio(s) to model; repeatable and/or comma-separated (e.g. --ratio 0.1,0.2). Default: 0.1.
• --json: emit a full machine-readable JSON dump instead of aligned text.
• --simulate-compression: additionally SIMULATE v2-style range compression (docs/v2-design.md §1/§4 go/no-go evidence) alongside — never mixed into — the deterministic results. See below.
• --sim-summary-fraction F: summary size as a fraction of range tokens; repeatable/comma-separated (e.g. --sim-summary-fraction 0.1,0.15,0.3). Default: 0.15.
• --sim-summary-min-tokens N: floor on summary tokens regardless of fraction. Default: 200.
• --sim-summarizer-cost-mult M: relative per-token price of the summarizer call vs. the main model. Default: 1.0 (same per-token price; a deliberate simplification that ignores summarizer latency/availability).
• --sim-min-range-tokens N: minimum contiguous range size to be considered. Default: 2000.
• --help: print usage.

Compression-simulation mode (--simulate-compression)

This is a v2 go/no-go evidence generator that does not require building v2: it identifies plausible v2-style compress (range mode) opportunities — contiguous spans of complete, non-recent, non-protected messages, never splitting a toolCall/toolResult pair — during the same replay pass, and models the outcome:

summaryTokens      = max(fraction * rangeTokens, summaryMinTokens)
oneTimeCost        = cacheBustPenalty(earliest range position)
                       + summarizerCostMult * (rangeTokens + summaryTokens)
recurringSaving     = r * (rangeTokens - summaryTokens)
realizedNetBenefit  = actualRemainingCalls * recurringSaving - oneTimeCost

Each distinct range is recorded once, at the earliest call boundary where it first exists and meets --sim-min-range-tokens; ranges never overlap. The same auto-expanding break-even threshold sweep as the deterministic strategies runs against this separate SIMULATED population, reported side-by-side with (but never combined with) the deterministic results — the two populations are computed independently; a simulated range may fully contain deterministic candidates, and no overlap/interaction between the two is modeled or netted out.

With no positional paths, it defaults to every *.jsonl file found recursively under ~/.pi/agent/sessions.

Corpus choice matters. ~/.pi/agent/sessions (the default) tends to be dominated by short orchestrator/delegation sessions, which structurally under-report candidates and skew the remaining-calls-after-position distribution low. The representative-corpus numbers in Roadmap instead come from ~/.the-last-harness/agent/sessions — the harness's own longer, tool-heavy agent sessions — which is the corpus to point the harness at when re-deriving or sanity-checking the gate default.

For every point in a session where an LLM call would have happened (approximated as immediately before each assistant message), the harness reports two kinds of numbers:

• Predicted: what the strategies would propose right now, and what the cache-aware cost model (penalty / recurring saving / break-even calls) estimates, without knowing what happens next in the session.
• Realized: since replay knows the session's actual future, the harness also reports how many subsequent LLM calls actually followed a candidate prune point, and the actual net benefit that prune would have produced — realizedNetBenefit = actualRemainingCalls * recurringSaving - penalty.

It also sweeps the break-even threshold T from 1 to 30 and reports the T that maximizes total realized net benefit across all gate-eligible candidates, both overall and split by idle/mid-loop agent state.

Comparison

• opencode-dynamic-context-pruning ("pi-dcp" upstream): the OpenCode plugin this extension takes its name and general problem space from. It ships a model-invoked compress tool (agentic, nested-summary compression), autonomous nudges, manual mode, and a full /dcp command surface. This extension's v1 implements none of that agentic layer — only the deterministic automatic strategies and cache-aware gate, which is the subset OpenCode DCP calls its own automatic dedup/error-purge strategies. See Roadmap for the full parity gap and what would be needed to close it.
• context-cap / context-inspector: siblings in this repo that address adjacent but different problems. context-cap shrinks the effective model context window so pi's own auto-compaction fires earlier; it doesn't remove or alter any content. context-inspector is a read-only dashboard that shows where context is going; it doesn't modify anything either. dynamic-context-pruning is the only one of the three that actually removes/replaces content from what gets sent to the model.

Roadmap

docs/v2-design.md is the authoritative v2 design spec: full OpenCode DCP parity, meaning a model-invoked compress tool, nested block summaries, autonomous nudges, manual mode, and the full /dcp- equivalent command surface (decompress/recompress, prompt overrides, notifications).

v2 proceeds only if benchmark evidence supports it. The design doc's benchmark decision gate requires a meaningfully positive aggregate realized net benefit, at sufficient candidate volume, with enough remaining-calls runway per session to plausibly amortize a compression summary's own token cost — see docs/v2-design.md §4 for the exact go/no-go bar.

The representative-corpus run of the benchmark harness (pe-c5n9, over ~/.the-last-harness/agent/sessions: 1,390+ session files, 556 gate-eligible candidates — see the corpus-choice note above) produced:

• Candidates and tokens removed by strategy: dedupe 111 candidates / 93,330 tokens removed; error-purge 187 / 28,191; superseded-file-ops 258 / 247,474.

• Remaining-calls-after-position distribution: p50 = 47, p90 = 156 — far more amortization runway than the earlier small-corpus run showed, because this corpus is dominated by long, tool-heavy agent sessions rather than short orchestrator sessions.

• Hindsight-optimal break-even threshold T (maximizes total realized net benefit), by cached-price ratio r, split overall / mid_loop / idle:

  | ratio r | overall T (benefit) | mid_loop T (benefit) | idle T (benefit) | | --- | --- | --- | --- | | 0.1 | T=22 (~20.6k) | T=22 (~20.6k) | T=1 (0 — zero benefit) | | 0.25 | T=29 (~1.54M) | T=29 (~1.54M) | T=12 (~4.0k) | | 0.5 | T=54 (~8.06M) | T=54 (~8.06M) | T=27 (~37.2k) | | 0.9 | T=58 (~24.7M) | T=58 (~24.2M) | T=21 (~459k) |

Reading: mid_loop candidates carry essentially all of the realized benefit; idle candidates carry essentially none (literally zero at r=0.1). At r=0.1 (aggressive prompt caching, the common Anthropic case) the total realized benefit across the whole 1,390-session corpus is only ~20.6k token-units — economically marginal, on the order of pennies. Savings only become material at weaker caching, r>=0.25.

pe-zy4s update (2026-07-08): this idle/mid_loop split used a turn-END definition ("idle" = this call IS the turn's final assistant message, only knowable in hindsight via replay) that was never runtime-observable. Once real runtime agent-state detection landed (turn-START definition: "idle" = this is the FIRST LLM call of a turn; see the config-notes bullet above), the benchmark's candidate labeling was aligned to that same definition and re-derived on the representative corpus (460 gate-eligible candidates on the current ~/.the-last-harness/agent/sessions corpus). The split reverses: at r=0.1, idle (T=22, ~20.6k realized net benefit) now carries essentially all the value, and mid_loop (T=1, ~0 realized net benefit) carries essentially none. Per the decision rule (stricter idle default only if idle is shown ~worthless), this evidence does not support a stricter idle default — parity (22/22) is kept. The table above is retained as the historical turn-END-definition record; it no longer reflects how the runtime classifies agent state.

This reframes, rather than weakens, the case for v2. Small deterministic removals (dedupe/error-purge/superseded-file-ops) structurally cannot beat the cache-bust penalty at r=0.1 — there just isn't enough single-message token volume in them. If pruning is going to matter economically under aggressive caching, it has to come from large removals: agentic range compression and nested summaries (the v2 compress tool), which remove orders of magnitude more tokens per operation than these deterministic strategies do. So this evidence argues for prototyping v2 (contingent on summary quality/hardening work), not against it — v1's mechanical strategies remain useful as a free, always-on baseline, but they were never going to be where the real savings live.

Install

Standalone npm package

pi install npm:pi-dynamic-context-pruning

Collection package

pi install npm:@diegopetrucci/pi-extensions

GitHub package

pi install git:github.com/diegopetrucci/pi-extensions

Then reload pi:

/reload

Notes

• This extension's package name is intentionally unscoped (pi-dynamic-context-pruning, not @diegopetrucci/pi-dynamic-context-pruning), a deliberate deviation from this repo's usual package-naming convention.
• Token counts throughout (status, stats, the benchmark harness) are estimates from a simple chars/4-style heuristic, not exact provider token counts.