@touchtechclub/pi-oc-prompt-cache

Opencode-inspired prompt-cache optimizer for the Pi coding agent.

opencode v2 gets its famously high prompt-cache hit rate from one architectural idea: an immutable Context Epoch / Baseline System Context. The system prompt is rendered once per epoch and reused byte-for-byte on every turn — and across process restarts — so the provider's prompt-cache prefix never invalidates. When a context source actually changes, opencode appends a durable Mid-Conversation System Message instead of re-rendering the system prompt.

Pi already implements the wire-level half of prompt caching (cache_control breakpoints, prompt_cache_key, store: false, x-session-affinity). This extension adds the missing half.

What it does

1. Frozen Context Epoch — freezes the system prompt into a stable per-session baseline and returns it unchanged on every turn, so the cache prefix stays valid.
2. Durable baseline — persists the frozen baseline to the session, so it survives restarts and /reload instead of being re-derived.
3. Mid-conversation updates — when the live system prompt drifts (AGENTS.md edited, tool set changed, skills changed, ...), it keeps the frozen baseline and appends a durable update message with a line diff of what changed.
4. Deterministic tool ordering — sorts tool definitions by name in the provider payload (opencode sorts tools alphabetically; pi does not).
5. Tighter cache breakpoints — when pi's built-in caching is active, extends Anthropic cache_control breakpoints to the last two messages (opencode's placement) rather than just the last one.

Install

pi extension add @touchtechclub/pi-oc-prompt-cache

Or add to settings.json:

{
  "extensions": ["@touchtechclub/pi-oc-prompt-cache"]
}

Behavior notes

• Chain-safe. It mutates payloads in place per Pi's hook contract, only adds cache hints (never removes a user's PI_CACHE_RETENTION=none opt-out), and freezes rather than clobbers the chained system prompt.
• Load it last if you also use extensions that append to the system prompt — its freeze then covers their mutations too, keeping the final prompt stable.
• Compaction starts a new epoch. After /compact or auto-compaction, the next turn re-freezes the current prompt as the new baseline (matching opencode).
• The injected update message uses customType pi-prompt-cache-update and is rendered in the transcript; the frozen baseline is stored as a hidden custom entry (pi-prompt-cache-epoch, not sent to the LLM).

Benchmark

bun bench drives pi's real request builders (@earendil-works/pi-ai) against the configured providers and measures prompt-cache hit rate on the second turn of three arms:

• steady — no context change (pi's own ceiling).
• drift — a context line changed and the system prompt rebuilt (pi without this extension).
• frozen — the same change, but the baseline is frozen and the delta is appended as a mid-conversation message (this extension).

Measured on ClinePass (api.cline.bot, OpenAI-compatible):

| model | steady | drift (no ext) | frozen (ext) | improvement | |---|---|---:|---:|---:| | cline-pass/deepseek-v4-pro | 93.9% | 0.0% | 96.7% | +96.7pp | | cline-pass/deepseek-v4-flash | 81.5% | 0.0% | 89.6% | +89.6pp | | cline-pass/kimi-k3 | 90.7% | 0.0% | 88.3% | +88.3pp |

When the system context changes, rebuilding the prompt drops the cache hit rate to 0%; freezing the baseline keeps it at ~88–97%. The steady column shows pi already caches well (~82–94%) when nothing changes — the extension's job is to preserve that across context drift and restarts.

License

MIT