pi-for-k3

Kimi K3 prefix-cache optimizer for the pi coding agent

Byte-stable request prefixes · per-turn miss attribution · K3-tuned system-prompt constraints

~43% cheaper than bare pi and ~56% cheaper than Kimi Code at equal resolve (SWE-bench Verified Mini)

Kimi K3 bills cached prompt tokens at a tenth of the normal price — but only if every request keeps the exact same byte prefix. One shuffled tool, one rewritten message, and you silently pay full price again.

pi-for-k3 is a single-file pi extension (k3-cache.ts) that keeps you on the cheap side. It watches every request, keeps the prefix byte-stable, tells you exactly why any cache miss happened, and makes cost-aware calls on things like compaction and context-window upgrades. It works on both K3 channels (Moonshot pay-per-use API and the Kimi Code subscription), and if you're on any other model it simply stays out of the way.

Contents — Install · System-prompt constraints · Commands · Configuration · Guarantees

Install

pi install npm:pi-for-k3   # or: pi install git:github.com/WuP1ao0/[email protected]
pi -e git:github.com/WuP1ao0/pi-for-k3               # try without installing

System-prompt constraints (K3-specific)

K3 is a strong model with one documented bad habit: it does too much when it isn't sure. Kimi's own docs recommend reining it in with explicit behavioral constraints, so on K3 sessions this extension appends a short <k3-cache-constraints> block to pi's system prompt. It never touches pi's own prompt text, and the block is byte-identical every turn, so the cache stays safe.

The block draws four simple lines:

• Scope — fix what the issue asks for; don't refactor, rename, or "improve" things nearby.
• Completeness — before finishing, check whether the same behavior lives elsewhere (a subclass, a parallel handler) and fix all of it.
• Verification — once something passes, leave it alone unless you can name a concrete failing case.
• Economy — don't dig through the whole git history; the issue text and current code are enough.

These aren't just vibes — in an A/B test on a task that baits the model into git archaeology, the full block solved it in 14 turns for $0.25, while dropping the Economy line ballooned the same task to 40 turns and $0.53.

Commands

| Command | What it does | |---|---| | /cache | Shows how your session is doing: hit rate, actual cost vs what you'd pay with no cache, any drift alerts, and the reason behind every miss | | /effort <level> | Changes the reasoning effort. Asks you to confirm first, because Kimi's docs say the server cache rebuilds after a switch | | /k3upgrade | On the subscription: jumps from k3-256k to k3 for the 1M window. Your cache survives the jump (an official exception), but quota burns about twice as fast |

Configuration

Everything is tunable. Drop overrides into ~/.pi/agent/k3-cache.json — you only need the keys you want to change:

| Key | Default | What it controls | |---|---|---| | enabled | true | The master switch. Set false and the extension does nothing at all | | hitRateAlertThreshold | 0.7 | If a turn's hit rate falls below this, you get a warning with the reason | | segmentSizeChars | 16384 | How finely the prefix is hashed for drift detection (~4K tokens per segment) | | idleTtlMs | 300000 | If you've been idle longer than this, the next miss is blamed on server-cache expiry rather than something you did | | defaultEffort | high | Reasoning effort to lock in when pi doesn't have one set | | subscriptionWarnFraction | 0.8 | Warns you when context grows past this share of the 256K window | | overflowUpgrade / overflowAutoContinue | true | When k3-256k overflows, switch to k3 (1M) instead of compacting — your cache survives an upgrade but not a compaction. Also auto-resumes the interrupted turn | | highWaterFraction | 0.85 | Holds off pi's automatic compaction until context is really full, since every compaction throws away the cached prefix | | summaryChainEnabled / MaxLayers / MaxTokens | true/5/8000 | When compaction does happen, add a new summary layer instead of rewriting old ones (and when to merge layers) | | prewarmEnabled / prewarmIdleMs | true/300000 | After a long break, quietly re-warm the server cache with one tiny request so your first real turn doesn't pay the miss | | externalizerEnabled | false | Park oversized tool results on disk and let the model fetch them back with a read_result tool. Off by default; the gentler head/tail truncation always runs | | externalizeMinChars / onDemandTools | 8192/["bash"] | Only touch a tool result if it's bigger than this AND comes from one of these low-reuse tools | | toolChangeAllowlist | [] | Tools that are allowed to appear and disappear mid-session without a warning. Globs work: "goal_*" | | compactionOwnership | "k3cache" | Set "external" if another extension manages compaction and this one should stay out of it | | k3Models | catalog map | Which provider/model ids count as K3. Everything else is ignored |

Guarantees (cache discipline)

A few promises about how this extension behaves:

• It never rewrites your history. Messages and payloads are left untouched; the only in-place change is putting tools in a stable order. Everything else is watching and warning.
• Tool-set changes don't spam you. A removed or reshaped tool gets flagged once per distinct change. If the same change keeps recurring (some extensions toggle tools by design), you get a ready-to-paste toolChangeAllowlist line instead of endless alerts.
• Every expensive turn comes with a reason. Misses are labeled: first turn, prefix drift, model switch, idle expiry, compaction, branch navigation, system-prompt rebuild, tool change, or a genuine server-side miss.
• Drift alerts point at the culprit. Not just "something changed" — you're told which message, tool, or prompt section diverged.
• Sessions don't leak into each other. State is kept per session, so subagent runners sharing one process can't trigger false alarms. /cache also shows a combined total when several sessions are active.