dsh-rerun-llm-cache

Content-addressed model response cache, mounted INSIDE an attempt.

The key is the hash of the complete assembled request — route, sampling parameters, system prompt, tool schemas, and every message. A hit means the request was byte-identical to one already recorded, so replaying its response changes nothing about the experiment; a miss means the request genuinely differs and honestly costs a model call.

This is the opposite of dsh-llm-replay, which serves recorded responses by CALL POSITION. Positional replay is right for snapshot tests, where the script and the harness move together. It is wrong here: the moment a harness change alters a prompt, position N no longer answers request N, and the run would quietly measure a conversation that never happened.

What it buys: re-running a baseline is free and bit-identical; two variants share every request up to the point the change bites; strict mode makes a run provably offline and exactly repeatable.

What it does not buy: a candidate's FIRST run. A changed system prompt diverges at step zero.

Why the key normalizes paths

DSH's persona carries the working directory (Your working directory is {{cwd}}), and every attempt gets its own — <root>/<run>/<cell>/workspace. So without normalization the system prompt differs for every repeat and every run, every request misses, and none of the above is true.

The key therefore tokenizes the attempt's OWN paths — workspace, home, session root, run id, and their macOS /private spellings — before hashing. What makes that sound rather than convenient is that the set is CLOSED: only paths the framework itself minted for this attempt, read from the environment it set. It is not fuzzy matching, and there is deliberately no knob for adding patterns — anything else that varies between two attempts of the same cell is a real confound, and hiding it from the cache would leave it in the experiment.

Normalization affects the KEY only. The request that reaches the provider and the response recorded against it are untouched.

A run whose served rate is near zero says so at the top of its report, because that is the symptom of exactly this class of bug reappearing for a new reason.

- id: rerun-llm-cache
  name: 'dsh-rerun-llm-cache'
  config: { root: .rerun/cache, mode: read-write }

Model Experience

None of its own. It intercepts the llm/stream waterfall and either replays a recorded chunk sequence or delegates unchanged; it never adds to, removes from, or reorders a request.

KV Cache effect

None on the assembled request — the prefix the provider sees is untouched. On a cache HIT no provider request is made at all, so that step reports no usage, which is why an attempt's cacheHitRatio and this cache's hit rate are separate figures.

Known Limitations and Deferred Work

• A partial stream is never recorded, so a response truncated by a cancel or a network fault is re-fetched next time rather than made permanent. The cost is that a genuinely long stream interrupted near the end is paid for twice.
• The cache never expires. It grows with every distinct request, and nothing prunes it; the directory is plain files and safe to delete.
• Entries store complete request bodies. See SECURITY.md.
• Normalization knows only the paths rerun minted. A prompt carrying a timestamp, a hostname, or a path from somewhere else still misses every time. That is reported rather than papered over: the report's dead-cache caveat fires below a 20% served rate.