dsh-rerun
v0.1.2
Published
The dsh-rerun orchestrator bundle: drives the task x variant x repeat matrix as isolated attempt subprocesses and writes the JSON report
Maintainers
Readme
dsh-rerun
The orchestrator and the profile bundle — the package users install.
dsh plugin --profile rerun add dsh-rerunOne app with five modes. run and aa execute the task x variant x repeat
matrix as isolated attempt subprocesses; rescore, explain, and trend are
read-only views over what the store already holds. They share one process
because they share one configuration — the suite, the variants, and the
baseline that define what "comparable" means here.
Each attempt is a subprocess, and that is load-bearing rather than incidental: it is what lets two variants be different BUILDS rather than only different compositions, it contains a hung or crashed harness inside one cell instead of one run, and it keeps the orchestrator's context from sharing anything with the context under measurement.
The bundle's cordis.patch.yml composes over an EMPTY
root, not over dsh-base. The orchestrator runs no agent of its own and
registers no tools; mounting the product spine here would put a second,
unmeasured harness in the room.
Model Experience
None. This package never contributes to a model request: it runs in the orchestrator process, and the model calls it cares about happen inside a separate attempt subprocess running a different build of DSH entirely.
KV Cache effect
None; it neither assembles nor sends a provider request.
Known Limitations and Deferred Work
- A git ref is not resolved to a build. A variant names an executable that
already exists; turning
HEAD~1into a runnable install is CI's job, and the shipped workflow does it. - Turn and token budgets are detected, not enforced. Only the wall clock actually stops a process. The other two ceilings reclassify an attempt after its log is folded, because neither is observable from outside the process.
- An interrupted run cannot be resumed. Attempt records are written as they finish, so nothing measured is lost, but the remaining cells must be re-run.
