dsh-rerun-grader

Service Definition for the grader seam: the registry of scoring providers and the one place the weighting rule lives.

A grader is a pure function of what the attempt left behind — the final workspace and the harvested session log — and never of how the attempt was produced. That is what makes rescoring possible: a grader added months later can be run against stored artifacts without re-running anything.

Pass and score are deliberately separate. passed is the conjunction of every grader's own verdict, so a task with a passing test suite but a violated invariant does not pass; score is the weighted mean, which carries partial credit for trend lines.

- id: rerun-graders
  name: 'dsh-rerun-grader'

Provides ctx.rerunGraders. Writing a grader: extending.md.

Model Experience

None. This package never contributes to a model request: it runs in the orchestrator process, and the model calls it cares about happen inside a separate attempt subprocess running a different build of DSH entirely.

KV Cache effect

None; it neither assembles nor sends a provider request.

Known Limitations and Deferred Work

• Graders run sequentially per attempt. A task with several slow command graders pays their sum, not their max.
• rescorable is honor-system. A grader that claims it and then reads cwd will score every historical attempt against an empty directory; nothing enforces the claim.