PR Replay

PR Replay turns real pull-request histories into portable, repeatable coding agent scenarios. A bundle pins a repository snapshot, reconstructed starting request, hidden review context, acceptance contract, and reproducible setup. The runner gives that bundle to a supervisor and drives an unmodified coding agent through the change while recording timings, tokens, tool activity, validation, and a resumable trace.

PR Replay is a standalone project. Its public interface includes:

• a generic CLI for capturing, validating, inspecting, and replaying bundles;
• adapters for standard agents such as Codex, Claude Code, and OpenCode;
• a direct Codex app-server adapter for richer event-level experiments;
• an optional OpenTUI client for live and human-steerable playback;
• JSONL traces and comparative reports that do not require the UI.

Private PR data does not belong in this repository. Keep private bundle collections in a separate directory or repository and supply them to the CLI by path. Curated public example bundles may live here when they are part of the documented user experience.

Start with the public GitHub examples: one human-reviewed replay from ai-dynamo/dynamo (the current home of nvidia/dynamo) and one from openai/codex. Both pin exact historical states, keep the review correction hidden from the worker, and have independently validated accepted patches. The same guide includes exact recapture, replay, matrix, trace, and report commands.

The synthetic Relayboard webhook-retry bundle is also available when you want to inspect the complete artifact shape without fetching a source repository.

Because bundles and coding agents execute repository code, read the threat model before using data you do not fully trust.

Status

Working prototype. Schema version 1, GitHub capture, base-only preparation, supervised runs, resumable matrices/reports, trace loading, and the OpenTUI client are operational. Standard Codex exec and Claude Code adapters have been exercised end to end. The OpenCode adapter follows the current documented JSON session contract but has not yet been runtime-verified end to end. Interfaces may still change before a stable release.

Development

The project uses a pinned Bun/TypeScript toolchain:

npm install
npm run check
npm test
npm run runtime:build
npm run pr-replay -- validate --bundle /path/to/bundle

For a first end-to-end run, follow examples/README.md. The shortest path validates and prepares the public Dynamo example, runs it with a supervised Codex worker, then replays the resulting trace or compares several worker strengths in a resumable matrix.

pr-replay view --trace PATH accepts either the original JSONL trace or a lossless Zstandard-compressed .zst file. Compressed traces load directly in both the headless view and the OpenTUI interface without a manual extraction step.

The runtime image supplies the common native build toolchain and libraries used by isolated workers and validation. --build-arg BASE_IMAGE=... can extend an existing compatible image; the default is the public rust:1.95-bookworm image. Repository-specific dependencies still belong in each bundle's setup.

Run a bundle with the standard Codex CLI while preserving normal Codex user configuration, MCP, hooks, and observability:

npm run pr-replay -- run \
  --bundle /path/to/bundle \
  --adapter codex-exec \
  --model gpt-5.4-mini \
  --supervisor-model gpt-5.6-sol \
  --mode auto

Codex exec uses its own workspace sandbox by default. Only callers already inside a separately enforced container or VM should add --codex-outer-sandboxed; that option deliberately invokes Codex's dangerous sandbox-bypass flag and is never enabled implicitly.

The richer codex-app-server adapter and OpenTUI client remain available for controlled experiments and live/interactive playback.

The isolated app-server worker can receive an explicit local skill without copying the rest of the caller's Codex home:

npm run pr-replay -- run \
  --bundle /path/to/bundle \
  --worker-skill /path/to/review-skill

--worker-skill is repeatable. Each source must be a real directory with a root SKILL.md; symlinks, special files, unsafe names, and duplicate directory names are rejected. Skills are copied only to the worker's isolated CODEX_HOME, never the supervisor's, and their deterministic content digests are recorded in the trace. The option deliberately fails with standard agent adapters, which retain their own native skill configuration instead.

External harnesses that create dispatch commits can preserve them before a matrix removes its trial checkout:

npm run pr-replay -- matrix \
  --experiment dispatch-study \
  --bundles /path/to/bundles \
  --export-dispatch-refs

This opt-in exports only direct commit refs named refs/dispatches/v1/session/<safe-session-id>. PR Replay verifies the bundle and its heads, writes artifacts/dispatch-refs-v1.bundle plus a SHA-256-bearing artifacts/dispatch-refs-v1.json manifest in each run directory, and then cleans the workspace. An empty namespace produces only the manifest. The programmatic ReplayRunOptions.artifactExport.onExport hook is awaited after agent shutdown and ownership restoration, while the checkout still exists. Neither feature changes default run or matrix behavior.

Capture a merged pull request with the local GitHub and Git credentials. Setup and validation stay explicit because inferring a development environment from CI is neither portable nor reliably safe:

npm run pr-replay -- capture github \
  --repo owner/repository \
  --pr 123 \
  --output /path/to/bundles/repository-pr-123 \
  --setup-command "npm ci" \
  --validation-command "npm test" \
  --success-contract "The test suite passes and the reviewed behavior is present."

When --prompt-file is a reconstruction rather than a known original request, set --prompt-confidence medium or low so the bundle records that uncertainty. For a review journey that ended before the PR's final commit, use --target-commit with --evidence-cutoff to pin the accepted state and exclude later discussion. Add --exclude-bots when the replay should preserve only the human conversation.

The capture includes the accepted binary patch, full visible issue/review discussion, commit sequence, and accepted-head checks. Only prompt.md is given to the worker. Validation commands are required, and setup environment values are stored in plaintext in the bundle, so credentials must not be put in --setup-env.

The same runner can drive installed Claude Code and OpenCode clients without reimplementing their tools:

npm run pr-replay -- adapters
npm run pr-replay -- run --bundle /path/to/bundle --adapter claude-code --model claude-sonnet-4-6
npm run pr-replay -- run --bundle /path/to/bundle --adapter opencode --model openai/gpt-5.4-mini

Claude Code's permission bypass is likewise only enabled by the explicit --claude-outer-sandboxed flag. OpenCode support is implemented against its JSON run/session contract; pr-replay adapters reports whether each client is actually available before a matrix is started.

OpenCode's permission bypass is only enabled by --opencode-outer-sandboxed; without that flag its configured permission policy remains in force.