review-quill

review-quill is a self-hosted GitHub PR review bot. It watches configured repositories, materializes the exact PR head SHA in a throwaway checkout, builds the review context locally, runs a read-only Codex review pass, and publishes a normal GitHub PR review with its GitHub App identity.

It fits alongside:

• patchrelay for delegated implementation and PR upkeep
• merge-steward for queue admission and merge execution

You can run review-quill on its own. PatchRelay is not required.

What It Does

For each eligible PR head, review-quill:

1. detects that a new reviewable PR head exists
2. materializes an ephemeral local checkout of that exact SHA
3. builds a curated diff against the PR base branch
4. loads repo review guidance such as REVIEW_WORKFLOW.md, CLAUDE.md, and AGENTS.md
5. runs a review pass through codex app-server
6. publishes an ordinary GitHub APPROVE or REQUEST_CHANGES review
7. cancels stale in-flight attempts when a newer PR head lands first

This keeps review grounded in the real repository state instead of only the GitHub files API.

Quick Start

1. Install

npm install -g review-quill

2. Bootstrap the local home

review-quill init https://patchrelay.example.com/review

That creates:

• ~/.config/review-quill/runtime.env
• ~/.config/review-quill/service.env
• ~/.config/review-quill/review-quill.json
• /etc/systemd/system/review-quill.service

3. Configure GitHub access

Quick-start path: put the non-secret GitHub App id in service.env and keep the webhook secret plus App private key in encrypted systemd credentials.

Typical machine-level config:

REVIEW_QUILL_GITHUB_APP_ID=123456
REVIEW_QUILL_GITHUB_APP_INSTALLATION_ID=12345678

Recommended encrypted credentials:

• review-quill-webhook-secret
• review-quill-github-app-pem

For first-time local bring-up you can also use environment-file secrets, but production should prefer encrypted systemd credentials.

4. Attach a repository

review-quill repo attach owner/repo

review-quill repo attach is the normal happy-path command:

• it adds or updates one watched repository
• it can auto-discover the default branch and required checks
• it stores repo-local review doc paths
• it reloads the service when needed

If you want machine review to count toward merge admission, include review-quill/verdict in the repository's required checks and in any downstream merge queue policy.

5. Validate the install

review-quill doctor --repo repo
review-quill service status
review-quill dashboard

That is the minimum “is this actually alive?” loop.

Public Ingress

Recommended public base URL:

• https://patchrelay.example.com/review

That gives these public endpoints:

• POST /review/webhooks/github for the GitHub App webhook
• GET /review/health for external health checks
• GET /review/attempts/:id for check-run detail links

Keep these local-only:

• /review/watch
• /review/attempts
• /review/admin/*

The package ships an example Caddy config at infra/Caddyfile.

GitHub App Permissions

This is the current known-good permission set:

Repository permissions:

• Contents: Read and write
• Issues: Read and write
• Pull requests: Read and write
• Actions: Read-only
• Metadata: Read-only

Webhook events:

• Pull request
• Check run
• Check suite

Notes:

• Pull requests: Read and write is what lets review-quill submit ordinary GitHub reviews.
• Actions: Read-only lets it observe CI state clearly.
• Contents: Read and write is part of the validated working setup today.

CLI Surface

The operator-facing commands are:

• review-quill init <public-base-url>
• review-quill repo attach <owner/repo>
• review-quill repo list
• review-quill repo show <id>
• review-quill doctor --repo <id>
• review-quill service status
• review-quill service logs --lines 100
• review-quill dashboard
• review-quill attempts <repo> <pr-number>
• review-quill transcript <repo> <pr-number>
• review-quill diff --repo <id>

watch is kept as an alias for dashboard, but dashboard is the name to document and use.

Validation, Visibility, And Troubleshooting

These are the key commands once the service is installed:

review-quill doctor --repo repo
review-quill service status
review-quill dashboard
review-quill attempts repo 123
review-quill transcript repo 123
review-quill diff --repo repo
review-quill service logs --lines 100

Use them this way:

• doctor checks config, binaries, service reachability, and GitHub review wiring.
• dashboard shows queued/running/completed review attempts and recent webhook wakeups.
• attempts shows recorded review history for one PR.
• transcript lets you inspect the visible Codex thread for a review attempt.
• diff shows the exact local diff/inventory the reviewer would see.
• service logs is where to look when webhooks are not arriving, Codex requests fail, or GitHub publishing fails.

If GitHub is not counting review-quill reviews toward branch protection:

1. verify the App permission set above
2. confirm the repository requires the review/check signals you expect
3. run review-quill doctor --repo <id>
4. inspect recent attempts and transcripts for the affected PR

Review Context

review-quill reviews from a real checked-out PR head, not just GitHub API metadata.

The default context path is:

• ephemeral local checkout at the exact PR head SHA
• local git diff <base>...HEAD inventory and curated patch set
• repo guidance from REVIEW_WORKFLOW.md, CLAUDE.md, and AGENTS.md
• prior formal PR reviews from GitHub
• optional Linear issue context when issue keys appear in the PR metadata

Diff context is intentionally filtered:

• noisy/generated paths can be ignored or summarized
• oversized patches are summarized instead of dumped whole
• repo config can tune ignore/summarize patterns and patch budgets

Relationship To PatchRelay And Merge Steward

The three services have distinct ownership:

• patchrelay owns implementation, branch upkeep, and issue/worktree orchestration
• review-quill owns PR review publication
• merge-steward owns queue admission, speculative validation, and landing

GitHub is the shared protocol boundary between them.

Happy Path

review-quill init https://patchrelay.example.com/review
review-quill repo attach owner/repo
review-quill doctor --repo repo
review-quill service status
review-quill dashboard