@runtime-judgement/rj-action

A Node.js GitHub Action that runs a Runtime Judgement snapshot suite from CI and gates the PR on the verdict. Sprint 12 stub — wire-level integration only. PR check-run posting, marketplace listing, and signed releases land in a later sprint.

What it does

1. Reads suite-id and rj-api-token from the workflow step inputs.
2. POSTs to /api/snapshot-suites/{suite-id}/run with Authorization: Bearer <token>.
3. Waits for the synchronous response (up to timeout-ms, default 5 minutes).
4. Reduces the result to a single verdict: pass / drift / regression / error / empty.
5. Prints a summary line to the Actions log and sets seven outputs.
6. Exits non-zero on regression (when fail-on-unexpected=true), error, or empty.

Inputs

| Input | Required | Default | Description | |----------------------|---------:|--------------------------------------|---| | suite-id | Yes | — | Snapshot suite ULID (visible at /app/snapshot-suites/<id>). | | rj-api-token | Yes | — | RJ API token (rj_live_*). Pass via ${{ secrets.RJ_API_TOKEN }}. Never inline. | | rj-base-url | No | https://app.runtimejudgement.com | Override for self-hosted or staging. | | fail-on-unexpected | No | true | Set to false for advisory-only runs. | | timeout-ms | No | 300000 | Hard caller-side timeout. Matches the API route's maxDuration. |

Outputs

| Output | Description | |-----------------------|---| | verdict | One of pass, drift, regression, error, empty. | | total | Total snapshots in the suite. | | passed | Number of snapshots that passed. | | changed-intentional | Number of snapshots that changed intentionally. | | changed-unexpected | Number of snapshots with unexpected regressions (the gate flag). | | errored | Number of snapshots that errored (infra failures, not regressions). | | run-ids | Comma-separated list of snapshot-run IDs (for deep-linking later). |

Setup

1. Create an API token. Visit /app/settings on your RJ deployment and generate a token. Tokens are prefixed rj_live_.
2. Add it as a repo secret. Settings → Secrets and variables → Actions → New repository secret. Name it RJ_API_TOKEN.
3. Find your suite ID. Open /app/snapshot-suites/<id>; the ULID is in the URL.
4. Drop in a workflow. See runtime-judgement-ci.sample.yml for a copy-paste-able example.

Example workflow

name: Runtime Judgement gate

on:
  workflow_dispatch:
  pull_request:
    branches: [main]

permissions:
  contents: read
  pull-requests: read

jobs:
  rj-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ./packages/rj-action
        with:
          suite-id: 01HZSUITE_REPLACE_WITH_YOURS
          rj-api-token: ${{ secrets.RJ_API_TOKEN }}

Verdict semantics

| Verdict | Meaning | Exit | |---------------|---------------------------------------------------------------------------------------------------|-----:| | pass | Every snapshot passed. | 0 | | drift | Some snapshots changed but the judge ruled the change intentional. Review. | 0 | | regression | At least one snapshot is changed-unexpected. The patch likely broke locked-in behaviour. | 1 | | error | At least one snapshot errored (infra failure). Verdict isn't trustworthy; investigate infra. | 1 | | empty | The suite has no snapshots. Most likely a misconfigured suite-id. | 1 |

Error precedence is deliberate: an infra failure is the operator's problem before regressions are, because regressions can't be trusted if half the suite errored before evaluation.

Limitations (Sprint 12 stub)

• No PR check-run. This stub doesn't post a GitHub Check or PR comment; it just sets the step's exit code. The full PR-verifier GitHub App (with auth, permissions, webhook semantics) is deferred until product clarity lands.
• No marketplace listing. The action is invoked as a local path (uses: ./packages/rj-action) until we publish to the marketplace.
• Synchronous run only. The /run endpoint is synchronous (up to its configured maxDuration). The action polls nothing — it just waits.

Development

# From the package directory:
pnpm install
pnpm test         # runs vitest
pnpm typecheck    # tsc --noEmit
pnpm build        # tsc to ./dist (what GH Actions consumes)

Tests live under src/__tests__/*.test.ts and run in isolation from the root app's vitest config. They cover input validation, the suite-run wire shape, the verdict-summary precedence, and the entry-point exit-code logic.

License

MIT.