@expo/verify

Portable engine for the /verify agent pipeline: an AI agent investigates a GitHub issue or pull request in an isolated sandbox, a second agent attacks the draft, and the revised findings post back to the thread. This package carries all the logic; the repos that use it hold only a thin workflow file and a .expo-agents/verify/ profile.

Design of record: LLP 0001 in this repository.

Usage

No install needed:

npx @expo/verify <command>

Commands

| Command | What it does | | --- | --- | | verify run [--dry-run] | Run the investigation here — on a CI runner, an EAS worker, or a laptop: context → investigate → critic → revise → publish. A comment body in env selects the comment path (gate, ack, announce, report). --dry-run investigates for real but posts nothing. | | verify mint / verify revoke | The scoped-token steps a workflow runs around run: mint exchanges the parent EXPO_SANDBOX_MCP_TOKEN for a scoped token and writes the agent's mcp.json to VERIFY_MCP_CONFIG (0600); revoke releases it afterwards. The run step then holds only the scoped token — the parent is mapped into the mint/revoke steps on GHA and blanked on the run step on EAS. Without VERIFY_MCP_CONFIG, run mints in-process (the laptop path). | | verify dispatch <target> | Start a run remotely on the profile's runner (GitHub Actions or EAS Workflows) for an issue/PR number or URL, quietly. verify <target> is shorthand. | | verify init --repo <owner/name> | Scaffold .expo-agents/verify/ and the thin runner workflow. Repos set up before 0.9 keep .verify/: every command looks for the new name first and falls back to the old one. | | verify verify-config | CI guard: profile validation, credential-env lock, version-pin assertion. | | verify doctor | Read-only environment/profile/credential triage. | | verify status / verify ls | Dashboard of in-flight and recent runs / in-flight only. On GitHub Actions this lists the dispatch and comment workflows, skipping gate-skipped runs, and reports each finished run by its jobs' outcome — the thin workflows use job-level continue-on-error, so the run itself always says success. EAS runs are titled from the job outputs the engine emits (target, mode, outcome), since EAS has no per-run name. |

Comment commands (/verify, @expo-bot verify, @expo-bot check, @expo-bot triage, and work requests) receive an 👀 acknowledgment without a "started" comment by default. Pick the agent model with --model <name>, a shorthand flag (--fable, --opus, --sonnet, --haiku), or the phrase with <name> on verify, check, and triage commands, for example @expo-bot verify with fable. Work requests accept only the flag forms, since "with" is ordinary prose in a task. Add --announce on the first line to post one, for example @expo-bot verify --announce. --no-announce takes precedence if both flags are present. Findings and failure reports still post to the thread. Use verify status to find runs without a start comment.

Duplicate pull requests

In pr fix mode the publisher checks the fix against every open pull request before opening one. File overlap only shortlists: an open PR that touches every non-changelog file the fix touches is a candidate, capped at five. For each candidate a tool-less, schema-bound model call reads the shared-file patches on both sides and answers whether the candidate already makes the same change (templates/duplicate.md is the policy). A confident duplicate keeps the pushed branch and opens no PR; anything else opens the PR and lists the candidates, with the judge's one-line reason, for the reviewer. Without a judgment (no model, a failed call, a malformed reply) the fallback is a symmetric file-set match. Everything the decision saw is written to .verify-out/duplicate-judge.json.

Profile field, all keys optional: "duplicateJudge": { "enabled": true, "model": "claude-haiku-4-5-20251001", "threshold": 0.7 }. The judge runs on the run's model credential with the same isolation as triage.

Triage

From this source checkout, with gh authenticated and a Claude Code version supporting --safe-mode and --json-schema:

bun src/cli.ts triage expo/expo#49760 --dry-run --out-dir /tmp/verify-triage

Triage supports Expo issues and PRs. It collects context and runs classify → critic → revise with Opus, without tools or sandbox access. --dry-run renders proposed results without writing to GitHub.

Maintainers can comment @expo-bot triage on an issue or PR to classify it and publish routing labels plus a preliminary triage comment. Commands receive an 👀 acknowledgment; the start reply is opt-in with --announce. The first verify run also attempts triage before investigation. Later verify runs skip targets with a completed triage comment authored by the configured bot; an explicit triage command always reruns it. Triage failure never blocks verify.

For local publishing, omit --dry-run and use the target repository's trusted profile (--config-dir PATH if needed), authenticated as its configured bot. Publishing only adds existing routing labels and updates the bot's triage comment; it never closes targets or removes labels. Bug issues may receive severity: critical, severity: high, severity: medium, severity: low, or severity: trivial. Any existing severity label is preserved, including on explicit reruns. Reports spell out the code, name, and meaning (for example, S3 - Low). Comments use short impact, scope, and workaround bullets rather than a narrative summary. Workaround status distinguishes a reported working option, unsuccessful attempts, and missing information. An uncertain completed verdict receives no labels but still counts as triaged. Opening an issue or PR alone does not trigger triage yet. The policy is templates/triage.md; configurable label maps remain deferred.

Each target gets a unique output directory containing context, phase responses, draft.json, review.json, result.json, and any proposed-comment.md. The result includes proposed/skipped labels, input provenance, elapsed time, and model-reported USD cost (an estimate, not a subscription invoice). Failures retain error.json and any completed phase artifacts.

Options: --model (default claude-opus-5-5), --timeout-seconds (default 120 per phase, maximum 300), and --max-budget-usd (default 0.50 per phase, maximum 5). Each run has three phases and no automatic retries. Inputs sent to the model are capped at 60,000 characters, with truncation explicitly recorded. The classifier has no tools or MCP servers; it receives an allowlisted environment and runs with customizations disabled in an empty temporary working directory. Only the fixed collector uses GitHub credentials.

Adopting a repo

1. In the repo's checkout: verify init --repo owner/name [--runner gha|eas] [--fix pr|gist|inline|comment|none] Fix modes: pr opens a pull request from the bot fork; gist/inline post a separate patch comment; comment folds the guarded patch into the findings comment itself (a <details> block with a concise explanation and a fenced diff — for repos that should never receive a second comment or a PR); none verifies only.
2. Fill in .expo-agents/verify/build.md (the repo's sandbox build recipe) and .expo-agents/verify/policy.md (repo-specific agent policy).
3. Add the secrets the generated workflow names, then run verify verify-config.
4. Dispatch: verify <issue-number> (or verify <n> --runner eas).

Running on EAS Workflows

verify <n> --runner eas from a repo's checkout uploads that checkout's verify files — .expo-agents/verify/, .eas/workflows/verify.yml, app.json, eas.json — and runs the investigation on EAS. The upload is a few KB (.easignore trims the rest; the run clones the target repo itself in a sandbox), uncommitted files are included, and there is no GitHub link or repo-ownership requirement. verify init --runner eas --eas-project <id> writes all of those files and resolves the project's owner/slug into app.json. On the worker the engine clones the target's default branch itself (shallow — the same trusted base a GitHub Actions checkout gives), so fix mode and the publisher work on EAS exactly as on GHA.

This is also how a repo you do not own gets verified: the same files, kept as local uncommitted changes, customized freely in .expo-agents/verify/; commit them only if and when you want to land verify upstream.

The profile is enforcement, not documentation: its capability fields drive the agent tool allowlist, the scoped-token mint, and which publish steps run.

Releases

Push a tag: git tag v<semver> && git push origin v<semver>. release.yml stamps the version from the tag and publishes via npm trusted publishing (OIDC). The repository is private; the published package contains only build/ and templates/.