@datadisco/qa

CI companion for DataDisco QA — PR-triggered persona simulation ("confirmation testing for agentic teams").

Install

npm i -g @datadisco/qa     # or run ad hoc: npx @datadisco/qa <command>

Most teams consume it through the Data-Disco-Inc/qa-action GitHub Action, which wraps this CLI — see its README for the workflow snippet.

Why this exists

The DataDisco QA GitHub App needs zero customer-side install for the common path: it resolves each PR's preview deploy from GitHub deployment_status events (or a configured URL pattern) and posts a "DataDisco QA — persona simulation" check run. This CLI covers the cases the App can't see:

| Command | For teams that… | | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | datadisco-qa report-preview --url <url> | deploy previews without emitting GitHub Deployment events — report the URL from CI so the pending run starts | | datadisco-qa wait --pr <n> | want a pipeline step to block on the QA verdict instead of relying on branch protection | | datadisco-qa tunnel --port <port> | want to run personas against a build served on their own machine |

Authentication

Every command needs a workspace API token (create one under Workspace settings → Integrations → API tokens). Pass it with --api-token or the DATADISCO_API_TOKEN env var. The API base URL defaults to the hosted app and is overridable with --api-url / DATADISCO_API_URL.

export DATADISCO_API_TOKEN=ddqa_xxx

Works in any CI

report-preview and wait auto-detect the repository, PR number, and head SHA from whatever CI they're running in — GitHub Actions gets full support out of the box, and every other CI (or a bare git checkout) still works with zero flags in the common case:

DATADISCO_API_TOKEN=ddqa_xxx npx @datadisco/qa report-preview --url https://pr-482.preview.acme.dev

Each value is resolved through its own chain of fallbacks, in order, stopping at the first one that resolves:

| Value | Resolution order | | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | Repository | --repo → $GITHUB_REPOSITORY → the git checkout's github.com origin remote | | Head SHA | --sha → the GitHub Actions pull_request event payload → $GITHUB_SHA → the git checkout's HEAD | | PR number | --pr → GitHub Actions (GITHUB_REF / event payload) → CircleCI, Jenkins, Buildkite, Travis CI, or Drone's PR env var → (unresolved) |

--repo, --pr, and --sha always override auto-detection when passed.

Unlike the repository and head SHA, the PR number is allowed to stay unresolved: report-preview will still report the preview URL with just the repo + head SHA, and the DataDisco server resolves the PR from the commit itself. wait needs either a PR number or a head SHA to poll by — pass --sha explicitly if neither auto-detects (e.g. a CI DataDisco doesn't recognize yet).

| CI | Repo | Head SHA | PR number | | ----------------- | ------------------- | ------------------------------------- | ----------------------------------------------------------------- | | GitHub Actions | GITHUB_REPOSITORY | pull_request payload / GITHUB_SHA | GITHUB_REF / pull_request payload | | CircleCI | git origin remote | git HEAD | CIRCLE_PULL_REQUEST / CIRCLE_PR_NUMBER | | Jenkins | git origin remote | git HEAD | CHANGE_ID | | Buildkite | git origin remote | git HEAD | BUILDKITE_PULL_REQUEST | | Travis CI | git origin remote | git HEAD | TRAVIS_PULL_REQUEST | | Drone | git origin remote | git HEAD | DRONE_PULL_REQUEST | | Bare git checkout | git origin remote | git HEAD | not auto-detected — pass --pr or rely on server-side resolution |

The git origin remote fallback only recognizes github.com remotes (SSH or HTTPS); for anything else, pass --repo explicitly.

Commands

report-preview

Report the deployed preview URL for a PR head SHA so its pending QA run leaves PENDING_PREVIEW and starts. --repo, --pr, and --sha are auto-detected — see Works in any CI — and the PR number is optional: if it can't be auto-detected, the server resolves it from the head SHA itself.

datadisco-qa report-preview --url https://pr-482.preview.acme.dev
# explicit, e.g. outside any auto-detected CI:
datadisco-qa report-preview \
  --url https://pr-482.preview.acme.dev \
  --repo acme/site --pr 482 --sha "$GIT_SHA"

The server debounces rapid pushes for up to 10 minutes before creating the QA run row, and CI often reports the preview URL before that row exists. By default report-preview handles this for you: on the server's "no QA run is waiting on a preview URL yet" response it retries every 30s for up to 12 minutes, then fails with a hint if the run never showed up. Any other failure (bad token, invalid URL, repo not connected) still fails immediately — only the "not created yet" case is retried. The server can also mark an error as permanently non-retryable (e.g. the PR is closed, a draft, or the head SHA is stale) — those fail immediately too, with the server's error code in the message, instead of burning the retry window.

| Flag | Effect | | ------------------------------ | -------------------------------------------------------- | | --no-wait-for-run | Fail immediately instead of retrying (previous behavior) | | --wait-run-timeout <minutes> | Cap the retry window (default 12) |

wait

Block until the PR's QA run reaches a verdict, printing each phase transition. Polls every 15s; --timeout (minutes, default 30) caps the wait. --repo and --pr are auto-detected the same way as report-preview; if the PR number isn't detectable, pass --sha to poll by head SHA instead.

datadisco-qa wait --pr 482
# or, when the PR number isn't auto-detected:
datadisco-qa wait --sha "$GIT_SHA"

Exit codes (so pipelines can branch on the outcome):

| Code | Meaning | | ---- | ---------------------------------------------- | | 0 | Passed, or passed with warnings (non-blocking) | | 1 | Blocked — one or more blocker findings | | 3 | Run failed before reaching a verdict | | 4 | No preview deploy arrived; run timed out | | 5 | wait gave up after --timeout |

Superseded / cancelled runs exit 0 — a newer commit's run governs the gate.

tunnel

Expose a locally served build over an HTTPS tunnel and start a QA run against it — no preview deploy required. --repo and --sha default to the local git origin remote and HEAD; --pr is required. Bring your own tunnel with --url instead of --port. Blocks on the verdict unless --no-wait is given.

datadisco-qa tunnel --port 5173 --pr 482

Development

npm install
npm test
npm run build   # tsup → dist/cli.js
node dist/cli.js --help

Same toolchain as datadisco-mcp: tsup, vitest, oxlint/oxfmt, cac.

Releasing

Publishing is automated via npm trusted publishing (OIDC — no token). Cut a release with:

npm version <patch|minor|major>   # bumps package.json + lockfile, commits, tags vX.Y.Z
git push origin main --follow-tags

Pushing the v* tag runs release.yml, which verifies the tag matches package.json, runs the tests, and npm publishes with provenance. Prerelease tags (vX.Y.Z-beta.1) publish under the next dist-tag so latest keeps pointing at the newest stable release.

Lockfile note

The toolchain (rolldown/rollup/esbuild) ships platform-native binaries plus a wasm fallback whose @emnapi/* deps npm drops from a macOS-generated lockfile — which then fails npm ci on the Linux CI runner. A plain npm install locally is fine for day-to-day work (just don't commit the resulting lockfile). If you actually change dependencies, regenerate the lockfile on Linux so it stays complete:

rm -f package-lock.json
docker run --rm --platform linux/amd64 -v "$PWD":/app -w /app node:20 npm install