@exodus/xqa

AI-powered QA agent CLI for Exodus applications.

Overview

xqa automates mobile app QA by connecting to physical devices or emulators and running intelligent exploration and spec-based testing. The CLI orchestrates the pipeline that spawns agents to interact with your app, capture screenshots, and generate findings based on user-defined specs or breadth-first exploration.

The tool manages configuration, project initialization, session state tracking, and interactive review workflows for triaging findings.

Commands

mcp

Run the xqa MCP server over stdio for consumption by Claude Code and other agentic clients.

xqa mcp                       # auto-detect booted simulator and connect to per-UDID daemon
xqa mcp --udid ABCD1234       # target a specific booted simulator

By default, xqa mcp multiplexes through a per-UDID daemon. The first invocation for a given UDID lazily spawns a detached daemon process; subsequent invocations from any working directory (other Claude Code sessions, other terminals, other worktrees) connect as thin stdio↔socket clients to the same daemon. This allows multiple concurrent Claude Code sessions to share a single simulator driver — the previous behavior, where each invocation tried to acquire its own per-UDID lock and the second one died with xqa is already running, no longer happens.

The daemon serializes every tools/call request through a FIFO queue so that concurrent clients cannot corrupt sim/WDA state mid-call. In-flight tool calls have a configurable wall-clock timeout (XQA_DAEMON_CALL_TIMEOUT_MS); a timed-out call releases the queue and surfaces an error to the originating client so a stuck Appium request cannot wedge the daemon.

Each connecting client gets its own apparent MCP session: its own initialize handshake, its own tools/list response (the same 12-tool set), and its own request-ID space. The daemon multiplexes by mapping client-id × request-id to internal routing.

Set XQA_DAEMON_DISABLE=1 to fall back to the legacy per-process model (each invocation runs its own MCP server protected by a per-UDID pid lock). Use this only for debugging or emergency rollback.

Other xqa subcommands (explore, spec, run, plan, triage, etc.) are unchanged and connect to Appium directly. If a daemon is running for a given UDID and you start an interactive xqa command against the same simulator, the interactive command will hit the existing per-UDID lock — stop the MCP session first.

daemon

Manage the per-UDID xqa daemon that backs xqa mcp.

xqa daemon --foreground            # start the daemon in the foreground (auto-detected UDID)
xqa daemon --foreground --udid X   # start the daemon for a specific UDID
xqa daemon --status                # print status for the daemon bound to <udid>
xqa daemon --stop                  # signal the daemon bound to <udid> to shut down
xqa daemon --list                  # list all known daemons under the socket directory

Flags:

• --udid <id> — Target simulator UDID. Falls back to XQA_UDID then auto-detect.
• --foreground — Run in the current process (for launchd/systemd or local debugging). When omitted, the daemon is normally spawned detached by xqa mcp itself; you usually do not invoke xqa daemon without --foreground directly.
• --status — Print whether a daemon is running for <udid>, its PID, and version.
• --stop — Send SIGTERM to the daemon bound to <udid> for a graceful shutdown.
• --list — Print all known daemons under the socket directory (running and stale).

These management subcommands operate regardless of XQA_DAEMON_DISABLE.

init

Initialize a new xqa project in the current directory.

Creates a .xqa/ directory with app.md, explore.md, and owners.md templates plus subdirectories for specs, designs, and suites. Installs bundled xqa skills. The owners.md template is fully commented out, so finding attribution stays off until it names at least one @user or @org/team handle.

xqa init

update

Update installed xqa skills to the current CLI version.

xqa update

explore [prompt]

Run the explorer agent; omit prompt for a full breadth-first sweep.

Optional focus hint for the explorer agent. Omit to explore the entire app from the starting state. Generates a findings JSON file in .xqa/output/ and prints the path upon completion.

xqa explore                          # breadth-first exploration
xqa explore "test the login flow"    # focused exploration
xqa explore -v prompt,snapshot         # verbose output for categories
xqa explore -v                       # verbose output for all categories
xqa explore -t 600                   # override explorer timeout (seconds)
xqa explore --debug                  # log timing and event details to stderr
xqa explore --udid ABCD1234          # target a specific booted simulator
xqa explore --asset pangolin_avax    # pin the run to a specific asset/network
xqa explore --visual                 # force the visual-quality pass on

Flags:

• -v, --verbose [categories] — Log categories (prompt, tools, snapshot, memory). Default: all if flag is present without value.
• -t, --timeout <seconds> — Explorer timeout in seconds (overrides agents.explorer.timeoutSeconds in .xqa/config.yaml).
• --debug — Log timing and event details to stderr.
• --udid <id> — Target simulator UDID. Overrides auto-detect of first booted; exits with code 2 if the UDID is not booted.
• --asset <asset_network> — Pin the run to a specific asset, formatted as ASSET or ASSET_NETWORK using [a-z0-9-] (e.g. pangolin_avax, btc). Injected into the explorer prompt so the agent operates on that exact asset (and network, for multi-network tokens) instead of auto-selecting; omit to let the spec self-select.
• --visual — Force the explorer visual-quality pass on, overriding agents.explorer.visual.enabled in .xqa/config.yaml. Without a designsDir it runs in no-designs heuristic mode (emits design-system-violation findings).

spec [spec-file]

Run the explorer agent against a spec file.

Loads a spec markdown file from .xqa/specs/ (or an absolute path) and executes the agent against it. Omit the argument to pick from available specs interactively.

xqa spec                                      # interactive spec picker
xqa spec .xqa/specs/authentication.test.md    # explicit spec file
xqa spec -v tools,memory                      # verbose output
xqa spec --debug                              # debug logging

Flags:

• -v, --verbose [categories] — Same as explore.
• --debug — Log timing and event details to stderr.

Spec file format (YAML frontmatter + markdown):

---
feature: 'Feature Name'
timeout: 300
---

# Spec content

Frontmatter fields: feature (required), timeout (optional, seconds).

run

Run a test suite or a set of spec files in parallel across booted simulators.

Exactly one of --suite or --spec is required.

xqa run --suite smoke                         # run .xqa/suites/smoke.suite.json
xqa run --spec 'specs/**/*.test.md'           # run matching spec files
xqa run --suite smoke --only spec-login       # run a single work item by id
xqa run --suite smoke --debug                 # debug logging
xqa run --suite smoke --udid ABCD1234         # constrain the suite to one booted simulator
xqa run --spec 'specs/send.round-trip.test.md' --asset pangolin_avax   # retest a send against one asset/network

Flags:

• --suite <name> — Name of the suite (<name>.suite.json) under .xqa/suites/.
• --spec <globs...> — Glob patterns matching spec files, resolved from the xqa directory.
• --only <id> — Run only the work item with the given id (requires --suite). Ids are deterministic: spec-<name-without-specs-prefix> for specs, freestyle-<index> for freestyle entries. Hooks still run. Output still lands at output/suite/<suiteId>/<date>/<runId>/findings.json with the single item in items[].
• --debug — Log timing and event details to stderr.
• --udid <id> — Target simulator UDID. When supplied, the suite is constrained to that one simulator; exits with code 2 if the UDID is not booted.
• --asset <asset_network> — Pin every work item in the run to a specific asset, formatted as ASSET or ASSET_NETWORK using [a-z0-9-] (e.g. pangolin_avax, btc). Injected into the explorer prompt so the agent operates on that exact asset (and network, for multi-network tokens) instead of auto-selecting. Most useful with --spec/--only; applies to all items when combined with a multi-item --suite.

plan

Generate or evolve the manual test plan for the current branch.

Inspects the git diff between the current branch and its upstream, asks the planner agent to emit Markdown scenario specs, and writes them to .xqa/test-plan/default/ (or a custom directory). Subcommands let you refine individual scenarios, append new scenarios after fresh commits, and correlate findings from a run against the plan.

xqa plan                                          # generate scenarios from current diff
xqa plan --intent "login changes" --out .xqa/test-plan/my-slug
xqa plan --base develop                           # diff against a branch other than origin/HEAD
xqa plan edit .xqa/test-plan/my-slug/scenario-1.test.md --feedback "rename step 2"
xqa plan extend                                   # append scenarios for fresh commits
xqa plan report --findings .xqa/output/.../findings.json --specs .xqa/test-plan/my-slug

Flags:

• --intent <text> — Optional focus hint passed to the planner.
• --out <dir> — Output directory for the generated scenarios (default: <xqa>/test-plan/default).
• --base <ref> — Base git ref to diff against. When omitted, xqa auto-detects the base from an open PR via gh pr view and falls back to origin/HEAD. Pass explicitly to override.
• --debug — Log base/head refs, diff summary, existing specs count, classification, the full prompt sent to the model, and the raw AI response to stderr. Useful for investigating model-abstained empty results.

Subcommands:

• xqa plan edit <file> --feedback <text> — apply user-requested edits to an existing scenario spec.
• xqa plan extend [--intent <text>] [--out <dir>] — append new scenarios for commits since the last plan was generated.
• xqa plan report --findings <path> [--specs <dir>] — correlate findings with scenarios and write report.json next to the plan.

What does it do?

xqa plan reads the branch diff, summarizes it, and feeds the context to the planner agent, which emits one Markdown scenario spec per suggested flow. The specs are written to the plan directory so you can review or hand them to xqa run. After running the scenarios, xqa plan report correlates the resulting findings back to each scenario so you can see which flows passed, which surfaced issues, and which were skipped. xqa plan edit lets you nudge a single scenario with natural-language feedback; xqa plan extend picks up commits added after the initial generation and appends new scenarios without touching the existing ones.

notify <path>

Post QA findings as a Slack notification.

Reads a suite findings JSON file, classifies each finding into buckets (wallet blockers, spec deviations, agent-detected bugs, incomplete legs), and posts a Slack Block Kit message to the configured channel. Exits 0 silently when SLACK_BOT_TOKEN is unset, so it is safe to add to CI pipelines without requiring Slack credentials in all environments. Slack transport failures (network errors, non-2xx responses) are non-fatal: the command warns and exits 0. A Slack API rejection such as channel_not_found exits 1 so real misconfiguration surfaces.

xqa notify .xqa/output/.../findings.json
xqa notify findings.json --channel C123ABC --pr-url https://github.com/org/repo/pull/42
xqa notify findings.json --run-url https://github.com/org/repo/actions/runs/123
xqa notify findings.json --dry-run   # print the message, no token needed

Flags:

• --channel <id> — Slack channel ID. Highest-priority channel source (see precedence below).
• --pr-url <url> — Pull request URL linked in the notification.
• --run-url <url> — CI run URL linked in the notification.
• --trigger <kind> — Kind of run that produced the report (pr, daily, or manual). Shown in the message context so daily reports are distinguishable from PR runs.
• --dry-run — Render and print the Block Kit message without posting. Requires no token; useful for local validation against a real findings file.

Channel resolution precedence (first match wins):

1. --channel <id> flag
2. SLACK_CHANNEL env var
3. report.slack.channel in .xqa/config.yaml

When none resolves a non-empty channel, the command warns and exits 0. A malformed or invalid .xqa/config.yaml does not fail the command: it warns with the underlying cause, the channel falls through to undefined, and the command skips. A missing config file stays silent.

The bot token is env-only (SLACK_BOT_TOKEN); it is never read from .xqa/config.yaml.

Environment variables:

• SLACK_BOT_TOKEN — Slack bot token with chat:write scope. When absent, the command exits 0 with a warning.
• SLACK_CHANNEL — Channel ID used when --channel is not passed; overridden by --channel, and takes precedence over .xqa/config.yaml.

xqa digest resolves its channel through the same precedence (--channel, then SLACK_CHANNEL, then report.slack.channel), so a branch whose .xqa/config.yaml predates the channel entry still posts its digest.

review [findings-path]

Review findings and mark false positives.

Interactive session for triaging findings generated by explore or spec runs. Mark findings as dismissed (with optional reason) or undo previous dismissals. Dismissals are written to dismissals.json next to the .xqa directory (override with run.dismissalsPath in .xqa/config.yaml). Defaults to the last findings path if omitted.

xqa review                                     # use last findings file
xqa review .xqa/output/findings-abc123.json    # explicit path

designs sync

Pull Figma page designs into the local designs directory.

Reads figma.pages from .xqa/config.yaml, fetches each Figma page via the REST API, classifies exported frames with the Haiku classifier, and writes approved PNGs to agents.explorer.visual.designsDir. Requires FIGMA_TOKEN and ANTHROPIC_API_KEY.

xqa designs sync             # sync all configured pages (uses cache)
xqa designs sync --no-cache  # bypass classifier cache and lastModified gate

designs rebuild

Force a full design sync from Figma.

Bypasses the cache and last-modified gate while preserving the manifest so stale managed files can be removed safely. If a previous sync quarantined a corrupt manifest, rebuild re-derives a new manifest from the configured pages.

xqa designs rebuild

visual

Compare captured component and screen screenshots against baselines (advisory).

One manifest declares both kinds of check, discriminated by type. Both kinds are baselined from master through the content-addressed R2 cache by default; component entries may opt into Figma baselining with baseline: figma. Check ids share a namespace and must not collide.

# .xqa/visual/manifest.yaml
version: 1
figma:
  fileKey: AbC123 # required only when a check uses baseline: figma
globalPaths:
  - design-tokens/**
checks:
  button-primary:
    type: component
    entry: src/ui/Button.tsx
    baseline: master # optional, default master
    paths: [] # optional extra globs marking the check affected and folded into the baseline key
    threshold: 0 # optional, default 0 for baseline master, 0.01 for baseline figma
    pixelThreshold: 0.05 # optional per-pixel YIQ tolerance
    component: button # optional harness render fields
    props: { children: Buy }
    frame: { width: 153, height: 72 }
    icon: image-placeholder
    scale: 3 # optional, default 3
  button-designed:
    type: component
    baseline: figma # opt in to design comparison
    node: '1:2' # required when baseline is figma
    entry: src/ui/Button.tsx
    fileKey: XyZ789 # optional per-check Figma file override
  WalletAssetDetail:
    type: screen
    entry: src/screens/Wallet/AssetDetail.js
    params: { assetName: bitcoin } # optional route params
    threshold: 0 # optional, default 0
    pixelThreshold: 0.01 # optional per-pixel YIQ tolerance

Baseline keys are content-addressed per check type: visual-baselines/components/<id>/<sha>.png and visual-baselines/snapshots/<id>/<sha>.png.

threshold defaults by baseline source: 0 for master-baselined checks (code against code, so any changed pixel is a change) and 0.01 for baseline: figma checks (rendering differences against a design file need slack). Screens default to 0.

Component ids are kebab-case (^[a-z0-9][a-z0-9.-]*$); screen ids are app route names (^[\w-]+$).

xqa visual --manifest .xqa/visual/manifest.yaml --captures-dir captures --changed-files changed.txt --out out
xqa visual --manifest .xqa/visual/manifest.yaml --snapshots-keys keys.json --snapshot-captures-dir snapshot-captures --out out

• --manifest <path> — required; the one registry for both check types.
• --captures-dir <dir> — directory with <checkId>.png component captures; enables the component phase. Requires exactly one of --changed-files / --all.
• --baselines-dir <dir> — use local <checkId>.png component baselines instead of the R2 cache or Figma; needs no keys file and no credentials. The compared image is local, so no baseline key is reported.
• --snapshots-keys <path> — keys.json produced by visual-snapshots-keys; supplies the master baseline key for entries of either type. Screen entries enable the screens phase and then require --snapshot-captures-dir; a components-only keys file needs neither, only --captures-dir. component checks with baseline: master that run without it (and without --baselines-dir) degrade to not-compared with a baseline-key-missing reason. All entries are resolved at diff time regardless of their recorded hit/miss status (the parallel seed job usually fills misses before the diff runs).
• --snapshot-captures-dir <dir> — directory with <screenId>.png captures; required when the keys file has screen entries.
• --snapshot-baselines-dir <dir> — use local pre-resolved <screenId>.png baselines instead of downloading from the R2 cache. The compared image is local, so no baseline key is reported.
• --snapshot-capture-results <path> — optional JSON {"<screenId>": {"status": "captured" | "render-failed" | "missing-param" | "baseline-build-failed", "reason": "..."}}; non-captured statuses become not-compared reasons instead of the generic missing-capture message.

At least one phase must be enabled. Every result carries its type (component or screen) into report.md, visual-report.json and the uploaded run payload. Missing R2 credentials, missing Figma credentials and storage failures degrade the affected checks to not-compared; they never fail the run.

visual-snapshots-keys

Resolve affected master-baselined checks and probe the content-addressed baseline cache. Runs before any build in CI.

Reads every type: screen entry plus every type: component entry with baseline: master (Figma-baselined checks are excluded), selects the affected ones from the changed-file list via the dependency graph (dependency-manifest changes such as lockfiles, and changes to the manifest itself, mark every check affected; component paths globs and manifest globalPaths also select components), derives each check's baseline key — visual-baselines/components/<id>/<sha>.png for components and visual-baselines/snapshots/<id>/<sha>.png for screens — where <sha> is the last merge-base commit touching the check's key file set: its reachable imports plus the manifest itself (so editing params, props or a threshold rotates the key), and for components also its own paths globs and the manifest globalPaths (so a design-token or asset change that selects the check also rotates its key), probes R2 for each key, and writes keys.json with a per-check type and hit/miss status. Entries without a type (a keys.json written by an older CLI) are read as screens.

xqa visual-snapshots-keys --manifest .xqa/visual/manifest.yaml --merge-base "$(git merge-base origin/master HEAD)" --changed-files changed.txt --out keys.json

• Requires exactly one of --changed-files / --all.
• Missing R2 credentials: every entry is written as miss and the command exits 0.
• Dependency-graph failure: every check is keyed with the merge-base sha (redundant rebuilds, never wrong pixels).
• Non-shallow git history at the merge-base is a caller-guaranteed precondition.

visual-snapshots-seed

Upload merge-base captures for cache-missed master baselines, of either check type. Runs in the parallel baseline-build job.

xqa visual-snapshots-seed --keys keys.json --captures-dir baseline-captures \
  --component-captures-dir baseline-component-captures

• --captures-dir <dir> — <screenId>.png captures; also used for components when --component-captures-dir is omitted. Required only for screen entries.
• --component-captures-dir <dir> — <checkId>.png component captures; a components-only seed job needs this one alone.

Only miss entries are uploaded; absent captures are skipped. Uploads are idempotent (content-addressed keys). Missing credentials log a warning and exit 0.

CI choreography: visual-snapshots-keys (no build, seconds) → on any miss, a parallel merge-base job captures the missing checks and runs visual-snapshots-seed while the PR build runs → xqa visual downloads baselines by key, diffs, and posts one report.

completion

Output shell completion script.

Generate completion script for bash or zsh. Pipe output to shell config file to enable tab completion.

xqa completion bash  # generate bash completions
xqa completion zsh   # generate zsh completions

Suite config

Suite files live at .xqa/suites/<name>.suite.json and declare the work items plus optional hooks.

{
  "specs": ["specs/send.test.md"],
  "freestyle": [{ "prompt": "explore settings", "timeoutSeconds": 300 }],
  "hooks": {
    "beforeEach": {
      "script": "qa/prepare-sim.mjs",
      "env": { "APP_PROFILE": "funded" },
      "timeoutSeconds": 120,
      "retries": 3
    }
  }
}

Fields:

• specs (optional) — glob patterns resolved from the xqa directory.
• freestyle (optional) — either a positive integer (N empty entries) or an array of { prompt?, timeoutSeconds } entries.
• At least one of specs or freestyle must resolve to a work item.
• hooks.beforeEach (optional) — runs before every work item on every simulator. Use for project-owned setup (wallet provisioning, cache warming, login seeding).

The hook script is invoked as a Node child process. It receives:

• Inherited process.env
• Suite-declared env overlaid with reserved keys (reserved wins)
• Reserved xqa-owned keys: XQA_SIM_UDID, XQA_ITEM_ID, XQA_ITEM_TYPE, XQA_ITEM_NAME, XQA_SUITE, and (when item type is spec) XQA_SPEC_PATH

Suite-declared env cannot override reserved keys — the parser rejects such configs.

Contract:

• Exit 0 → proceed with item.
• Non-zero exit → item marked failed, executeItem skipped, counts toward simulator-unhealthy threshold.
• Default 120s timeout, overridable via hooks.beforeEach.timeoutSeconds.
• Default 3 retries on failure (HOOK_EXIT_NONZERO, HOOK_TIMEOUT, HOOK_SPAWN_FAILED), overridable via hooks.beforeEach.retries (range 0..10). Set to 0 to disable retries. Aborts (HOOK_ABORTED) are never retried.
• A HOOK_RETRY suite event is emitted before each retry attempt with attempt, maxAttempts, and previousErrorType.
• Honors the suite abort signal.

Configuration

Configuration splits in two: non-sensitive runtime settings in .xqa/config.yaml, secrets in the environment.

.xqa/config.yaml

xqa init writes this file with sensible defaults. It's the canonical home for agent toggles and tunables:

version: 1

run:
  # id: my-run
  # dismissalsPath: .xqa/dismissals.json

suites:
  directory: .xqa/suites

agents:
  explorer:
    enabled: true
    timeoutSeconds: 1200
    buildEnv: dev
    bundleId: com.example.app
    capabilities:
      videoRecording: false
      viewUiServer: true
      findingScreenshots: true

  consolidator:
    enabled: true

  triager:
    enabled: false

figma:
  pages:
    - https://www.figma.com/design/<fileKey>/My-App?node-id=1-2
  scale: 2
  classifierModel: claude-haiku-4-5-20251001
  maxConcurrentPages: 1

models:
  planner: claude-opus-5
  consolidator: opus
  attributor: sonnet
  explorerVisualPass: claude-fable-5

See Models for how these resolve and what each one defaults to.

Field reference:

| Field | Default | Description | | ------------------------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | version | 1 | Config schema version. | | a11y | false | Reports accessibility gaps. Off by default: no missing-a11y-element findings from the explorer and no accessibility-bugs group for the analyst. | | run.id | (auto) | Fixed run ID. Omit for sequential per-run IDs. | | run.dismissalsPath | .xqa/dismissals.json | Where xqa review persists dismissals. | | suites.directory | .xqa/suites | Directory containing *.suite.json files. | | agents.explorer.enabled | true | Runs the explorer agent. | | agents.explorer.timeoutSeconds | 1200 | Wall-clock limit per explore/spec run. | | agents.explorer.buildEnv | dev | dev or prod. dev ignores debug overlays as findings. | | agents.explorer.bundleId | (none) | Bundle ID of the app under test. When set, the explorer's WebDriverAgent session activates that app on start, so the agent observes the app instead of the iOS home snapshot and never relaunches it mid-run. | | agents.explorer.capabilities.videoRecording | false | Records the simulator snapshot to MP4. | | agents.explorer.capabilities.viewUiServer | true | Registers the view_ui MCP tool for reading the UI tree. | | agents.explorer.capabilities.findingScreenshots | true | Writes per-finding PNGs. | | agents.consolidator.enabled | true | Merges and deduplicates findings from every agent. | | agents.triager.enabled | false | Runs the PR suite matcher. Needs GITHUB_TOKEN. | | agents.analyst.enabled | true | Accepted for symmetry with the other agents and currently ignored. Nothing reads it; the analyst runs over every planned cluster regardless. | | agents.analyst.appContext | (shipped copy) | Prose that opens the analyst prompt, describing the app under test. | | agents.analyst.guidance | (shipped copy) | Cross-group prose: how to separate look-alikes and which group to fall back to. | | agents.analyst.classifications | (shipped groups) | The classification groups this repo reports under. Declaring any replaces all shipped groups; see below. | | figma.pages | [] | Figma page URLs to sync (xqa designs sync). Needs FIGMA_TOKEN. | | figma.scale | 2 | Export scale factor (0.01–4). | | figma.classifierModel | claude-haiku-4-5-20251001 | Haiku model used to classify frames as UI designs. | | figma.maxConcurrentPages | 1 | Reserved for future parallel sync; only 1 is accepted today. |

Models

Each agent runs on a model pinned in code. Override any of them under models:

models:
  planner: claude-opus-5 # full model id — used verbatim
  consolidator: opus # family name — resolves to the newest opus release
  attributor: sonnet
  explorerVisualPass: claude-fable-5

A value is either a full Anthropic model id (anything starting with claude-), passed through unchanged, or one of the family names opus, sonnet, haiku. A family name resolves to the newest released model in that family through the Anthropic models API on first use, then is cached for the lifetime of the process.

Omit a key — or the whole models block — to keep its default:

| Key | Default | Drives | | -------------------- | ----------------- | --------------------------------- | | planner | claude-opus-5 | xqa plan (generate/extend/edit) | | consolidator | claude-opus-5 | Findings consolidator | | attributor | claude-sonnet-5 | Owner attributor | | explorerVisualPass | claude-opus-5 | Explorer's visual-quality pass |

Capabilities

Each agent has a capabilities block of opt-in feature flags. Enabling a capability doesn't enable the agent — both enabled: true and capabilities.<name>: true are required.

The explorer's videoRecording capability records the simulator snapshot to an MP4 that the viewer app uses for playback.

Visual analysis

Explorer can perform visual review in addition to structural exploration. Enable via agents.explorer.visual.enabled: true in .xqa/config.yaml. When designs are available, point designsDir at a directory of *.png artboards.

| visual.enabled | designsDir | Behavior | | ---------------- | -------------- | ----------------------------------------------------------------------- | | false | any | Structural only (default; identical to today's explorer). | | true | set, non-empty | Full visual review with artboard comparison and read_artboard budget. | | true | unset or empty | Visual review without artboard reference (design-system-blind). |

explorer:
  visual:
    enabled: true
    designsDir: .xqa/designs
    matchTolerance: balanced # strict | balanced | loose
    candidateCount: 3
    readArtboardImageTokenBudget: 80000
    # Per-run in-memory cache that skips visual_pass on snapshots already
    # analysed during this run. Every analysed snapshot's fingerprint is
    # kept for the lifetime of the run.
    cacheTolerance: strict # strict | balanced | loose — how lenient to be when calling two snapshots "the same"

Environment variables

Secrets stay in .env.local (loaded by dotenv) or your shell. Lock the file down:

chmod 600 .env.local

• ANTHROPIC_API_KEY (required) — Anthropic Claude API key for agent reasoning
• FIGMA_TOKEN (required for xqa designs sync) — Figma personal access token with file_read scope; format: figd_…
• GITHUB_TOKEN (optional) — required for xqa triage
• SLACK_BOT_TOKEN (optional) — Slack bot token with chat:write scope for xqa notify and xqa digest; when absent, the command warns and exits 0
• SLACK_CHANNEL (optional) — default Slack channel ID for xqa notify and xqa digest; overridden by --channel, takes precedence over report.slack.channel in .xqa/config.yaml
• XQA_UDID (optional) — default simulator UDID for xqa mcp; overridden by --udid
• XQA_DAEMON_DISABLE (optional) — set to 1/true to disable daemon mode for xqa mcp; for debugging/rollback only
• XQA_DAEMON_IDLE_TIMEOUT_MS (optional) — daemon idle shutdown delay (default 600000 ms = 10 min)
• XQA_DAEMON_SOCKET_DIR (optional) — override socket/pid/log parent dir (default $TMPDIR/xqa)
• XQA_DAEMON_LOG_LEVEL (optional) — daemon log level: debug / info / warn / error (default info)
• XQA_DAEMON_CALL_TIMEOUT_MS (optional) — per-tool-call wall-clock timeout (default 120000 ms = 2 min). When exceeded, the daemon releases the FIFO queue and returns a -32603 error to the originating client.
• APPIUM_URL (optional) — connect to an existing Appium server instead of auto-spawning one

Video recording

videoRecording is an independent capability that records the simulator snapshot to an MP4 (used by the viewer app for playback):

agents:
  explorer:
    enabled: true
    capabilities:
      videoRecording: true

Classification groups

The nightly digest reports each cluster under one classification. XQA ships five — likely-bugs, maybe-bugs, accessibility-bugs, spec-issues, environment — and always appends three reserved ids a repo may not declare: xqa-bugs, test-flake, unclassified.

Declaring agents.analyst.classifications replaces all five shipped groups, and drops the shipped appContext and guidance copy with them: once you own the vocabulary you own the prose that explains it.

Author guidance when you declare your own groups. It is where the analyst learns which group to fall back to, and the shipped copy that used to say so is gone. Without it the analyst is told it may not emit unclassified and given no fallback, so a cluster it cannot place has nowhere to go.

Declaring ids outside the shipped set requires a viewer worker deployed at or after this release. The night write validates against the viewer's schema, and an older worker rejects an id it does not know with a 400.

A config the schema rejects fails the digest rather than falling back to the shipped groups. Only a missing config file falls back, so a typo in a tone: cannot quietly reclassify a whole night into a vocabulary the repo does not use.

agents:
  analyst:
    appContext: |
      The application under test is a mobile wallet exercised nightly on device.
    guidance: |
      When two groups fit, prefer the one whose owner can act on it today.
    classifications:
      - id: likely-bugs
        heading: Likely bugs
        tone: red
        rated: true
        slack: section
        countsAsBug: true
        summary: |
          a defect in the application, traced well enough that a human should investigate now.
        rules: |
          Earn this only when the defective code path or the incorrect on-snapshot result is
          identified, not merely suspected.
      - id: perf-regressions
        heading: Performance regressions
        pill: [performance regression, performance regressions]
        tone: amber
        emoji: 🐢
        summary: |
          a step that finished but missed its budget.
        rules: |
          Pick this when the step completed and the only failure is how long it took.

| Field | Required | Description | | ------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | yes | Kebab-case, unique, and not one of the three reserved ids. | | heading | yes | The section heading the viewer renders. | | summary | yes | One-line definition, rendered in the analyst prompt's group list. | | rules | yes | The elaboration the analyst reads when deciding between groups. Name the group in it: the prompt renders these bodies back to back, so prose opening with "this" or "it" has no antecedent. | | tone | yes | amber, blue, green, neutral, red or violet. | | pill | no | [singular, plural] count labels. Both default to the lowercased heading; set it to get a real singular. | | emoji | no | Prefix for this group's Slack summary term. | | rated | no | Whether clusters in this group must carry a severity. Defaults to false. Shipped: likely-bugs and maybe-bugs are rated. | | slack | no | section for a bulleted Slack section, summary (default) for a count in the summary line. | | countsAsBug | no | Whether this group feeds the bug trend. Defaults to false. |

The former headline field on a group is gone — the Slack header now carries a severity count instead of per-group counts, and a config still declaring headline is rejected.

Group definitions travel inside each published report, so the viewer renders headings it has never seen. Changing the config does not rewrite published reports: last night's report keeps last night's headings, while the trend strip recounts history under today's countsAsBug. An id that no longer appears in the config still renders — under the id itself, with a neutral tone.

Severity levels

Clusters in a rated group carry a severity: how much the failure hurts a customer, assuming it is real. Severity sorts clusters within each report section and each Slack section, puts one count in the Slack header (🌙 Nightly QA — 2 showstoppers · 18/24 legs), prefixes each Slack bullet with the level's emoji, and prints a legend in the digest footer. The analyst must rate every cluster it places in a rated group and may not rate any other — both are enforced by the verdict schema.

XQA ships showstopper, high and low, in that order — list order is rank, first is most severe. Declaring agents.analyst.severities replaces the shipped levels wholesale; unlike classifications, the shipped levels survive when you declare only classifications, because they name no group id.

agents:
  analyst:
    severities:
      - id: showstopper
        label: [showstopper, showstoppers]
        tone: red
        emoji: '🛑'
        headline: when-present
        rules: |
          Anything blocking completion or signing of a transaction. Crashes. Security issues.
      - id: high
        tone: amber
        emoji: '🔶'
        rules: |
          Anything between low and showstopper. When you cannot tell, it is high.
      - id: low
        tone: neutral
        emoji: '⚪'
        rules: |
          Low visibility or minor UI issues.

| Field | Required | Description | | ---------- | ------------------- | ------------------------------------------------------------------------------------------------------- | | id | yes | Kebab-case, unique. | | label | no | [singular, plural] for the pill and the headline term. Defaults to the id twice. | | tone | yes | amber, blue, green, neutral, red or violet. | | emoji | yes | Slack bullet marker and legend entry. | | headline | no, default never | when-present prints the count when non-zero, always prints it at zero. At most one level may print. | | rules | yes | Prose telling the analyst when to pick this level. When two levels fit, the lower one wins. |

Severity levels travel inside the report like classification groups, and an id no level describes degrades to a neutral pill. Severity is stored in nightly history, so a recovered cluster shows how bad it was — unless its group is unrated, in which case the stored value is suppressed.

Migration from legacy env vars

Legacy QA_* and XQA_* environment variables are rejected at startup with a LEGACY_ENV_DETECTED error. Move their values into .xqa/config.yaml:

| Legacy env var | New config path | | ---------------------------- | -------------------------------- | | QA_RUN_ID | run.id | | QA_EXPLORE_TIMEOUT_SECONDS | agents.explorer.timeoutSeconds | | QA_BUILD_ENV | agents.explorer.buildEnv | | QA_DISMISSALS_PATH | run.dismissalsPath | | XQA_SUITES_DIR | suites.directory |

Architecture

Key files and directories:

• src/index.ts — CLI entry point; wires commander commands and manages graceful shutdown via process locks
• src/commands/ — Command implementations (init, update, explore, spec, review, completion)
• src/suite/ — Suite runner: config parsing, work-item building, worker pool, hooks
• src/core/ — Pure functions: completion generation, verbose/timeout option parsing, last-path tracking
• src/shell/ — I/O wrappers: app/explore context reading, debug logging, display factory, preflight, xqa directory discovery
• src/config.ts, src/config-schema.ts — Configuration loading and validation with Zod
• src/review-session.ts — Interactive finding review loop with dismissal tracking
• src/spec-frontmatter.ts — Spec markdown frontmatter parsing (YAML)
• src/spec-slug.ts — Spec filename to slug derivation for output organization
• src/pid-lock.ts — Process-level mutual exclusion to prevent concurrent runs

Error Types

Core error discriminated unions:

• ConfigError — Configuration validation failed (INVALID_CONFIG)
• AppContextError — Failed to read app.md or explore.md (READ_FAILED)
• XqaDirectoryError — No .xqa directory found (XQA_NOT_INITIALIZED)
• SpecFrontmatterError — Malformed spec markdown (MISSING_FRONTMATTER, MISSING_FIELD, PARSE_ERROR)
• LastPathError — No findings path provided and no prior session (NO_ARG_AND_NO_STATE)
• SuiteConfigError — Suite config JSON malformed or schema-invalid (INVALID_SUITE_CONFIG)
• HookError — Suite hook failure (HOOK_SPAWN_FAILED, HOOK_EXIT_NONZERO, HOOK_TIMEOUT, HOOK_ABORTED)

Development

Install dependencies:

pnpm install

Build the CLI:

pnpm run build

Run tests:

pnpm run test

Type check:

pnpm run typecheck

Lint and format:

pnpm run lint
pnpm run lint:fix

Full quality check (lint, typecheck, test):

pnpm run check
pnpm run check:fix

Watch mode (build + re-run on file changes):

pnpm run dev

Link binary globally (symlinks dist/xqa.cjs to ~/.local/bin/xqa):

pnpm run build:link

Unlink binary:

pnpm run build:unlink

Project Structure

src/
  index.ts                    # CLI entry point
  config.ts                   # Config loading and types
  config-schema.ts            # Zod schema for env vars
  constants.ts                # Tool lists and timeouts
  pid-lock.ts                 # Process exclusion lock
  spec-slug.ts                # Spec file to slug conversion
  spec-frontmatter.ts         # Spec YAML parsing
  review-session.ts           # Interactive finding review loop

  commands/
    init-command.ts           # Project initialization
    update-command.ts         # Skill updates
    install-skills.ts         # Bundled skill installer
    explore-command.ts        # Breadth-first exploration
    spec-command.ts           # Spec-based exploration
    spec-resolver.ts          # Spec file discovery and parsing
    review-command.ts         # Finding triage workflow
    completion-command.ts     # Shell completion generation
    item-events.ts            # Start/complete/fail event emitters

  core/
    parse-verbose.ts          # Verbose flag parsing
    parse-timeout-seconds.ts  # Timeout flag parsing
    completion-generator.ts   # Bash/zsh completion script generation
    last-path.ts              # Last findings path tracking

  shell/
    app-context.ts            # Read app.md and explore.md
    xqa-directory.ts          # Locate .xqa directory
    preflight.ts              # Environment preflight checks
    display-factory.ts        # Solo and suite display factories
    debug-logger.ts           # Debug event logging
    debug-agent-events.ts     # Agent event debug formatter
    debug-suite-events.ts     # Suite event debug formatter
    debug-logger-core.ts      # Pure logging helpers
    trigger-abort.ts          # Abort signal plumbing

  suite/
    types.ts                  # Suite work-item and findings types
    core/                     # Pure: config parser, item builders, hook env, queue
    shell/                    # I/O: worker pool, hook runner, findings writer
    commands/                 # run-command, execute-item, suite-run-context

  __tests__/
    *.test.ts                 # Test files co-located with src/