@tapground/cli

Run YAML mobile UI tests against the Tapground API from your CI/CD pipeline. AI-grounded, deterministic, JUnit-XML-friendly.

- run: |
    npx --yes @tapground/[email protected] run "tests/**/*.yaml" \
      --platform ios \
      --app "$TAPGROUND_APP_UUID" \
      --binary ./build/MyApp.app.zip \
      --junit-xml results.xml
  env:
    TAPGROUND_API_KEY: ${{ secrets.TAPGROUND_API_KEY }}

Install

# In CI: zero-install via npx, pinned to the version you tested with
npx --yes @tapground/[email protected] run …

# Globally
npm i -g @tapground/cli

# As a devDependency (recommended for repos with package.json)
pnpm add -D @tapground/cli
pnpm exec tapground run …

Requires Node.js 20+.

Quick start

1. Get an API key from Settings → API keys in the Tapground app (tg_live_… format).
2. Note your App UUID (tapground apps list).
3. Add the key as a CI secret named TAPGROUND_API_KEY.
4. Drop a step into your pipeline:

- run: |
    npx --yes @tapground/[email protected] run "tests/**/*.yaml" \
      --platform ios \
      --app a1b2c3d4-… \
      --binary ./build/MyApp.app.zip \
      --junit-xml results.xml
  env:
    TAPGROUND_API_KEY: ${{ secrets.TAPGROUND_API_KEY }}
- uses: actions/upload-artifact@v4
  if: failure()
  with:
    name: tapground-artifacts
    path: tapground-artifacts/

Commands

| Command | What it does | | --------------------------- | ------------------------------------------------------- | | tapground run [paths…] | Submit YAML files, poll, write JUnit, download artifacts | | tapground apps list | List apps in your organization | | tapground builds list | List builds for --app <uuid> | | tapground version | Print CLI + Node versions; --server probes API too |

Run tapground <cmd> --help for the full flag surface.

tapground run — anatomy

Targeting (one of these is required)

| Flag | Description | | ------------------- | ------------------------------------------------------------ | | --app <uuid> + --binary <path> | Upload the binary as a fresh build, then run | | --build <uuid> | Run against an existing build | | --platform <ios\|android> | Required regardless of upload vs. build |

--binary and --build are mutually exclusive. iOS tests run on simulators, so upload a simulator build zipped as .app.zip; an .ipa works only if it is a universal build that also contains a simulator slice.

With CLI 0.3.0+, add --reuse-build to --app + --binary to check for an existing build before uploading:

npx --yes @tapground/[email protected] run "tests/**/*.yaml" \
  --platform ios --app "$TAPGROUND_APP_UUID" \
  --binary ./dist/ios-simulator.app.zip --reuse-build

The CLI compares SHA-256 and byte size within the selected app. Identical bytes reuse the existing build ID and version, even if the current commit or --build-version label differs. A confirmed miss uploads a new build. An API error stops the run; it does not silently upload. Test execution still uses the current YAML snapshot.

This is opt-in: omit the flag to retain the existing upload behavior. You can also set TAPGROUND_REUSE_BUILD=true (1, false and 0 are accepted) or defaults.reuseBuild: true in tapground.yml; --no-reuse-build overrides both. The flag requires --binary and --app and cannot be used with --build. It does not cache compilation: separately rebuilt archives can differ even when their source code is unchanged. NDJSON output emits build.reused with the existing build ID and version instead of upload events on a match.

Variables and secrets

Tests can use {{name}} placeholders. Values for a run come from flags; both flags can be repeated.

tapground run "tests/**/*.yaml" --platform ios --build "$BUILD_ID" \
  --var plan=pro \
  --secret password=APP_PASSWORD

• --var name=value passes a plain value.
• --secret name=ENV_VAR passes a secret whose value is read from the environment variable ENV_VAR; --secret NAME reads $NAME. Secret values are never taken from the command line, so they stay out of shell history and CI logs. A missing variable is a usage error (exit 4).
• Names may contain letters, digits, underscores and hyphens. runId, runShortId, mailboxPrefix and mailboxDomain are built in and can't be set. A name can't be both a variable and a secret.
• Secrets must be at least 4 characters. They can only be used in deterministic steps such as type: and deepLink:, never in plain-English text (do:, validate:, extract:); the API refuses such a run. Run reports show {{name}} instead of the value.

Discovery

Positional arguments accept files, directories, and glob patterns:

tapground run tests/login.yaml
tapground run tests/                              # recurse for *.yaml + *.yml
tapground run "tests/**/*.yaml"                   # quote the glob
tapground run tests/auth/ tests/payments/         # mix

*.yaml and *.yml. Hidden files (.foo.yaml) skipped. Default ignores: node_modules, .git, dist, build, coverage, .next, .turbo, .cache. Override with --ignore <glob> (repeatable) or tapground.yml.

Filter with --grep <substring>:

tapground run "tests/**/*.yaml" --grep login

Empty match → exit 4 with a hint.

Build metadata (auto-derived from CI env)

| Flag | Default source | | ------------------------- | -------------------------------------------------------- | | --build-version | ${branch}-${shortSha} (GitHub / GitLab / Circle / Bitbucket) | | --build-environment | Deprecated and ignored | | --external-id | Prefix: gha-{repo}-{runId}-{attempt} etc., per CI provider | | --device-type | server picks any compatible | | --os-version | server picks any compatible |

Every submitted test gets a unique suffix on this prefix, including when CI matrix jobs or repeated CLI invocations share the same workflow ID. Run submissions are sent once; transient polling failures may be retried.

With --app, linked CLI runs explicitly use the submitted YAML snapshot. App re-runs use the current saved definition.

The CLI also sends testSource: the YAML file path and, when available from the CI provider, its repository HTTPS URL, commit SHA and branch. These describe the test source, independently of the uploaded or reused binary. Missing metadata stays unknown; credentials in repository URLs are never sent. The API stores the source on each run, so a later registration cannot change a historical run's commit. This also works when definition registration fails.

The run page checks for an existing saved definition. Open test opens it; Save as test creates an independent, editable Tapground definition when none exists. That copy is not synchronized with the repository. CI-registered definitions remain managed from their source file.

Output + reporting

| Flag | Description | | ---------------------------- | ------------------------------------------------------ | | --junit-xml <path> | Opt-in JUnit XML report. Atomic write (temp + rename). | | --artifacts <mode> | failed (default) / all / off | | --artifacts-dir <path> | Default ./tapground-artifacts | | --logs <mode> | failed (default) / all / off — embeds in JUnit failure/error details | | --output <human\|json> | NDJSON event stream when json | | --quiet / --verbose | Only the summary / also each test's job ID | | --no-groups | Disable GitHub Actions ::group:: markers (auto-on inside GH Actions) | | --no-color | Disable ANSI colors (also honors NO_COLOR env) |

Permanent polling failures print POLL-FAIL with the file, job ID and reason, including in --quiet mode. JSON output emits a poll.fail event. JUnit uses <failure> for failed tests and <error> for infrastructure or polling errors. If a requested JUnit report cannot be written, an otherwise passing run exits 2.

Concurrency + timeouts

| Flag | Default | Description | | -------------------- | ------- | ------------------------------------------------------ | | --max-concurrency | 8 | Cap on parallel submit + poll requests | | --timeout | 30m | Total CLI runtime ceiling (e.g. 30m, 1h30m, 90s) |

Artifact bodies use this total timeout rather than the 30-second API request deadline, so a large recording can keep downloading beyond 30 seconds.

Artifact downloads have a separate fixed pool of 3 (independent of --max-concurrency).

--dry-run

tapground run --dry-run "tests/**/*.yaml"

Discovers + parses + validates YAML against the same schema the worker uses. Surfaces typos as <path>:<key>: <message> and exits 4. No HTTP traffic.

tapground.yml — checked-in defaults

Place at the repo root (or any parent directory of cwd). Shape:

defaults:
  app: a1b2c3d4-…              # App UUID
  platform: ios
  buildVersionFormat: "rc-{shortSha}-{runNumber}"
  device:
    type: "iPhone 15"
    osVersion: "18.0"
  junitXml: ./test-results/tapground.xml
  artifacts: failed
  artifactsDir: ./tapground-artifacts
  logs: failed
  timeout: 30m
  maxConcurrency: 8
  output: human

ignore:
  - tests/manual/**
  - "**/_drafts/**"

Precedence: CLI flag > env var > tapground.yml > built-in default. Override the file path with --config <path> or skip entirely with --no-config.

Build-version template placeholders: {sha}, {shortSha}, {branch}, {runNumber}, {timestamp}.

Environment variables

| Variable | Purpose | | ---------------------------- | --------------------------------------------------------- | | TAPGROUND_API_KEY | Required. API key (tg_live_…) | | TAPGROUND_API_URL | API origin override. Default https://api.tapground.com | | TAPGROUND_APP_URL | App origin override (used for run URLs in JUnit body) | | TAPGROUND_APP | Default --app value | | TAPGROUND_BUILD | Default --build value | | TAPGROUND_REUSE_BUILD | Default --reuse-build (true, false, 1 or 0) | | TAPGROUND_BUILD_VERSION | Default --build-version | | TAPGROUND_BUILD_ENVIRONMENT| Deprecated and ignored | | TAPGROUND_EXTERNAL_ID | Default --external-id | | TAPGROUND_PLATFORM | Default --platform | | TAPGROUND_DEVICE_TYPE | Default --device-type | | TAPGROUND_OS_VERSION | Default --os-version | | TAPGROUND_TIMEOUT | Default --timeout | | TAPGROUND_MAX_CONCURRENCY | Default --max-concurrency | | TAPGROUND_JUNIT_XML | Default --junit-xml | | TAPGROUND_ARTIFACTS | Default --artifacts | | TAPGROUND_ARTIFACTS_DIR | Default --artifacts-dir | | TAPGROUND_LOGS | Default --logs | | TAPGROUND_OUTPUT | Default --output | | TAPGROUND_QUIET | 1 enables --quiet | | TAPGROUND_VERBOSE | 1 enables --verbose | | TAPGROUND_GREP | Default --grep | | NO_COLOR | Disable ANSI colors | | FORCE_COLOR | 1 re-enables colors when not a TTY |

CI provider env vars (GITHUB_ACTIONS, GITLAB_CI, CIRCLECI, BITBUCKET_BUILD_NUMBER, CI) are auto-detected to derive externalId and buildVersion defaults.

Exit codes

| Code | Meaning | | ---- | -------------------------------------------------------- | | 0 | All tests passed | | 1 | At least one test failed (excluding infrastructure/configuration/timeouts) | | 2 | Infrastructure/configuration/server timeout, or JUnit write failure, without test failures | | 3 | Suite incomplete (--timeout exceeded, or a run was cancelled or skipped) | | 4 | Usage or auth error (bad flags or config, YAML parse error, missing input, missing API key, HTTP 401/403, Node.js older than 20) | | 130 | Cancelled via Ctrl+C (SIGINT). POSIX 128 + 2. | | 143 | Cancelled via SIGTERM (e.g. CI workflow cancelled). 128 + 15. |

Priority when multiple categories apply: 4 > OS-signal > 3 > 1 > 2 > 0.

CI integration patterns

GitHub Actions

- uses: actions/setup-node@v4
  with: { node-version: 22 }
- run: |
    npx --yes @tapground/[email protected] run "tests/**/*.yaml" \
      --platform ios \
      --app "${{ vars.TAPGROUND_APP_UUID }}" \
      --binary ./build/MyApp.app.zip \
      --junit-xml results.xml
  env:
    TAPGROUND_API_KEY: ${{ secrets.TAPGROUND_API_KEY }}
- uses: dorny/test-reporter@v1
  if: always()
  with:
    name: Tapground
    path: results.xml
    reporter: jest-junit
- uses: actions/upload-artifact@v4
  if: failure()
  with:
    name: tapground-artifacts
    path: tapground-artifacts/

Keep the | after run:: without it YAML folds the lines and the trailing backslashes reach the CLI as part of the arguments.

The CLI emits ::group::tests/<file>.yaml — PASS|FAIL markers automatically when GITHUB_ACTIONS=true so per-test progress is collapsible in the run log.

GitLab CI

test_ios:
  image: node:22
  script:
    - npx --yes @tapground/[email protected] run "tests/**/*.yaml"
        --platform ios
        --app "$TAPGROUND_APP_UUID"
        --binary ./build/MyApp.app.zip
        --junit-xml results.xml
  artifacts:
    when: always
    paths:
      - tapground-artifacts/
    reports:
      junit: results.xml

Add TAPGROUND_API_KEY as a masked CI/CD variable.

Troubleshooting

"no test files matched" — quote your glob so the shell doesn't expand it into nothing: "tests/**/*.yaml". Paths are resolved relative to cwd.

"--platform is required" — every tapground run needs --platform ios or --platform android. The CLI doesn't auto-detect from the binary extension (intentional — explicit > magic).

HTTP 401 / 403 — verify TAPGROUND_API_KEY is set, has not been revoked, and has the required permissions. Authentication failures during submission or polling stop further submissions/polls, attempt to cancel known active jobs, and exit 4. The HTTP status is preserved even when the error response body is interrupted. A requested JUnit report still records the partial run.

Versioning

Pre-1.0: minor versions may include breaking changes. Pin tightly in CI (@tapground/[email protected]) and review the CHANGELOG before upgrading.

License

Free to use with the Tapground service. See LICENSE; bundled third-party licences are in dist/THIRD_PARTY_NOTICES.txt.