@testingbot/cli

v1.4.1

Published

10 days ago

CLI tool to run Espresso, XCUITest and Maestro tests on TestingBot's cloud infrastructure

0High
0Medium
0Low

testingbot mobile-testing espresso xcuitest maestro android ios test-automation cloud-testing cli real-devices emulators simulators

Run Espresso, XCUITest and Maestro tests on real devices in the cloud.

Real Devices — Test on thousands of real iOS and Android devices
Emulators & Simulators — Fast feedback with virtual devices
Parallel Execution — Split tests across multiple devices with sharding
CI/CD Ready — Integrates with GitHub Actions, Jenkins, and more
Live Results — Watch tests run in real-time
Artifacts — Download videos, screenshots, and logs

Installation

npm install -g @testingbot/cli

Requirements: NodeJS 20 or higher

Authentication

The CLI requires TestingBot API credentials. You can authenticate in several ways:

Browser Login (Recommended)

testingbot login

This opens your browser for authentication. After logging in, your credentials are saved to ~/.testingbot.

Other Methods

Command-line options: --api-key and --api-secret
Environment variables: TB_KEY and TB_SECRET
Config file: Create ~/.testingbot with content key:secret

Commands

Maestro

Run Maestro UI tests on real devices and emulators/simulators.

testingbot maestro <app> <flows...> [options]

Arguments:

app - Path to your app file (.apk, .ipa, .app, .zip, or an EAS iOS simulator .tar.gz)
flows - One or more paths to flow files (.yaml/.yml), directories, .zip files, or glob patterns

App Options:

| Option | Description | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | --app <path> | Path to the application under test (alternative to the positional app argument) | | --other-app <path-or-url> | Additional companion app to install on the device alongside --app. Accepts a local file path (.apk, .ipa, .app, .zip) or a tb://<appkey> / http(s)://... URL — local paths are uploaded; URLs are passed through to the run as-is. Repeatable, max 4 entries. | | --app-url <url> | Download the app from an http(s) URL instead of a local file: .apk, .ipa, .zip or an EAS Build iOS .tar.gz (the .app inside is extracted automatically). Every positional argument is then a flow. Signed URLs such as EAS links expire after about an hour, so pass a fresh one | | --app-binary-id <projectId> | Reuse the app of a project uploaded earlier (testingbot upload, or any previous run's Project ID) instead of uploading one. Every positional argument is then a flow. The platform is taken from the stored app unless --platform is given |

Device Options:

| Option | Description | | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --device <name> | Device name (e.g., "Pixel 9", "iPhone 16") | | --platform <name> | Platform: Android or iOS | | --deviceVersion <version> | OS version (e.g., "14", "17.2") | | --real-device | Use a real device instead of emulator/simulator | | --device-matrix <cells> | Run every flow on each listed device in one go. Cells are <device>[:<version>][:real], comma-separated or repeatable. Cannot be combined with --device or --deviceVersion; --real-device (or an .ipa app) applies to every cell | | --orientation <orientation> | Screen orientation: PORTRAIT or LANDSCAPE | | --device-locale <locale> | Device locale (e.g., "en_US", "de_DE") | | --timezone <timezone> | Timezone (e.g., "America/New_York", "Europe/London") |

Test Configuration:

| Option | Description | | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --name <name> | Test name for dashboard identification | | --build <build> | Build identifier for grouping test runs | | --groups <names> | Tag the test session with one or more groups (comma-separated). Groups appear on the test in the TestingBot dashboard | | --include-tags <tags> | Only run flows with these tags (comma-separated) | | --exclude-tags <tags> | Exclude flows with these tags (comma-separated) | | --exclude-flows <paths> | Flow files, directories or glob patterns to leave out of the run (comma-separated, repeatable). An excluded flow that another flow still invokes via runFlow is bundled as a subflow but never runs on its own | | -e, --env <KEY=VALUE> | Environment variable for flows (can be repeated) | | --config <path> | Path to a custom Maestro config file (default: config.yaml in project root) | | --maestro-version <version> | Maestro version to use (e.g., "2.0.10") |

Network & Location:

| Option | Description | | ---------------------------- | ----------------------------------------------------------- | | --throttle-network <speed> | Network throttling: 4G, 3G, Edge, airplane, or disable | | --geo-country-code <code> | Geographic IP location (ISO country code, e.g., "US", "DE") |

Tunnel:

| Option | Description | | -------------------------- | ------------------------------------------------------------------------------- | | -t, --tunnel | Start a TestingBot tunnel for this test run (cannot be combined with --async) | | --tunnel-identifier <id> | Identifier for the tunnel, allowing multiple tunnels in parallel |

Output Options:

| Option | Description | | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --async | Start tests and exit without waiting for results | | -q, --quiet | Suppress progress output | | --json | Print results as a single JSON document on stdout (logs move to stderr). Implies --quiet. Exit code 2 when tests fail | | --json-file | Write results as JSON to a file (default: <appId>_testingbot.json in the current directory). Implies --quiet. Exit code stays 0 when tests fail so the pipeline can gate on the file | | --json-file-name <path> | Custom path for the JSON results file (requires --json-file) | | --report <format> | Download report after completion: html, html-detailed, junit or allure | | --report-output-dir <path> | Directory to save reports (required with --report) | | --download-artifacts [mode] | Download test artifacts (logs, screenshots, video). Mode: all (default) or failed | | --artifacts-output-dir <path> | Directory to save artifacts zip (defaults to current directory) |

Advanced Options:

| Option | Description | | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | --shard-split <number> | Split flows into N parallel sessions for faster execution | | --retry <count> | Retry failed flows up to N times (0-2, default 0). Re-runs only the flows (or shards) that failed, the moment they fail, while the rest of the run continues. Cannot be combined with --async. | | --ignore-checksum-check | Skip checksum verification and always upload the app |

Note on --retry: a failed flow/shard is retried immediately — as soon as it fails — without waiting for the other flows in the run to finish. Retry attempts appear live in the flow table marked with a ↻ icon. Each flow is retried independently up to N times, stopping as soon as that flow passes. Pass/fail uses the result of the last attempt per flow (last-attempt-wins), consistently across the CLI exit code, the TestingBot dashboard, and reports downloaded via --report.

CI/CD Integration:

| Option | Description | | ---------------------------- | -------------------------------------------------------------------------------------------------------------------- | | --branch <name> | Git branch this test run was built from | | --commit-sha <sha> | Git commit SHA associated with this test run | | --pull-request-id <id> | Pull request ID this test run originated from | | --pr-url <url> | Pull request URL this test run originated from | | --repo-name <name> | Repository name (e.g., GitHub repo slug) | | --repo-owner <owner> | Repository owner (e.g., GitHub organization or username) | | -m, --metadata <KEY=VALUE> | Free-form metadata attached to the run and shown in the dashboard (repeatable, e.g. -m team=mobile -m env=staging) | | --check-name <name> | Name the GitHub pull request check TestingBot / <name> so several runs on one commit (iOS, Android) post separate checks that branch protection can require individually |

Allure reports: --report allure converts each run's results into Allure result files under <report-output-dir>/allure-results/, one JSON per flow with its steps, status and failure details. Render them with allure serve <report-output-dir>/allure-results (requires the Allure CLI). Results from several runs or shards accumulate in the same directory.

Migrating from Maestro Cloud: the maestro cloud spelling of common flags is accepted as hidden aliases, so an existing command line runs unchanged: --app-file, --flows <a,b>, --apiKey, --device-model iPhone-17-Pro, --device-os iOS-18-2 / android-34, --format JUNIT|HTML, --output <file> (its directory becomes --report-output-dir) and --test-suite-name. The canonical flag wins when both are given.

Examples:

# Basic usage
testingbot maestro app.apk ./flows

# Multiple flow directories
testingbot maestro app.apk ./flows/smoke ./flows/regression ./flows/e2e

# With device selection
testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14"

# Android app on real device with tags
testingbot maestro app.apk ./flows --device "Samsung Galaxy S24" --real-device --include-tags "smoke,regression"

# Tag the test session with groups (visible in the dashboard)
testingbot maestro app.apk ./flows --groups "smoke,critical"

# With environment variables
testingbot maestro app.apk ./flows -e API_URL=https://staging.example.com -e API_KEY=secret

# With companion apps installed alongside the main app (up to 4)
# Each --other-app can be a local file (uploaded) or a tb:// / http(s):// URL (passed through)
testingbot maestro --app main.apk \
  --other-app helper.apk \
  --other-app tb://existing-appkey \
  --other-app https://example.com/mock-server.apk \
  ./flows

# Download JUnit report
testingbot maestro app.apk ./flows --report junit --report-output-dir ./reports

# Download all artifacts (logs, screenshots, video)
testingbot maestro app.apk ./flows --download-artifacts --build "build-123"

# Download artifacts only for failed tests
testingbot maestro app.apk ./flows --download-artifacts failed --artifacts-output-dir ./artifacts

# Use a custom config file
testingbot maestro app.apk ./flows --config .maestro/ci-config.yaml

# Run in background (async)
testingbot maestro app.apk ./flows --async

# Split flows across 3 shards, grouping all flows over 3 parallel sessions
testingbot maestro app.apk ./flows --shard-split 3

# Retry failed flows up to 2 times (re-runs only the flows that failed)
testingbot maestro app.apk ./flows --retry 2

# CI/CD integration with Git metadata
testingbot maestro app.apk ./flows \
  --commit-sha "abc123def" \
  --pull-request-id "42" \
  --repo-owner "myorg" \
  --repo-name "myapp"

Device matrix

Run the same flows across several devices in a single command. Each cell names exactly one device; there is no cross-product, because not every device exists in every OS version. Every flow runs once per device, so the cost is devices × flows, and the CLI prints that summary before submitting.

testingbot maestro app.apk ./flows \
  --device-matrix "Pixel 9:14" \
  --device-matrix "Samsung Galaxy S24:14:real" \
  --device-matrix "Pixel 8"

Each device becomes its own run with its own results, live table rows and dashboard link; --json lists the device per run, and --retry re-runs only the flow that failed on the device it failed on. If any cell is not a valid device/OS combination the whole request is rejected and nothing runs, so a matrix never partially submits.

Organizing flows and subflows

Every top-level flow you pass runs as its own test. A subflow (a reusable flow another flow pulls in with runFlow) should not be passed as a top-level flow — if it is, it runs twice: once standalone and once as part of the flow that calls it.

Maestro has no notion of a "subflow-only" file. A .yaml sitting alongside your real flows is a runnable flow, regardless of its name. Naming it *.shared.yaml does not make Maestro treat it as shared.

Recommended structure — keep subflows in their own directory:

flows/
  login.yaml            # top-level, runs
  checkout.yaml         # top-level, runs
  subflows/
    sign-in.yaml        # only runs when a flow calls it via runFlow

# flows/login.yaml
- runFlow:
    file: subflows/sign-in.yaml
    env:
      APP_ID: com.example.app

Then pass only the directory of top-level flows:

# Runs login.yaml and checkout.yaml; sign-in.yaml is bundled automatically
# (as a runFlow dependency) but never runs on its own.
testingbot maestro app.apk ./flows

When you pass individual files, list only the flows you want to run — their runFlow targets are discovered and uploaded for you:

# Correct: only the top-level flow. sign-in.yaml is bundled automatically.
testingbot maestro app.apk ./flows/login.yaml

# Wrong: sign-in.yaml would run twice.
testingbot maestro app.apk ./flows/login.yaml ./flows/subflows/sign-in.yaml

Other ways to keep a subflow out of a run:

config.yaml globs — list only the folders that hold top-level flows (e.g. flows: ["*.yaml"]), leaving subflow folders out of discovery.
Tags — add tags: [subflow] to the subflow's header and pass --exclude-tags subflow.

Preview before you run. --dry-run prints exactly which flows run standalone and which are bundled as runFlow subflows, without spending any device minutes:

testingbot maestro app.apk ./flows --dry-run

testingbotctl also prints a warning if a flow you passed will run more than once because another top-level flow calls it via runFlow.

Upload once, run many times

testingbot upload pushes an app once and prints a Project ID. Later runs pass that ID with --app-binary-id and skip the upload entirely; each run still gets its own project and results.

testingbot upload app.apk
#   Uploaded app.apk. Project ID: 4321
#   Run flows against it with: testingbot maestro --app-binary-id 4321 ./flows

APP_ID=$(testingbot upload app.apk --json | jq -r .appId)
testingbot maestro --app-binary-id "$APP_ID" ./flows/smoke
testingbot maestro --app-binary-id "$APP_ID" ./flows/regression --device "Pixel 9"

Every maestro run also prints its Project ID after the app upload, so any previous run's ID works with --app-binary-id too. Unchanged binaries are deduplicated by checksum on upload as well; pass --ignore-checksum-check to force a fresh upload.

upload <appFile>

| Option | Description | | ------------------------- | ---------------------------------------------------- | | --ignore-checksum-check | Skip checksum verification and always upload the app | | -q, --quiet | Suppress upload progress |

--json returns { provider, appId, file, url }. Fails with exit code 1 if the upload was rejected.

Expo / EAS Build

eas build produces a download URL rather than a local file, and for iOS simulator builds the artifact is a .tar.gz containing the .app. Both are handled directly:

# iOS simulator build from EAS (tar.gz with the .app inside)
URL=$(eas build --platform ios --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl')
testingbot maestro --app-url "$URL" ./flows --device "iPhone 16"

# Android build from EAS
URL=$(eas build --platform android --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl')
testingbot maestro --app-url "$URL" ./flows --device "Pixel 9"

# A local tar.gz works too
testingbot maestro build.tar.gz ./flows

Inside an EAS Build or EAS Workflows job, the run is tagged automatically with the EAS build id, profile and platform, and EAS_BUILD_GIT_COMMIT_HASH is used as the commit SHA unless --commit-sha is given. testingbot upload <url> accepts URLs as well.

Status, artifacts and list

Commands for working with Maestro projects after they were started, typically together with --async. Every command accepts --api-key / --api-secret, --debug, and the --json, --json-file, --json-file-name output flags described under JSON Output.

# Start tests without waiting and capture the project id
testingbot maestro app.apk ./flows --async --json | jq -r .appId

# Check on it later; --wait blocks with live progress and exits 2 on failure
testingbot status --id 1234
testingbot status --id 1234 --wait

# Fetch reports and artifacts once it finished
testingbot artifacts --id 1234 --report junit --report-output-dir ./reports
testingbot artifacts --id 1234 --download-artifacts failed --artifacts-output-dir ./artifacts

# Browse recent projects
testingbot list
testingbot list --count 25 --offset 25 --json

status --id <projectId>

| Option | Description | | ------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | -w, --wait | Block until every run has finished, showing the same live flow table as a foreground run. Ctrl-C detaches without cancelling the runs | | -q, --quiet | Suppress progress output |

Exit code is 0 while the project is still running (JSON outcome: "running"), 0/2 once it completed, 1 on errors.

artifacts --id <projectId>

| Option | Description | | ------------------------------- | ----------------------------------------------------------------------- | | --report <format> | Download report: html, html-detailed, junit or allure | | --report-output-dir <path> | Directory to save reports (required with --report) | | --download-artifacts [mode] | Download logs, screenshots and video. Mode: all (default) or failed | | --artifacts-output-dir <path> | Directory to save the artifacts zip (defaults to current directory) |

Fails with exit code 1 if the project is still running; use status --wait first.

list

| Option | Description | | ------------------- | ------------------------------------------------- | | --count <number> | Maximum number of projects to return (default 10) | | --offset <number> | Number of projects to skip, for pagination |

Projects are listed newest first with id, name, state, run and flow counts. --json returns { provider, meta: { offset, count, total }, projects: [...] } with a dashboard url per project.

Espresso

Run Android Espresso tests on real devices and emulators.

testingbot espresso [appFile] [testAppFile] [options]

Arguments:

appFile - Path to application APK file
testAppFile - Path to test APK file containing Espresso tests

Device Options:

| Option | Description | | ------------------------------ | ---------------------------------------------------- | | --app <path> | Path to application APK file | | --test-app <path> | Path to test APK file | | --device <name> | Device name (e.g., "Pixel 6", "Samsung.*") | | --platform-version <version> | Android OS version (e.g., "12", "13", "14") | | --real-device | Use a real device instead of an emulator | | --tablet-only | Only allocate tablet devices | | --phone-only | Only allocate phone devices | | --locale <locale> | Device locale (e.g., "en_US", "de_DE") | | --timezone <timezone> | Timezone (e.g., "America/New_York", "Europe/London") |

Test Configuration:

| Option | Description | | ------------------------ | ----------------------------------------------------- | | --name <name> | Test name for dashboard identification | | --build <build> | Build identifier for grouping test runs | | --test-runner <runner> | Custom test instrumentation runner | | --language <lang> | App language (ISO 639-1 code, e.g., "en", "fr", "de") |

Test Filtering:

| Option | Description | | -------------------------------- | --------------------------------------------------------------------- | | --class <classes> | Run tests in specific classes (comma-separated fully qualified names) | | --not-class <classes> | Exclude tests in specific classes | | --package <packages> | Run tests in specific packages (comma-separated) | | --not-package <packages> | Exclude tests in specific packages | | --annotation <annotations> | Run tests with specific annotations (comma-separated) | | --not-annotation <annotations> | Exclude tests with specific annotations | | --size <sizes> | Run tests by size: small, medium, large (comma-separated) |

Network & Location:

| Option | Description | | ---------------------------- | ----------------------------------------------------------- | | --throttle-network <speed> | Network throttling: 4G, 3G, Edge, or airplane | | --geo-location <code> | Geographic IP location (ISO country code, e.g., "US", "DE") |

Tunnel:

Output Options:

| Option | Description | | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --async | Start tests and exit without waiting for results | | -q, --quiet | Suppress progress output | | --json | Print results as a single JSON document on stdout (logs move to stderr). Implies --quiet. Exit code 2 when tests fail | | --json-file | Write results as JSON to a file (default: <appId>_testingbot.json in the current directory). Implies --quiet. Exit code stays 0 when tests fail so the pipeline can gate on the file | | --json-file-name <path> | Custom path for the JSON results file (requires --json-file) | | --report <format> | Download report after completion: html or junit | | --report-output-dir <path> | Directory to save reports (required with --report) |

Examples:

# Basic usage with positional arguments
testingbot espresso app.apk app-test.apk --device "Pixel 8"

# Using named options
testingbot espresso --app app.apk --test-app app-test.apk --device "Pixel 8"

# Real device with specific Android version
testingbot espresso app.apk app-test.apk \
  --device "Samsung Galaxy S24" \
  --platform-version "14" \
  --real-device

# Run specific test classes
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --class "com.example.LoginTest,com.example.HomeTest"

# Run tests with annotations
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --annotation "com.example.SmokeTest" \
  --size "small,medium"

# With network throttling and geolocation
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --throttle-network "3G" \
  --geo-location "DE" \
  --language "de"

# Download JUnit report
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --report junit \
  --report-output-dir ./reports

XCUITest

Run iOS XCUITest tests on real devices and simulators.

testingbot xcuitest [appFile] [testAppFile] [options]

Arguments:

appFile - Path to application IPA file
testAppFile - Path to test ZIP file containing XCUITests

Device Options:

| Option | Description | | ------------------------------ | ---------------------------------------------------- | | --app <path> | Path to application IPA file | | --test-app <path> | Path to test ZIP file | | --device <name> | Device name (e.g., "iPhone 15", "iPad.*") | | --platform-version <version> | iOS version (e.g., "17.0", "18.2") | | --real-device | Use a real device instead of a simulator | | --tablet-only | Only allocate tablet devices | | --phone-only | Only allocate phone devices | | --orientation <orientation> | Screen orientation: PORTRAIT or LANDSCAPE | | --locale <locale> | Device locale (e.g., "DE", "US") | | --timezone <timezone> | Timezone (e.g., "America/New_York", "Europe/London") |

Test Configuration:

| Option | Description | | ------------------- | ----------------------------------------------------- | | --name <name> | Test name for dashboard identification | | --build <build> | Build identifier for grouping test runs | | --language <lang> | App language (ISO 639-1 code, e.g., "en", "fr", "de") |

Network & Location:

| Option | Description | | ---------------------------- | ----------------------------------------------------------- | | --throttle-network <speed> | Network throttling: 4G, 3G, Edge, or airplane | | --geo-location <code> | Geographic IP location (ISO country code, e.g., "US", "DE") |

Tunnel:

Output Options:

| Option | Description | | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --async | Start tests and exit without waiting for results | | -q, --quiet | Suppress progress output | | --json | Print results as a single JSON document on stdout (logs move to stderr). Implies --quiet. Exit code 2 when tests fail | | --json-file | Write results as JSON to a file (default: <appId>_testingbot.json in the current directory). Implies --quiet. Exit code stays 0 when tests fail so the pipeline can gate on the file | | --json-file-name <path> | Custom path for the JSON results file (requires --json-file) | | --report <format> | Download report after completion: html or junit | | --report-output-dir <path> | Directory to save reports (required with --report) |

Examples:

# Basic usage with positional arguments
testingbot xcuitest app.ipa app-test.zip --device "iPhone 16"

# Using named options
testingbot xcuitest --app app.ipa --test-app app-test.zip --device "iPhone 16"

# Real device with specific iOS version
testingbot xcuitest app.ipa app-test.zip \
  --device "iPhone 15 Pro" \
  --platform-version "17.2" \
  --real-device

# iPad in landscape mode
testingbot xcuitest app.ipa app-test.zip \
  --device "iPad Pro" \
  --tablet-only \
  --orientation LANDSCAPE

# With localization settings
testingbot xcuitest app.ipa app-test.zip \
  --device "iPhone 16" \
  --locale "DE" \
  --language "de" \
  --timezone "Europe/Berlin"

# With network throttling and geolocation
testingbot xcuitest app.ipa app-test.zip \
  --device "iPhone 16" \
  --throttle-network "3G" \
  --geo-location "DE"

# Download HTML report
testingbot xcuitest app.ipa app-test.zip \
  --device "iPhone 16" \
  --report html \
  --report-output-dir ./reports

# Run in background
testingbot xcuitest app.ipa app-test.zip \
  --device "iPhone 16" \
  --async

Common Features

Real-time Progress

By default, the CLI shows real-time progress updates including:

Test status updates with actual device names (even when using wildcards)
Device allocation status
Live output from Maestro flows

Use --quiet to suppress progress output.

Graceful Shutdown

Press Ctrl+C to gracefully stop running tests. The CLI will:

Stop all active test runs on TestingBot
Clean up resources
Exit with appropriate status code

Press Ctrl+C twice to force exit immediately.

Report Downloads

All test frameworks support downloading reports after completion:

# JUnit XML format (for CI integration)
--report junit --report-output-dir ./reports

# HTML format (for human viewing)
--report html --report-output-dir ./reports

Artifact Downloads (Maestro only)

Download all test artifacts including logs, screenshots, and video recordings:

testingbot maestro app.apk ./flows --download-artifacts --build "my-build"

Artifacts are saved as a zip file named after the --build value (or with a timestamp if not provided).

Exit Codes

| Code | Meaning | | ---- | -------------------------------------------------------------------------------------------- | | 0 | All tests passed (also for --async, --dry-run, and failed tests with --json-file) | | 1 | CLI or infrastructure error: invalid arguments, missing credentials, upload failure, timeout | | 2 | One or more tests failed |

Distinguishing 1 from 2 lets CI decide whether to retry the job or fail the build.

JSON Output

--json prints one JSON document on stdout and moves all log lines to stderr, so testingbot maestro app.apk ./flows --json | jq works. --json-file writes the same document to disk while keeping the normal console output. Both flags imply --quiet.

{
  "provider": "maestro",
  "outcome": "failed",
  "success": false,
  "appId": 1234,
  "url": "https://testingbot.com/members/maestro/1234",
  "runs": [
    {
      "id": 5678,
      "status": "DONE",
      "passed": false,
      "device": { "name": "Pixel 6", "platform": "Android", "version": "14" },
      "url": "https://testingbot.com/members/maestro/1234/runs/5678",
      "flows": [
        {
          "id": 1,
          "runId": 5678,
          "name": "login",
          "status": "DONE",
          "passed": true,
          "attempt": 1,
          "latest": true,
          "startedAt": "2026-01-01T00:00:00Z",
          "completedAt": "2026-01-01T00:00:30Z",
          "durationSeconds": 30,
          "errors": []
        }
      ]
    }
  ]
}

outcome is one of passed, failed, started (--async), dry-run, or error. On error the document carries an error message and the exit code is 1.
flows (Maestro only) lists every attempt, including --retry re-runs. attempt counts from 1; latest marks the attempt whose verdict counts for the run.
runs is empty for --async, --dry-run, and errors raised before tests were submitted.

Documentation

For more information, visit TestingBot Documentation.

License

MIT

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

Installation

Authentication

Browser Login (Recommended)

Other Methods

Commands

Maestro

Device matrix

Organizing flows and subflows

Upload once, run many times

Expo / EAS Build

Status, artifacts and list

Espresso

XCUITest

Common Features

Real-time Progress

Graceful Shutdown

Report Downloads

Artifact Downloads (Maestro only)

Exit Codes

JSON Output

Documentation

License