nth — the NthCase CLI

Run your NthCase tests from CI or a script, and let the result decide whether the pipeline passes. Its reason to exist is nth trigger <scenarioId> --wait: start a run, wait for its verdict, and set an exit code your pipeline can gate on.

Install

With Node.js 18 or newer — npm installs a self-contained program for macOS (Apple silicon and Intel), Linux (x64 and arm64, with glibc or musl, so Alpine too) and Windows (x64), so nothing else needs installing:

npx @nthcase/[email protected] whoami      # run it without installing; the @version pins this release
npm install -g @nthcase/[email protected]  # or install the `nth` command

npm picks the program: @nthcase/cli names one package per computer as an optional dependency, and npm downloads only the one whose operating system, processor and C library match yours. So keep optional dependencies: installed with --omit=optional or --no-optional, or from a lockfile made on another operating system, the program is left out, and nth says so and exits 11 — reinstall without omitting optional dependencies.

With .NET 10 or newer, as a .NET tool:

dotnet tool install -g nthcase --version 0.1.8

Then nth --help. nth --version says which release you are running.

Auth — from the environment, and only from there

export NTH_API_KEY=gp_your_key   # required: mint one in NthCase → Settings → API keys

The key is environment-only by design: there is no --api-key flag, because a flag lands in shell history, in ps, and in CI logs. The key never appears in output, in an error, or in a --verbose line.

nth talks to https://api.nthcase.com. NTH_API_URL points it elsewhere and must be an https:// address: plain http:// is refused before anything is sent, unless it is this machine itself (localhost, 127.0.0.1 or ::1). nth follows no redirect, so your key goes only to the address you configured.

Commands

Every command takes --json (output a program can read: the API's own record) and --verbose (extra diagnostics on stderr). nth <command> --help says more about each.

| Command | Does | |---|---| | nth whoami | the key's workspace {id, name} — verifies the key + config | | nth trigger <scenarioId> [--wait] [--timeout <sec>] [--model <m>] [--device-spec <json>] | trigger a run — the CI command | | nth runs list [--scenario <id>] [--status <s>] [--failed] | recent runs (paged) | | nth runs show <runId> | one run summary (status, timing, cost) | | nth runs logs <runId> [--since <n>] | redacted device/server logs | | nth runs network <runId> | captured network entries (sensitive headers and saved secrets masked), or why there are none | | nth runs cancel <runId> [--force] | stop a run — a queued run cancels outright; an executing one needs --force. The record goes terminal at once; the device finishes the step it is already on before it stops | | nth runs detail <runId> | one run WITH its per-step results; --json carries each step's full record | | nth scenarios list | the projected lookup table (id, name, state, source, stepCount, updatedAt) | | nth scenarios show <scenarioId> | one scenario, full definition, and each action that stands between it and a schedule, with its reason | | nth scenarios compile <scenarioId> [--wait] [--timeout <sec>] [--model <m>] [--operation-id <guid>] | compile a scenario into the plan its runs replay — COSTS CREDITS, and the new plan is what future runs (scheduled ones included) replay by default. --wait exits 0 Completed / 1 Failed / 3 timeout | | nth scenarios compile-run <scenarioId> <compileRunId> | one compile-run — the authoritative outcome of a compile; poll until terminal | | nth scenarios plans <scenarioId> | every compiled plan version plus the replay pointer; an unreadable version's UNTRUSTED prints unreadable, never 0 | | nth scenarios set-plan <scenarioId> [--version <n>] | choose which compiled version unnamed runs replay — omit --version to clear back to 'follow the newest' | | nth scenarios decisions <scenarioId> [--version <n>] | the compile pipeline's per-action audit trail (--json for the full detail) | | nth scenarios create --file <path\|-> | create a scenario from JSON (the same DTO the MCP tool takes) — only ever a DRAFT: compile before it runs, approving stays human-only | | nth scenarios update <scenarioId> --file <path\|-> | edit a scenario from JSON — RESETS it to DRAFT; it cannot run until compiled again (costs credits) | | nth scenarios unpin <scenarioId> <stepOrder> --selector <s> | take a pin off one step: the selector (or drop end) it carries now, which the server checks before it changes anything — RESETS the scenario to DRAFT; compile it again before relying on a run | | nth scenarios baselines <scenarioId> | every picture a person approved as the scenario's visual baselines — per step, one per screen, with who approved it and through which door, and the run it came from; --json adds each run's device and the picture's address, which opens for minutes. Approving stays a person's, on the run's page | | nth scenarios delete-baseline <scenarioId> <baselineId> --yes | delete one approved baseline — CANNOT BE UNDONE: the next run of that step on that screen asks for its picture to be approved again; without --yes nothing is deleted | | nth schedules list [--search <s>] [--status <s>] | list schedules — check here BEFORE creating one; a duplicate runs forever on its own cadence | | nth schedules show <scheduleId> | one schedule AND which scenarios it runs (state + pinned plan per member); the right call after a create that timed out | | nth schedules preview <cron> <tz> [--count <n>] | the next firings of a cron + timezone, computed server-side — the check on the timezone, not just the cron | | nth schedules estimate <scenarioIds> [--model <m>] [--cron <c>] [--tz <t>] | the advisory price tag of a schedule before it exists — per run, per cycle, and per month when cron+tz are given | | nth schedules create --name <n> --cron <c> --tz <t> [--scenarios <ids>] [--member <id:ver>] […] | create an UNATTENDED RECURRING SPEND — no operation id: a retried timeout makes a SECOND schedule; recover with list/show, never a retry | | nth schedules update <scheduleId> --name <n> --cron <c> --tz <t> […] | FULL REPLACE (members wholesale; the firing clock recomputes from now); enabled/paused state untouched | | nth schedules pause <scheduleId> | stop future cron firings (idempotent, no confirmation) — a paused schedule is NOT inert: run-now still fires it, deliberately | | nth schedules resume <scheduleId> | re-enable — the next run recomputes FROM NOW; missed cycles are never fired retroactively; an unevaluable cron/tz is refused honestly | | nth schedules run-now <scheduleId> | fire ONE Manual cycle through the same launcher and billing gates as cron — A SPEND (check schedules estimate first); works on a PAUSED schedule; NO idempotency across calls: every call is a NEW billed cycle — check schedules show before retrying | | nth discovery trigger <targetId> [--wait] [--timeout <sec>] [--focus <f>] [--avoid <a>] [--max-steps <n>] [--max-tokens <n>] [--max-pages <n>] [--model <m>] [--operation-id <guid>] [--device-spec <json>] | start a discovery — COSTS CREDITS and holds a real device for minutes; budgets are clamped server-side and the response echoes what was authorized. --wait exits 0 Completed / 1 Failed / 2 Error / 4 Cancelled / 3 timeout | | nth discovery cancel <discoveryId> | stop a discovery and stop it spending — the device is released; the partial map is kept but can never ground a compile. Already finished = benign, exit 0 | | nth discovery list [--target <id>] [--page <n>] [--page-size <n>] | list discovery runs, newest first — the only way to obtain a discoveryId | | nth discovery show <discoveryId> | one discovery's live state — the phase line says what it is doing while it runs | | nth discovery detail <discoveryId> [--include-elements] | the full report; element lists withheld unless asked for (they are large — the same shaping the MCP applies) | | nth discovery logs <discoveryId> [--since <n>] [--source <s>] | REDACTED device/server logs, delta pages like runs logs | | nth discovery pages <discoveryId> [--seq <n>] | the page sources a phone discovery kept, with typed values taken out; --seq prints one page's XML whole to stdout and its facts to stderr | | nth apps register --file <path\|-> [--build <buildId>] | register a NEW app — created PENDING and inert until a workspace member attests ownership in the app UI (that endpoint takes no API key: hand off to a person). --build names a build uploaded first | | nth apps grounding <targetId> | which discovery map a compile grounds in — mode, pin, and what they resolve to; pinning stays human | | nth apps suggestions <targetId> | the AI-proposed scenarios awaiting review, each with its SOURCE (the scope requirement or the discovery) — approving stays human | | nth apps builds <targetId> | the app's uploaded builds, newest first (* marks the one it runs) — name, app id, version, size, when | | nth apps update <targetId> --build <buildId> | make an uploaded build the one the app runs ("Use this build"); a build of a different app or platform is refused in the server's words. Only the build changes | | nth builds upload <path> --platform <Android\|iOS> | upload a build from this machine (.apk, .ipa, or a .zip holding one simulator .app), streamed as it is read; prints the build's id and the app id and version read from the file. The size limit is read first, so a file over it is never sent | | nth devices list [--platform <p>] | the farm's leasable devices, so a run targets one that exists | | nth devices provision --platform <p> [--os-version <v>] [--form-factor <f>] [--profile <p>] [--wait] [--timeout <sec>] | ask the farm to BUILD an emulator — spends the operator's MACHINE (two per node), not money; NO idempotency key: poll before retrying. --wait exits 0 Ready / 1 Failed / 3 timeout | | nth devices provision-status <provisioningId> | poll a provision: Queued / Provisioning / Ready (deviceId) / Failed (the farm's own reason) | | nth catalog [--platform <p>] | the provisionable capability catalog (valid deviceSpec values per platform) | | nth nodes | THIS workspace's connected machines and their health — not a device picker; the value is WHY a phone is not usable yet: the default output carries each outcome's reason and remedy, the unusable devices, and the phones the machine has | | nth notices | print the third-party notices: the licences of the open-source code inside the CLI. Needs no key and sends nothing |

trigger --wait and the exit codes

Without --wait, nth trigger starts the run, prints its id and exits 0. With --wait it checks the run on a gentle interval (backing off when the API asks it to) until the run has a verdict, then exits with the code below. --timeout (default 600 seconds) bounds the wait, so a pipeline never hangs.

When the run ends, it prints the verdict, what the run cost and the link to the run's page. A run that failed adds why: the first step that failed and its reason or, with no step to name, the failure class and cause. A run that passed:

$ nth trigger 7c2f9a14-5b3e-4d8a-a1f0-3e9b6c2d8f51 --wait
Run 4e70d5d5-8f1c-4a3b-9d62-0b7e5c1a9f34 Passed. 2 credits. https://nthcase.com/runs/4e70d5d5-8f1c-4a3b-9d62-0b7e5c1a9f34
$ echo $?
0

A run that failed:

$ nth trigger 9b1f3c57-2e84-4d6a-a0c9-6e5d4b3a2f18 --wait
Run 3a7e19c4-5b2d-4f60-9c81-d24e6b0f7a35 Failed. 2 credits. https://nthcase.com/runs/3a7e19c4-5b2d-4f60-9c81-d24e6b0f7a35
  step 5 failed (assert): Verify the cart reads "Cart (1)"
  reason: Text of [id="cart-count"] is not exactly "Cart (1)"
$ echo $?
1

| Exit | Meaning | |---|---| | 0 | --wait: run Passed; or any non-wait command succeeded | | 1 | --wait: run Failed (the test-failure signal); or a non-wait command was refused by the API | | 2 | --wait: run Error (no verdict: engine/infra broke, or the run could not start — not a test verdict) | | 3 | --wait: --timeout elapsed before the run had a verdict | | 4 | --wait: run Cancelled — somebody stopped it. Neither a pass, a failure, nor an engine error | | 10 | the command line was wrong: an unknown option, a missing argument, an id that is not an id | | 11 | the key or its configuration: no key; a key the API refused (invalid, revoked, expired, or a workspace plan without API access); an NTH_API_URL that would expose the key; a redirect; an API that has moved on and needs a newer nth; or, from npm, a computer the package has no program for | | 12 | the API could not be reached |

Stopped by Ctrl-C or a stop signal, nth exits 130, the shell's own number for an interrupt, and says so: a run or a discovery it already started keeps running, and spending, until it ends or is cancelled (nth runs cancel <runId>).

CI recipe (GitHub Actions)

- name: Smoke test on a real device
  env:
    NTH_API_KEY: ${{ secrets.NTH_API_KEY }}
  run: npx @nthcase/[email protected] trigger 22222222-2222-2222-2222-222222222222 --wait --timeout 900
  # the job fails on any non-zero exit; the codes are the table above

Store the key as a repository or organisation secret. The recipe talks to https://api.nthcase.com, so it sets no NTH_API_URL.

Read more

The full CLI reference is at https://nthcase.com/docs/cli.

Licence and third-party notices

nth is proprietary software: its use is subject to the NthCase Terms of Service at https://nthcase.com/terms (see LICENSE). It includes open-source components under their own licences, listed in THIRD_PARTY_NOTICES.txt beside this file. nth notices prints them.