hwtest CLI

hwtest is the command-line client for the remote hardware test farm. It uploads a PlayStation executable, submits it as a job, and the farm runs it on a real console and hands back the captured serial log, any files the program wrote, and a structured result. It is the human and CI entry point to the coordinator's REST API.

If you have used a self-hosted CI runner, the model will feel familiar: you submit work, it gets queued against a pool of physical machines, one is leased to your job, and you get artifacts back. The difference is that the "machine" is a 30-year-old console on a bench, and the queue is what stops two people from driving the same console over the same serial cable at once.

Contents

• Build and install
• Configuration
• Authentication
• Quickstart
• Selecting hardware
• Commands
• Job manifest files
• Declared outputs (capturing files)
• Assets (host files for the run)
• Understanding results
• Scripting and CI
• Troubleshooting

Build and install

Run it without installing anything:

npx @consoledev/hwtest devices

Or install it, if you use it often, which also gets you the shorter hwtest:

npm install -g @consoledev/hwtest
hwtest devices

It needs Node 20 or newer and pulls in no dependencies.

From a checkout

The CLI is also a workspace package in the farm monorepo:

pnpm --filter @consoledev/hwtest build      # or: pnpm -r build

That bundles packages/cli/dist/index.js into a single self-contained file with an executable shebang. Without linking the bin, invoke it through node:

node packages/cli/dist/index.js <command> [options]

The rest of this document writes hwtest for brevity.

Configuration

Two things point the CLI at a farm: the coordinator URL and your API token.

| Setting | Flag | Environment | Default | |---|---|---|---| | Coordinator API root | --server <url> | HWTEST_SERVER | https://hwfarm.consoledev.net/api/v1 | | API token | --token <tok> | HWTEST_TOKEN | ~/.hwtest/token (written by hwtest login) |

The API root must include the /api/v1 path segment. A typical setup exports both once per shell:

export HWTEST_SERVER=https://hwfarm.consoledev.net/api/v1
export HWTEST_TOKEN=hwt_live_xxxxxxxxxxxxxxxxxxxxxxxx

Global flags must come after the subcommand. hwtest devices --server URL works; hwtest --server URL devices does not (the parser reads the first word as the command).

Authentication

The farm uses bearer API tokens. Read-only listing endpoints are open, but anything that submits work or changes state requires a token. Get one by signing in to the web UI (GitHub or Discord) and minting a token there, then save it:

hwtest login --token hwt_live_xxxxxxxxxxxx   # verifies, then writes ~/.hwtest/token
hwtest whoami                                # nicolasnoble [admin, user]

login verifies the token against the coordinator before saving it, so a bad paste fails loudly instead of silently. You can also pipe the token on stdin (pbpaste | hwtest login). Once you have one good token you can mint more without the web UI:

hwtest token-create --name ci-pipeline       # prints the secret exactly once
hwtest logout                                 # forget the saved token

Tokens are shown once at creation and only their hash is stored, so copy the secret when it is printed.

Quickstart

Submit a .ps-exe, wait for it to finish, and pull the results:

hwtest submit --exe build/mytest.ps-exe --pool ps1 --watch
# ticket ticket_e48ae3bb-... (QUEUED)  job job_032ca016-...
# [..] QUEUED
# [..] STAGING
# [..] UPLOADING
# [..] DONE

hwtest result ticket_e48ae3bb-... --out ./results
# verdict PASS  (run run_...) -> ./results/result.json
# downloading 1 artifact(s) to ./results
#   RAW_LOG   serial.log   506B  verified -> ./results/serial.log

--watch polls the ticket until it reaches a terminal state (DONE or FAILED), printing each transition. Without it, submit returns the ticket id immediately and you poll yourself with hwtest status <ticketId> --watch.

Selecting hardware

A job is matched to a device by capability, not by name. The relevant flags:

• --pool <id> restricts the job to a pool. The PS1 console pool is ps1. Repeatable.
• --target-family <fam> is the capability family, default ps1.
• --feature <name> requires a device that advertises a capability feature. Repeatable.
• --exclude-feature <name> avoids devices that advertise a feature. Repeatable.

Devices advertise feature labels you can filter on. List them to see what is available:

hwtest devices
# seele-scph1001-0  AVAILABLE  pool=ps1  SCPH-1001 NTSC-U  "SCPH-1001 #1 (seele, NTSC-U)"
# seele-scph9002-6  AVAILABLE  pool=ps1  SCPH-9002 PAL     "SCPH-9002 #6 (PAL, DFO, disc-boot Unirom)"

There is no flag to pin a single device by id; feature filters are how you steer. A common one: the PAL disc-boot console boots slowly and occasionally needs a retry, so it carries a flaky feature label. To keep a quick job off it, exclude that label and let the scheduler pick any of the cart consoles:

hwtest submit --exe build/mytest.ps-exe --pool ps1 --exclude-feature flaky --watch

Commands

All list commands accept --limit <n>. Any command accepts --json to emit raw JSON instead of formatted text.

submit

Upload an executable and submit it as a job.

hwtest submit --exe <path> [options]
hwtest submit <path> [options]            # the .ps-exe may be the first positional

| Flag | Meaning | |---|---| | --exe <path> | The .ps-exe to run. Required (or pass it as the first positional). | | --manifest <path> | A JSON job file (see Job manifest files). | | --output name=glob[=maxBytes] | Declare an expected output file to capture. Repeatable. | | --pool <id> | Restrict to a pool. Repeatable. | | --target-family <fam> | Capability target family. Default ps1. | | --feature <name> | Required capability feature. Repeatable. | | --exclude-feature <name> | Exclude devices advertising this feature. Repeatable. | | --priority <class> | INTERACTIVE, CI, BATCH, or MAINTENANCE. Default INTERACTIVE. | | --run-seconds <n> | Run-phase timeout (ceiling on the program's own runtime). | | --total-seconds <n> | End-to-end timeout including boot and upload. | | --param key=value | A job parameter passed through to the run. Repeatable. | | --asset name[=pcdrvName] | Expose one of your stored assets to the run over PCDRV. Repeatable. | | --idempotency-key <k> | Make the submit idempotent (see below). | | --watch | Poll the ticket to a terminal state after submitting. |

The executable is uploaded by content hash. Re-submitting the identical bytes is a no-op upload; the blob already exists server-side.

Idempotency. Pass --idempotency-key <k> and a repeated submit with the same key returns the same ticket instead of creating a new one. Note that the key covers the whole job spec, including timeout flags, so changing a timeout produces a different job and a new ticket. Use a stable key per logical CI run.

status

Show a ticket's current state, or follow it to completion.

hwtest status <ticketId> [--watch] [--interval <sec>]

--watch polls until DONE/FAILED, printing each state transition. --interval sets the poll period in seconds (default 2).

result

Fetch the result envelope and download the run's artifacts by name.

hwtest result <ticketId> [--out <dir>] [--no-verify]

Writes result.json (the structured result envelope) plus each artifact (for example serial.log) into --out (default the current directory). Every downloaded blob is re-hashed and checked against its content digest; --no-verify skips that check.

Listing

hwtest tickets        # recent tickets, newest first
hwtest runners        # runners and the devices each one serves
hwtest devices        # devices, their console revision/region, and state
hwtest pools          # pools

Assets

Per-account stored files you can hand to a run over PCDRV. See Assets.

hwtest assets                       # list your assets and quota usage
hwtest asset-put <name> --file <p>  # upload/overwrite (or pipe on stdin)
hwtest asset-get <name> --out <p>   # download (or to stdout)
hwtest asset-rm <name>              # delete

Auth

hwtest login --token <tok>   # verify and save a token
hwtest logout                # remove the saved token
hwtest whoami                # show account and roles
hwtest token-create --name <label>   # mint a new token (printed once)

Job manifest files

For anything beyond a few flags, pass a JSON job file with --manifest. The file may set any of these top-level keys; each is optional and merged with sane defaults:

{
  "manifest": {
    "manifestVersion": "v1",
    "declaredOutputs": [
      { "name": "dump", "pathGlob": "*.test.pcm", "maxBytes": 4194304, "contentType": "application/octet-stream" }
    ],
    "capturePlan": [],
    "logPolicy": { "maxBytes": 4194304 }
  },
  "capabilityRequest": {
    "targetFamily": "ps1",
    "requiredFeatures": ["stock-unirom"],
    "excludedFeatures": ["flaky"]
  },
  "poolSelector": { "poolIds": ["ps1"] },
  "timeoutPolicy": { "runSeconds": 60, "totalSeconds": 300 },
  "priorityClass": "CI",
  "params": { "seed": "1234" }
}

Merge order: built-in defaults, then the manifest file, then CLI flags (flags win). The one exception is --output: declared outputs from flags are appended to whatever the file declared, not replaced. A bare --exe submit with no manifest is valid; the defaults produce a v1 manifest with no declared outputs, a 4 MB log cap, the ps1 capability family, and INTERACTIVE priority, and the server fills in timeouts.

Declared outputs (capturing files)

By default a run gives you the serial log and nothing else. If your program writes files over PCDRV (for example a test that dumps *.test.pcm golden data), you must declare them so the runner captures and uploads them. Declaring outputs is also what arms the host-side PCDRV file server for the run.

hwtest submit --exe build/spudump.ps-exe \
  --pool ps1 \
  --output pcm=*.test.pcm \
  --output rev=*.test.rev \
  --watch

hwtest result <ticketId> --out ./goldens

Each --output is name=glob[=maxBytes]. Declared files are promoted to artifacts on the run and downloaded by hwtest result, content-verified by hash.

Assets (host files for the run)

Assets are files you store under your account and expose to a run as PCDRV-readable inputs, the inverse of declared outputs. Upload once, reference per job:

hwtest asset-put my-rom.dat --file ./my-rom.dat
hwtest submit --exe build/loader.ps-exe --asset my-rom.dat=DATA.BIN --watch

--asset name[=pcdrvName] exposes the stored asset name to the run; the optional =pcdrvName is the filename the program sees over PCDRV. hwtest assets shows your stored assets and quota usage.

Understanding results

Lifecycle. A ticket moves QUEUED -> STAGING -> UPLOADING -> RUNNING -> DRAINING and ends DONE or FAILED. The runner powers the console on for the lease, uploads the executable over the serial link, runs it, captures output, then powers the console off and releases the device.

Verdict. The result envelope carries a verdict: PASS, FAIL, ERROR, TIMEOUT, ABORTED, or RUNNER_LOST. The raw serial log is always captured and attached as the serial.log (RAW_LOG) artifact, regardless of verdict, so even a failed or timed-out run gives you the trace.

How a run ends. A run terminates one of two ways:

• Deterministically, if the program signals an exit break. Test binaries built with the uC-sdk exit-break glue emit a debugger halt (HLTD) on exit; the runner sees it, reads back the exit code, and stops immediately. This is the clean path and gives a precise DONE/PASS.
• By timeout, if the program emits no exit signal (a free-running or streaming binary). The run ends when --run-seconds elapses, with verdict TIMEOUT. This is expected for such binaries; the proof of a successful run is a populated serial.log, not the verdict. Set a short --run-seconds so the timeout resolves quickly.

Scripting and CI

• --json on any command emits raw JSON for piping into jq.
• submit --watch and status --watch exit non-zero if the ticket ends FAILED, so a CI step fails when the test fails.
• Errors are returned as { code, message, requestId } and the CLI exits non-zero; include the requestId when reporting a coordinator problem.
• Use a stable --idempotency-key per CI run so retried pipeline steps do not pile up duplicate tickets.

hwtest submit --exe build/mytest.ps-exe --pool ps1 --priority CI \
  --idempotency-key "ci-$GIT_SHA" --run-seconds 30 --watch || exit 1
hwtest result "$(hwtest tickets --json | jq -r '.items[0].ticketId')" --out artifacts

Troubleshooting

• auth_required: Authentication required - no token, or an expired/revoked one. Mint a fresh token in the web UI and hwtest login --token .... Read-only commands (devices, runners, tickets, pools) work without a token; submitting does not.
• unknown command: --server - global flags must come after the subcommand. Use hwtest devices --server URL, or set HWTEST_SERVER.
• Connection refused / wrong host - check HWTEST_SERVER includes /api/v1.
• A submit seems to do nothing new - if you reused an --idempotency-key, you got the original ticket back by design. Use a new key or drop the flag.
• No artifacts beyond serial.log - files written by the program are only captured if you declared them with --output. Declaring outputs also arms the PCDRV host file server for the run.
• Verdict TIMEOUT on a test you expected to pass - the binary emitted no exit signal. Either build it with the exit-break glue for a deterministic verdict, or treat the captured serial.log as the result and lower --run-seconds.