The verification layer for the agentic coding era.

AI ships code in minutes — verifying it hasn't. testsprite opens your live app, uses it like a real user, and shows your coding agent exactly what broke — so it fixes its own work before a bug ever reaches you.

Proof, in public: verification beats model size. With this CLI in the loop, the cheapest model in the field shipped the most correct app on an open leaderboard — 89%, at half the cost of the priciest one. See the leaderboard →

⭐ Help us reach more developers and grow the TestSprite community. Star this repo!

What is it?

TestSprite is the AI testing platform 100,000+ teams use to test their software, frontend and backend — in the cloud, against the live product, not mocks. This repo is its official CLI.

It puts that platform in your coding agent's hands: the agent verifies every behavior it ships, and what broke comes back as one self-consistent bundle it can act on — no dashboard scraping. Humans drive the same surface from a terminal or CI.

⭐️ Star the Repository

If you find testsprite useful, a GitHub Star ⭐️ would be greatly appreciated — it helps other builders (and their agents) find the project, and stars notify you about new releases.

Quickstart

Requires Node.js 20.19+, 22.13+, or 24+. (No global install? npx @testsprite/testsprite-cli works too.)

npm install -g @testsprite/testsprite-cli
testsprite setup

testsprite setup prompts for your API key, verifies it, and installs the verification-loop skill for your coding agent (claude, cursor, cline, windsurf, antigravity, codex, etc.) — one command, so your agent is wired to verify its own work. Non-interactive (CI / onboarding scripts):

TESTSPRITE_API_KEY=sk-... testsprite setup --from-env --yes --agent claude

Pointing a coding agent (Claude Code, Cursor, Codex, Cline, …) at TestSprite? Have it run testsprite setup first — that installs the verification skill, so the agent knows how to create, run, and triage tests on its own (instead of guessing from this README). New here? Start with the getting-started overview.

Privacy note: interactive runs check the npm registry at most once per 24 h to offer a "new version available" notice — package name only, never your key or data; TESTSPRITE_NO_UPDATE_NOTIFIER=1 disables it. Details in DOCUMENTATION.md → Update notice.

From there, the loop runs on its own — an example session, typed by the coding agent:

# 1 — describe the behavior you want to guarantee, run it, wait
testsprite test create --project proj_8f0f6 --type frontend \
  --plan-from ./checkout-flow.plan.json --run --wait --output json
#   → exits 1: the run failed

# 2 — pull ONE self-consistent failure bundle
testsprite test failure get test_3a9f21c7 --out ./.testsprite/failure

# 3 — the agent reads the bundle, fixes the code, then replays
testsprite test rerun test_3a9f21c7 --wait --output json
#   → exits 0: passed. The test now lives in your durable suite.

Prefer to configure each step by hand (or learn the surface offline with --dry-run first)? See Manual setup and Install & verify.

Commands

| Group | Command | What it does | | --------- | --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | Setup | setup | Start here — one command: configure your API key, verify it, and install the agent verification skill | | | doctor | Environment diagnostic — CLI/Node versions, profile, endpoint, credentials, connectivity, agent skill; exits non-zero on failure | | Auth | auth status | Resolve the active profile to its user, key, env, and scopes | | | auth remove | Remove the active profile from the credentials file | | | usage (alias credits) | Account pre-flight: identity, plus credit balance / plan info when the backend supplies them | | Read | project list / project get | List projects / fetch one by id | | | test list / test get | List tests under a project / fetch one by id | | | test code get | Print (or write) the generated test source | | | test steps | List the latest run's steps with screenshot / DOM pointers | | | test result | Latest result; --history lists a test's prior runs | | | test failure get | The agent entry point: one self-contained latest-failure bundle | | | test failure summary | One-screen triage card (no media download) | | | test diff | Compare two runs — verdict, failure kind, per-step status flips, code-version drift | | Write | test scaffold / test lint | Author plans locally: emit a schema-correct starter, validate plan files offline — no network, no credentials | | | test create / test create-batch | Create a test (or bulk-create from a plan file); --produces / --needs / --category wire BE dependency metadata | | | test update / test delete / test delete-batch | Edit metadata / permanently delete (no restore window; --confirm required) | | | test code put | Replace generated code (etag-guarded) | | | test plan put | Replace a frontend test's plan-steps | | | project create / project update | Manage projects | | | project credential / project auto-auth | Configure backend-test auth: a static injected credential, or auto-refresh login (Pro) | | Run | test run | Trigger a fresh run; --wait blocks until terminal; --all --project <id> runs all tests in a project in wave order | | | test rerun | Cheap replay of one/many tests (FE verbatim; BE with deps); --all --project <id> reruns all tests | | | test flaky | Replay a test several times (auto-heal off) and report a stability score | | | test wait | Block on one or more runIds until terminal | | | test artifact get | Download the failure bundle for a specific runId | | Agent | agent install / agent list / agent status | Add, list, or health-check coding-agent skills (pure-local): claude, codex, cursor, cline, antigravity, kiro, windsurf, copilot |

The earlier command names — init, auth configure, auth whoami, auth logout — still work as hidden, deprecated aliases (each prints a one-line notice pointing at the new name), so existing scripts keep running. auth configure now runs the full setup (it also installs the skill).

📚 Full reference — every command, flag, and example: DOCUMENTATION.md, including configuration & profiles, scripting, and exit codes.

Why a CLI for coding agents?

• 🧪 Tests like a real user. Runs against a live browser or API in the cloud — real clicks, real navigation, real assertions. Not a mock.
• 🤖 Agent-shaped output. test failure get returns one bundle — the failing step, its neighbors, screenshots, DOM snapshots, the test source, a root-cause hypothesis, and a recommended fix target — all sharing a single snapshotId. The CLI refuses to stitch data from two different runs, so an agent never reasons over a frankenstein context.
• ♻️ A loop, not a one-shot. create → run → failure get → fix → rerun — every pass is banked, not thrown away.
• 📐 Scriptable & deterministic. Stable --output json contract, predictable exit codes, and a --dry-run that exercises the full code path offline with canned data.
• 🔌 One command to onboard your agent. testsprite agent install claude drops a ready-made skill file into your repo so your coding agent knows how to drive the loop on its own.

How it works

Every time your agent changes code, it asks one question: is this behavior already covered by the suite?

• Not yet covered → testsprite test create — describe the new behavior, run it.
• Already covered → testsprite test rerun — replay the existing tests, so nothing that used to work breaks silently.
• Something fails → testsprite test failure get — one self-consistent bundle; the agent fixes the code and reruns.

Every pass is banked into a durable suite, so coverage compounds as the project grows — a lasting record of every requirement it has ever gotten right, far bigger than any context window.

flowchart TD
    A["🤖 Your coding agent<br/>Claude Code · Codex · Antigravity · Kimi · Cursor · Trae …"]
    D{"behavior already<br/>covered by the suite?"}
    B["<b>testsprite test create</b><br/>new behavior → new test"]
    R["<b>testsprite test rerun</b><br/>replay the existing tests"]
    C{{"☁️ TestSprite testing agent<br/>runs the test like a real user against<br/>real browsers & real APIs on Cloud"}}
    F["<b>testsprite test failure get</b><br/>ONE self-consistent bundle:<br/>failing step · screenshots · DOM ·<br/>root-cause · recommended fix"]
    S[("📚 Durable integration suite<br/>grows with every pass")]

    A -->|"writes / changes code"| D
    D -->|"no — new behavior"| B
    D -->|"yes"| R
    B --> C
    R --> C
    C -->|"pass ✅"| S
    C -->|"fail ❌"| F
    F -->|"agent reads the bundle<br/>& fixes the code"| A
    S -.->|"defines what's covered"| D

The cloud is a black box on purpose: your agent describes intent and reads results. It never has to know how the test was driven — only what a real user experienced.

Proved in public

On CoderCup — an open leaderboard where frontier coding agents build the same app under the same rules, with TestSprite as the referee — the cheapest model in the field shipped the most correct app on the board: 89%, at half the cost of the priciest one.

That's the point of all of this: you no longer need the biggest, most expensive model to ship software you can trust — top-tier quality, without paying top-tier prices, within reach of every team.

Getting help

• 📚 CLI reference — DOCUMENTATION.md
• 🌐 Platform docs — testsprite.com/docs
• 🐛 Issues & feature requests — GitHub issues
• 💬 Quick questions — Discord, or testsprite --help / testsprite test run --help right in your terminal
• 📝 Changelog — CHANGELOG.md

Contributing

Contributions are welcome — the CLI is plain TypeScript/Node (20.19+, 22.13+, or 24+), tested with Vitest, built with tsc. Getting a dev loop running takes a minute:

git clone https://github.com/TestSprite/testsprite-cli.git
cd testsprite-cli && npm install
npm run build       # tsc → dist/ ; try it: node dist/index.js --help
npm test            # unit suite — no network or credentials needed
npm run lint:fix    # ESLint
npm run typecheck   # tsc --noEmit

Pull requests target the dev branch. The full guide — build from source, test tiers, CI gates, branches, project layout — is in CONTRIBUTING.md.

Support — if you need any help, we're responsive on Discord, and feel free to email us at [email protected] too.

License

Apache-2.0 © TestSprite