Seldon QA

Generate a QA test plan from any codebase, then run AI agents that use your product like a real user — from your terminal or CI.

seldon turns a repository into a baseline of prioritized, user-facing test cases, runs those cases against a live URL with autonomous agents, and returns a clean, actionable report for you or your coding agent to act on.

npm install -g @seldonqa/cli

Why

AI is shipping more code than any team can test by hand. Seldon closes the gap: point it at your repo and it proposes the tests that matter; point it at a URL and agents actually exercise them like users — click, type, navigate, verify — then tell you exactly what broke and where.

Get started

npm install -g @seldonqa/cli
seldon login seldon_your_api_key       # or: seldon config set-key <key>

From inside a project (one baseline per project):

seldon init                            # create seldon.yml, preview what will be analyzed
seldon generate                        # build a QA baseline from this codebase
seldon cases                           # review / scope test cases
seldon run --url https://staging.example.com
seldon logs --follow                   # watch the agents test, live
seldon report --md                     # clean, actionable results

Highlights

• Baselines from your code. seldon generate reads the project and produces a prioritized plan of real user journeys — no scripts to write.
• Agents that behave like users. seldon run drives your live app and judges each case pass / fail / blocked.
• Watch it think. seldon logs --follow streams the agent's reasoning, framed by test case and step.
• Reports built for action. --md for pull requests, --json for coding agents, plain text for the terminal.
• Re-evaluate as code changes. seldon update refreshes the plan when the code moves; pinned cases are always preserved.
• CI-native gate. seldon run --fail-on fail exits non-zero so a red QA run can block a merge.

Configuration — seldon.yml

Configuration lives in a single committed file at your project root (config-as-code). With no file, every command still works with sensible defaults.

baseline:
  run:
    url: https://staging.example.com   # where runs target by default
testCases:
  only: []      # run ONLY these case ids (wins over skip)
  skip: []      # never run these
  fixed: []     # always keep these when the plan is updated (accepts: [ALL])

In CI

seldon run --url "$PREVIEW_URL" --fail-on fail

--fail-on fail returns a non-zero exit code when any case fails; use --fail-on any to also gate on blocked or timed-out cases. With --json the report is stable, machine-readable output you can post to a pull request or hand to a coding agent for an automated fix loop.

Commands

| Command | What it does | | --- | --- | | seldon init | Create seldon.yml and preview what will be analyzed | | seldon generate | Build a QA baseline (test plan) from the current codebase | | seldon cases | Review and scope test cases — only / skip / fixed | | seldon run --url <url> | Run the baseline against a live URL | | seldon logs --follow | Stream the agent's live reasoning (thinking only) | | seldon report [--md \| --json] | Show the latest report | | seldon runs | List QA runs with color-coded status | | seldon cancel --id <run> | Stop a run mid-flight | | seldon update | Re-evaluate the baseline after code changes | | seldon review | PR-style: evaluate your changes and run the affected cases | | seldon list · versions · promote | Manage baselines and versions |

Run seldon --help or seldon <command> --help for the full reference.

Coding agents (MCP)

Seldon is also available as an MCP server, so coding agents can generate baselines, run them, and read reports directly — enabling a closed test-and-fix loop.

Requirements

• Node.js 20 or newer
• A Seldon API key

Run seldon --help to get started.