runhuman

AI-orchestrated human QA testing from your terminal

What is Runhuman?

Runhuman is an AI-orchestrated human QA testing service. Get real human testing of your web applications on-demand, with structured results extracted automatically.

Perfect for:

• AI coding agents that need human verification
• CI/CD pipelines with human QA gates
• Visual/UX testing requiring real human judgment
• On-demand QA without hiring/managing teams

Installation

# Install globally
npm install -g runhuman

# Or use with npx
npx runhuman --help

Quick Start

# 1. Login (opens browser for OAuth)
runhuman login

# 2. Create your first test
runhuman job create https://google.com \
  -d "Search for 'recursion' and confirm the tester experiences a joke about recursion"

# 3. Check the status
runhuman job status job_abc123

# 4. Get results
runhuman job results job_abc123

Job commands live under the runhuman job namespace. The flat shims (runhuman create, runhuman status, runhuman wait, runhuman results, runhuman list, runhuman watch) still work but are deprecated and emit a one-line warning on use; new scripts should use the job form.

Truncated IDs

All commands accept truncated ID prefixes, similar to git short hashes. Instead of typing a full ID, you can use just the first few characters:

# Full ID
runhuman job status 712e42ad-8f3b-4a1c-9d5e-1234567890ab

# Truncated — resolves to the same job
runhuman job status 712e

# Hyphens are ignored during matching
runhuman job status 712e42ad8f3b

If a prefix matches multiple resources, you'll be prompted to provide more characters. Matching is case-insensitive and hyphen-insensitive.

Destructive operations (projects delete, keys delete, projects transfer --to-org) require the full ID to prevent accidental mismatches.

Global Flags

Every command accepts the same four global flags:

• -j, --json — emit machine-readable JSON to stdout (otherwise pretty-printed output)
• -q, --quiet — suppress success and info messages (stderr deprecation/error output is preserved)
• -y, --yes — skip confirmation prompts on destructive operations
• --no-color — disable ANSI color codes (also honored from RUNHUMAN_NO_COLOR=1)

-f, --force is reserved for override-a-guard semantics, not confirmation-skip. Today it only appears on templates create, where it means "overwrite an existing template with the same name."

Output modes

-j, --json is the single output-mode flag. The CLI does not offer a -o, --output / --format choice à la kubectl — pretty output for humans and JSON for scripts are the two supported shapes. Any future structured-text formats (YAML, CSV, etc.) would be added as their own flags rather than via a format matrix.

This also means -o remains bound to --organization on every command that has one, matching the noun-first mental model the rest of the CLI uses.

JSON envelope

Every command that emits JSON uses the same envelope:

// Success
{
  "success": true,
  "data": <payload>,              // see below
  "pagination": {                 // only on list commands
    "total": 42,
    "limit": 20,
    "offset": 0,
    "hasMore": true
  },
  "warnings": ["..."],            // optional, non-fatal server-side warnings
  "timestamp": "2026-04-24T12:34:56.789Z"
}

// Failure
{
  "success": false,
  "error": {
    "message": "...",
    "code": "OPTIONAL_ERROR_CODE",
    "details": { /* shape varies */ }
  },
  "timestamp": "2026-04-24T12:34:56.789Z"
}

Payload conventions:

• List commands (projects list, jobs list, orgs list, etc.) put their items under data.items and their pagination on the envelope. Never data.jobs or data.projects — always data.items.
• Show / create / update commands put the entity directly as data (e.g. data: { id, name, ... }).
• Delete / archive / cancel commands return a structured action payload such as data: { id, deleted: true }, data: { id, archived: true }, data: { id, cancelled: true }. The envelope already carries success: true; data never echoes it.

A generic script can parse any CLI command's output without inspecting which command was run:

runhuman projects list --json | jq '.data.items[].id'
runhuman projects show <id> --json | jq '.data.id'
runhuman projects delete <id> --json | jq '.data.deleted'

Commands

Jobs

All job commands live under the runhuman job namespace. Flat shims (runhuman create, runhuman status, runhuman wait, runhuman results, runhuman list, runhuman watch) still resolve but emit a deprecation warning.

job create [url]

Create a new QA test job.

# Basic usage
runhuman job create https://google.com \
  -d "Search for 'recursion' and confirm the tester experiences a joke about recursion"

# With template
runhuman job create https://google.com --template tmpl_abc123

# Synchronous (wait for result)
runhuman job create https://google.com -d "Test search" --sync

# With custom schema
runhuman job create https://google.com -d "Test search" --schema ./schema.json

# With tester pool requirements
runhuman job create https://google.com -d "Test mobile layout" \
  --required-devices ios,android \
  --required-languages english

Options:

• -d, --description <text> - Test instructions for the human tester
• -t, --template <name> - Use a pre-defined template
• --duration <minutes> - Target duration (1-60 minutes)
• --device-class <class> - Device class: desktop, mobile, or both (default: both)
• -S, --schema <file> - Path to JSON schema for structured output
• --schema-inline <json> - Inline JSON schema string
• --metadata <json> - Metadata for tracking (JSON string)
• --github-repo <owner/repo> - GitHub repo for context
• --required-devices <devices> - Required tester devices (comma-separated: ios, android, pc, mac)
• --required-languages <languages> - Required tester languages (comma-separated: english, spanish)
• --require-social-videos - Require tester capable of recording social-media-style videos (Max+ tier)
• -s, --sync - Wait for result before exiting
• --wait <seconds> - Max wait time in sync mode (default: 300)
• -p, --project <id> - Project ID (or use default from config)
• -j, --json - Output as JSON
• -q, --quiet - Minimal output (only job ID)

job status <jobId>

Check the status of a test job. Aliases: show, get, info.

runhuman job status job_abc123
runhuman job status 712e    # truncated ID works

job wait <jobId>

Wait for a job to complete with live status updates.

runhuman job wait job_abc123
runhuman job wait job_abc123 --timeout 300

Options:

• --timeout <seconds> - Maximum wait time (default: 600)
• -j, --json - Output as JSON

job results <jobId>

Display detailed test results, with optional rich-data sections.

runhuman job results job_abc123
runhuman job results job_abc123 --json
runhuman job results job_abc123 --schema-only   # Only extracted schema data
runhuman job results job_abc123 --raw           # Raw tester response
runhuman job results job_abc123 --transcript --console-logs
runhuman job results job_abc123 --all           # Every rich section

Options:

• --schema-only - Show only extracted schema data
• --raw - Show raw tester response without processing
• --transcript - Include the voice transcript
• --console-logs - Include captured console logs
• --network - Include captured network requests
• --events - Include user-interaction events
• --key-moments - Include key-moments timeline
• --conversation - Include the AI ↔ tester conversation
• -a, --all - Include every rich-data section above
• -j, --json - Output as JSON

Rich-data sections (transcript, logs, network, events, key moments, conversation) are paid-tier gated server-side.

job list [filter]

List all your test jobs with optional status filtering. The [filter] matches the chip labels shown on the web Jobs page.

# List all jobs
runhuman job list

# Filter by status (positional argument, case-insensitive)
runhuman job list Completed
runhuman job list pending

# Filter by multiple labels at once
runhuman job list "Pending,Working,Completed"

# Filter by project, or scope to an entire org
runhuman job list --project proj_abc123
runhuman job list --org org_abc123

# Pagination
runhuman job list --limit 20 --offset 40

Arguments:

• [filter] - Chip label (comma-separated, case-insensitive): all, Pending, Queued, Working, Completed, Cancelled, Expired, Failed. Admins may pass fine-grain labels.

Options:

• -p, --project <id> - Filter by project
• --org <id> - Scope to all projects in an organization (mutually exclusive with --project)
• -n, --limit <number> - Maximum number of jobs (default: 20)
• --offset <number> - Pagination offset (default: 0)
• -j, --json - Output as JSON

job artifact <jobId> <type>

Download a raw session artifact. Useful for piping transcripts, console logs, or network data into your own tools.

# Print the transcript
runhuman job artifact job_abc123 transcript

# Pipe raw console logs into jq
runhuman job artifact job_abc123 console-logs --json | jq '.data.consoleMessages[]'

Arguments:

• <jobId> - Job ID
• <type> - One of: structured-output, transcript, console-logs, network-requests, conversation, events, key-moments

Options:

• -j, --json - Output as JSON (raw artifact shape wrapped in the standard envelope)

Rich artifacts are paid-tier gated server-side; ungated requests return a clean "subscription required" error.

job share <jobId> / job unshare <jobId>

Toggle public sharing for a job. share enables and prints the public URL; unshare disables.

runhuman job share job_abc123
runhuman job unshare job_abc123

job create-issue <jobId> --index <n>

File one extracted finding from a completed job as a GitHub issue. Mirrors the web UI's per-finding "Create Issue" button. Simple form only — the CLI submits the finding unedited; use the web UI or gh issue edit if you want to change title/body.

# File the first extracted finding to the job's default GitHub repo
runhuman job create-issue job_abc123 --index 0

# Override the target repo
runhuman job create-issue job_abc123 --index 0 --repo owner/repo

Required options:

• --index <n> - Zero-based index into job.extractedIssues

Options:

• --repo <owner/repo> - Override the job's default GitHub repo
• -j, --json - Output as JSON

job delete <jobId>

Delete a job (admin-only — customers cannot delete their own jobs; use the dashboard's archive flow to organize your job list).

runhuman job delete job_abc123 --force

Options:

• -f, --force - Skip confirmation prompt
• -j, --json - Output as JSON

There is no job cancel command; once a tester is engaged, jobs run through to a terminal state.

Organizations

orgs list

List all organizations you belong to.

runhuman orgs list
runhuman orgs ls    # alias

orgs show <organizationId>

Show organization details.

runhuman orgs show org_abc123
runhuman orgs info org_abc123   # alias

orgs balance <organizationId>

Check billing balance.

runhuman orgs balance org_abc123

orgs projects <organizationId>

List all projects in an organization.

runhuman orgs projects org_abc123

orgs switch <organizationId>

Set the default organization for CLI commands.

runhuman orgs switch org_abc123
runhuman orgs switch org_abc123 --global   # all directories (default)

orgs usage [organizationId]

Show usage analytics: total jobs, cost, per-project breakdown, over-time series.

# Default: last 30 days
runhuman orgs usage

# Preset period
runhuman orgs usage --period 7d
runhuman orgs usage --period all

# Specific month
runhuman orgs usage --year 2026 --month 4

# Pipe the full response into your own tool
runhuman orgs usage --json | jq '.data.byHour'

Default rendering shows the summary plus the top 5 projects by job count. --json returns the full response including overTime, byHour, and bySource arrays for scripting.

Options:

• --period <window> - 7d | 30d | 90d | all (mutually exclusive with --year/--month)
• --year <y> / --month <m> - Specific time window (month requires year)
• -j, --json - Full response JSON

orgs invitations list|revoke

Manage pending invitations sent via orgs invite.

# List pending invitations
runhuman orgs invitations list --organization org_abc123

# Revoke one
runhuman orgs invitations revoke inv_xyz789 --organization org_abc123 --yes

Projects

projects list

List all your projects.

runhuman projects list
runhuman projects ls    # alias

projects create <name>

Create a new project in an organization.

runhuman projects create "My App" --organization org_abc123
runhuman projects create "My App" --organization org_abc123 --set-default

Options:

• -o, --organization <id> - Organization ID (required)
• --default-url <url> - Default URL for tests
• --github-repo <owner/repo> - Link GitHub repository
• --set-default - Set as default project after creation

projects show <projectId>

Show project details including organization and balance.

runhuman projects show proj_abc123
runhuman projects info proj_abc123    # alias
runhuman projects get proj_abc123     # alias

projects update <projectId>

Update project settings.

runhuman projects update proj_abc123 --name "New Name"
runhuman projects update proj_abc123 --default-url https://google.com

Options:

• -n, --name <text> - New project name
• -u, --default-url <url> - New default URL
• -g, --github-repo <owner/repo> - New GitHub repository

projects switch <projectId>

Set the default project for CLI commands.

runhuman projects switch proj_abc123
runhuman projects use proj_abc123     # alias

Options:

• -g, --global - Set as global default instead of local (default: true)

projects transfer <projectId>

Transfer a project to another organization.

runhuman projects transfer proj_abc123 --to-org org_target123

Options:

• --to-org <organizationId> - Target organization ID (required, full ID required)
• -f, --force - Skip confirmation prompt

projects delete <projectId>

Delete a project. Requires the full project ID.

runhuman projects delete proj_abc123 --force

Options:

• -f, --force - Skip confirmation prompt

Transfers

transfers list

List pending and outgoing project transfers.

runhuman transfers list
runhuman transfers ls    # alias

transfers accept <transferId>

Accept an incoming transfer.

runhuman transfers accept xfer_abc123

transfers reject <transferId>

Reject an incoming transfer.

runhuman transfers reject xfer_abc123

transfers cancel <transferId>

Cancel an outgoing transfer you initiated.

runhuman transfers cancel xfer_abc123

API Keys

keys list

List all API keys for an organization.

runhuman keys list --organization org_abc123
runhuman keys ls --organization org_abc123    # alias
runhuman keys list --organization org_abc123 --show-keys

Options:

• -o, --organization <id> - Organization ID (required)
• --show-keys - Show full API keys (default: masked)
• -j, --json - Output as JSON

keys create <name>

Create a new API key for an organization.

runhuman keys create "CI/CD Key" --organization org_abc123
runhuman keys new "CI/CD Key" --organization org_abc123    # alias

Options:

• -o, --organization <id> - Organization ID (required)

keys show <keyId>

Show details of a specific API key.

runhuman keys show key_abc123
runhuman keys show key_abc123 --show-key

Options:

• --show-key - Show full API key (default: masked)

keys delete <keyId>

Delete an API key. Requires the full key ID.

runhuman keys delete key_abc123 --force
runhuman keys rm key_abc123 --force       # alias
runhuman keys revoke key_abc123 --force   # alias

Options:

• -f, --force - Skip confirmation prompt

Templates

templates list

List all templates for a project.

runhuman templates list --project proj_abc123
runhuman templates ls --project proj_abc123    # alias

Options:

• -p, --project <id> - Project ID (required, or use default from config)

templates create <name>

Create a new template.

runhuman templates create "Search Flow Test" --project proj_abc123 \
  -d "Search for 'recursion' and confirm the joke appears" \
  --duration 180 \
  --device-class desktop \
  --schema ./schema.json

Options:

• -p, --project <id> - Project ID (required, or use default from config)
• -d, --description <text> - Template description
• --duration <seconds> - Target test duration in seconds
• --device-class <class> - Default device class (desktop/mobile/both)
• -S, --schema <path> - Path to JSON schema file

templates show <templateId>

Show template details.

runhuman templates show tmpl_abc123 --project proj_abc123

templates update <templateId>

Update a template.

runhuman templates update tmpl_abc123 --project proj_abc123 --name "New Name"

Options:

• -n, --name <name> - New template name
• -d, --description <text> - New description
• --duration <seconds> - New target duration
• --device-class <class> - New default device class
• -S, --schema <path> - Path to new JSON schema file

templates delete <templateId>

Delete a template.

runhuman templates delete tmpl_abc123 --project proj_abc123 --force

Options:

• -f, --force - Skip confirmation prompt

GitHub Integration

github link <repo>

Link a GitHub repository to your project.

runhuman github link owner/repo --project proj_abc123

Options:

• -p, --project <id> - Project ID (required)

github repos

List GitHub repositories accessible to an organization.

runhuman github repos --organization org_abc123
runhuman github repos --organization org_abc123 --search "my-app"

Options:

• -o, --organization <id> - Organization ID (required)
• -s, --search <query> - Filter by repository name

github issues <repo>

List GitHub issues for a repository.

runhuman github issues owner/repo
runhuman github issues owner/repo --state open
runhuman github issues owner/repo --labels "bug,needs-qa"

Options:

• -s, --state <state> - Filter by state (open/closed/all, default: open)
• -l, --labels <labels> - Filter by comma-separated labels

github test <issueNumber>

Create a QA test job for a GitHub issue.

runhuman github test 123 -r owner/repo -u https://google.com
runhuman github test 123 -u https://google.com --sync

Options:

• -r, --repo <owner/repo> - Repository (or use default from config)
• -u, --url <url> - URL to test (required)
• -t, --template <id> - Template ID to use
• -s, --sync - Wait for result before exiting

github bulk-test

Create QA test jobs for multiple GitHub issues.

runhuman github bulk-test -r owner/repo -u https://google.com
runhuman github bulk-test -r owner/repo -u https://google.com \
  -l "bug,needs-qa" \
  -n 5

Options:

• -r, --repo <owner/repo> - Repository (or use default from config)
• -u, --url <url> - URL to test (required)
• -l, --labels <labels> - Filter issues by comma-separated labels
• -s, --state <state> - Filter by state (open/closed/all, default: open)
• -t, --template <id> - Template ID to use
• -n, --limit <number> - Maximum number of jobs to create (default: 10)

github installation refresh

Force-refresh a GitHub App installation's repo/permissions list. Use this after granting the app access to new repos if they aren't visible yet.

runhuman github installation refresh --installation 987654321 --organization org_abc123

Required options:

• --installation <id> - GitHub installation ID

Options:

• -o, --organization <id> - Organization ID (defaults to current org context)

Billing

User-scoped reads. Purchase flows and plan changes stay in the web UI — use billing portal to get there.

billing balance

Show credit balance for the authenticated user's primary organization (or override with -o).

runhuman billing balance
runhuman billing balance --organization org_abc123

billing subscription

Show the active subscription (tier, billing cycle, renewal date) plus any active add-ons.

runhuman billing subscription

billing portal [--open]

Return a one-time URL to the Polar customer portal (payment method, invoices, cancellation).

# Print the URL
runhuman billing portal

# Open it in the browser
runhuman billing portal --open

Authentication

login

Login to Runhuman. Opens your browser for OAuth by default.

runhuman login

# Login with API key (skip browser)
runhuman login --token <your-api-key>

# Print auth URL without opening browser
runhuman login --no-browser

logout

Clear saved credentials.

runhuman logout

whoami

Display current user info.

runhuman whoami

Configuration

config get <key>

Get a configuration value.

runhuman config get apiUrl
runhuman config get project

config set <key> <value>

Set a configuration value.

runhuman config set project proj_abc123
runhuman config set color false
runhuman config set project proj_abc123 --global

Options:

• --global - Save to global config instead of project config

config list

List all configuration values.

runhuman config list
runhuman config list --show-secrets

Options:

• --show-secrets - Show API keys (default: masked)

config reset

Reset configuration to defaults.

runhuman config reset --project --force
runhuman config reset --global --force
runhuman config reset --all --force

init

Initialize a new Runhuman project with interactive prompts.

runhuman init
runhuman init --name "My App" --url https://google.com --yes

Creates a .runhumanrc file in the current directory.

Configuration Hierarchy

Configuration is loaded from multiple sources with the following priority (highest to lowest):

1. CLI flags - --api-key, --project, etc.
2. Environment variables - RUNHUMAN_API_KEY, RUNHUMAN_API_URL, etc.
3. Project config - .runhumanrc in current directory
4. Global config - ~/.config/runhuman/config.json
5. Defaults

Environment Variables

RUNHUMAN_API_KEY           # API key for authentication
RUNHUMAN_API_URL           # API base URL (default: https://runhuman.com)
RUNHUMAN_PROJECT_ID        # Default project ID
RUNHUMAN_NO_COLOR=true     # Disable colored output

JSON Output

All commands support -j, --json for machine-readable output:

runhuman job create https://google.com -d "Test search" --json
runhuman job status job_abc123 --json
runhuman job list --json

Exit Codes

• 0 - Success
• 1 - General error / ambiguous ID / full ID required
• 2 - Authentication error
• 3 - Not found / no match for ID prefix
• 4 - Validation error
• 5 - Timeout error

Examples

Basic Workflow

# 1. Login
runhuman login

# 2. Initialize project
runhuman init

# 3. Create a test
runhuman job create https://google.com \
  -d "Search for 'recursion' and confirm the tester experiences a joke about recursion"

# 4. Wait for results (truncated ID)
runhuman job wait 712e

# 5. View detailed results
runhuman job results 712e

CI/CD Integration

runhuman job create https://staging.google.com \
  -d "Search for 'recursion' and confirm the joke appears" \
  --sync \
  --json > result.json

How It Works

1. You create a job - Define test instructions and optional output schema
2. Job posted to testers - Request goes to human tester pool
3. Human performs test - Real person tests with video/screenshot recording
4. AI extracts data - AI processes feedback into structured JSON
5. You get results - Clean, typed data ready for automation

Pricing: Pay-per-second ($0.0085/sec of tester time). No monthly fees, no minimums.

Learn More

• Website: runhuman.com
• Documentation: runhuman.com/docs
• GitHub: github.com/volter-ai/runhuman

Support

• Email: [email protected]
• Issues: GitHub Issues

Built by the Volter AI team