runhuman

AI-orchestrated human QA testing from your terminal

What is Runhuman?

Runhuman is an AI-orchestrated human QA testing service. Get real human testing of your web applications on-demand, with structured results extracted automatically.

Perfect for:

• AI coding agents that need human verification
• CI/CD pipelines with human QA gates
• Visual/UX testing requiring real human judgment
• On-demand QA without hiring/managing teams

Installation

# Install globally
npm install -g runhuman

# Or use with npx
npx runhuman --help

Quick Start

# 1. Login (opens browser for OAuth)
runhuman login

# 2. Create your first test
runhuman create https://google.com \
  -d "Search for 'recursion' and confirm the tester experiences a joke about recursion"

# 3. Check the status
runhuman status job_abc123

# 4. Get results
runhuman results job_abc123

Truncated IDs

All commands accept truncated ID prefixes, similar to git short hashes. Instead of typing a full ID, you can use just the first few characters:

# Full ID
runhuman status 712e42ad-8f3b-4a1c-9d5e-1234567890ab

# Truncated — resolves to the same job
runhuman status 712e

# Hyphens are ignored during matching
runhuman status 712e42ad8f3b

If a prefix matches multiple resources, you'll be prompted to provide more characters. Matching is case-insensitive and hyphen-insensitive.

Destructive operations (projects delete, keys delete, projects transfer --to-org) require the full ID to prevent accidental mismatches.

Commands

Jobs

create [url]

Create a new QA test job.

# Basic usage
runhuman create https://google.com \
  -d "Search for 'recursion' and confirm the tester experiences a joke about recursion"

# With template
runhuman create https://google.com --template tmpl_abc123

# Synchronous (wait for result)
runhuman create https://google.com -d "Test search" --sync

# With custom schema
runhuman create https://google.com -d "Test search" --schema ./schema.json

# With tester pool requirements
runhuman create https://google.com -d "Test mobile layout" \
  --required-devices ios,android \
  --required-languages english

Options:

• -d, --description <text> - Test instructions for the human tester
• -t, --template <name> - Use a pre-defined template
• --duration <minutes> - Target duration (1-60 minutes)
• --device-class <class> - Device class: desktop or mobile
• -S, --schema <file> - Path to JSON schema for structured output
• --schema-inline <json> - Inline JSON schema string
• --metadata <json> - Metadata for tracking (JSON string)
• --github-repo <owner/repo> - GitHub repo for context
• --required-devices <devices> - Required tester devices (comma-separated: ios, android, pc, mac)
• --required-languages <languages> - Required tester languages (comma-separated: english, spanish)
• --require-social-videos - Require tester to create social videos
• -s, --sync - Wait for result before exiting
• --wait <seconds> - Max wait time in sync mode (default: 300)
• -p, --project <id> - Project ID (or use default from config)
• -j, --json - Output as JSON
• -q, --quiet - Minimal output (only job ID)

status <jobId>

Check the status of a test job.

runhuman status job_abc123
runhuman status 712e    # truncated ID works

wait <jobId>

Wait for a job to complete with live status updates.

runhuman wait job_abc123
runhuman wait job_abc123 --timeout 300

Options:

• --timeout <seconds> - Maximum wait time (default: 600)
• -j, --json - Output as JSON

results <jobId>

Display detailed test results.

runhuman results job_abc123
runhuman results job_abc123 --json
runhuman results job_abc123 --schema-only   # Only extracted schema data
runhuman results job_abc123 --raw           # Raw tester response

Options:

• --schema-only - Show only extracted schema data
• --raw - Show raw tester response without processing
• -j, --json - Output as JSON

list [filter]

List all your test jobs with optional status filtering.

# List all jobs
runhuman list

# Filter by status (positional argument)
runhuman list completed
runhuman list pending

# Filter by project
runhuman list --project proj_abc123

# Pagination
runhuman list --limit 20 --offset 40

Arguments:

• [filter] - Status filter: all, pending, claimed, in_progress, completed, failed, timeout

Options:

• -p, --project <id> - Filter by project
• -n, --limit <number> - Maximum number of jobs (default: 20)
• --offset <number> - Pagination offset (default: 0)
• --format <type> - Output format: table, json, compact
• -j, --json - Output as JSON

delete <jobId>

Delete a job permanently.

runhuman delete job_abc123 --force

Options:

• -f, --force - Skip confirmation prompt
• -j, --json - Output as JSON

Organizations

orgs list

List all organizations you belong to.

runhuman orgs list
runhuman orgs ls    # alias

orgs show <organizationId>

Show organization details.

runhuman orgs show org_abc123
runhuman orgs info org_abc123   # alias

orgs balance <organizationId>

Check billing balance.

runhuman orgs balance org_abc123

orgs projects <organizationId>

List all projects in an organization.

runhuman orgs projects org_abc123

orgs switch <organizationId>

Set the default organization for CLI commands.

runhuman orgs switch org_abc123
runhuman orgs switch org_abc123 --global   # all directories (default)

Projects

projects list

List all your projects.

runhuman projects list
runhuman projects ls    # alias

projects create <name>

Create a new project in an organization.

runhuman projects create "My App" --organization org_abc123
runhuman projects create "My App" --organization org_abc123 --set-default

Options:

• -o, --organization <id> - Organization ID (required)
• --default-url <url> - Default URL for tests
• --github-repo <owner/repo> - Link GitHub repository
• --set-default - Set as default project after creation

projects show <projectId>

Show project details including organization and balance.

runhuman projects show proj_abc123
runhuman projects info proj_abc123    # alias
runhuman projects get proj_abc123     # alias

projects update <projectId>

Update project settings.

runhuman projects update proj_abc123 --name "New Name"
runhuman projects update proj_abc123 --default-url https://google.com

Options:

• -n, --name <text> - New project name
• -u, --default-url <url> - New default URL
• -g, --github-repo <owner/repo> - New GitHub repository

projects switch <projectId>

Set the default project for CLI commands.

runhuman projects switch proj_abc123
runhuman projects use proj_abc123     # alias

Options:

• -g, --global - Set as global default instead of local (default: true)

projects transfer <projectId>

Transfer a project to another organization.

runhuman projects transfer proj_abc123 --to-org org_target123

Options:

• --to-org <organizationId> - Target organization ID (required, full ID required)
• -f, --force - Skip confirmation prompt

projects delete <projectId>

Delete a project permanently. Requires the full project ID.

runhuman projects delete proj_abc123 --force

Options:

• -f, --force - Skip confirmation prompt

Transfers

transfers list

List pending and outgoing project transfers.

runhuman transfers list
runhuman transfers ls    # alias

transfers accept <transferId>

Accept an incoming transfer.

runhuman transfers accept xfer_abc123

transfers reject <transferId>

Reject an incoming transfer.

runhuman transfers reject xfer_abc123

transfers cancel <transferId>

Cancel an outgoing transfer you initiated.

runhuman transfers cancel xfer_abc123

API Keys

keys list

List all API keys for an organization.

runhuman keys list --organization org_abc123
runhuman keys ls --organization org_abc123    # alias
runhuman keys list --organization org_abc123 --show-keys

Options:

• -o, --organization <id> - Organization ID (required)
• --show-keys - Show full API keys (default: masked)
• -j, --json - Output as JSON

keys create <name>

Create a new API key for an organization.

runhuman keys create "CI/CD Key" --organization org_abc123
runhuman keys new "CI/CD Key" --organization org_abc123    # alias

Options:

• -o, --organization <id> - Organization ID (required)
• --copy - Copy key to clipboard

keys show <keyId>

Show details of a specific API key.

runhuman keys show key_abc123
runhuman keys show key_abc123 --show-key

Options:

• --show-key - Show full API key (default: masked)

keys delete <keyId>

Delete an API key permanently. Requires the full key ID.

runhuman keys delete key_abc123 --force
runhuman keys rm key_abc123 --force       # alias
runhuman keys revoke key_abc123 --force   # alias

Options:

• -f, --force - Skip confirmation prompt

Templates

templates list

List all templates for a project.

runhuman templates list --project proj_abc123
runhuman templates ls --project proj_abc123    # alias

Options:

• -p, --project <id> - Project ID (required, or use default from config)

templates create <name>

Create a new template.

runhuman templates create "Search Flow Test" --project proj_abc123 \
  -d "Search for 'recursion' and confirm the joke appears" \
  --duration 180 \
  --device-class desktop \
  --schema ./schema.json

Options:

• -p, --project <id> - Project ID (required, or use default from config)
• -d, --description <text> - Template description
• --duration <seconds> - Target test duration in seconds
• --device-class <class> - Default device class (desktop/mobile)
• -S, --schema <path> - Path to JSON schema file

templates show <templateId>

Show template details.

runhuman templates show tmpl_abc123 --project proj_abc123

templates update <templateId>

Update a template.

runhuman templates update tmpl_abc123 --project proj_abc123 --name "New Name"

Options:

• -n, --name <name> - New template name
• -d, --description <text> - New description
• --duration <seconds> - New target duration
• --device-class <class> - New default device class
• -S, --schema <path> - Path to new JSON schema file

templates delete <templateId>

Delete a template.

runhuman templates delete tmpl_abc123 --project proj_abc123 --force

Options:

• -f, --force - Skip confirmation prompt

GitHub Integration

github link <repo>

Link a GitHub repository to your project.

runhuman github link owner/repo --project proj_abc123

Options:

• -p, --project <id> - Project ID (required)

github repos

List GitHub repositories accessible to an organization.

runhuman github repos --organization org_abc123
runhuman github repos --organization org_abc123 --search "my-app"

Options:

• -o, --organization <id> - Organization ID (required)
• -s, --search <query> - Filter by repository name

github issues <repo>

List GitHub issues for a repository.

runhuman github issues owner/repo
runhuman github issues owner/repo --state open
runhuman github issues owner/repo --labels "bug,needs-qa"

Options:

• -s, --state <state> - Filter by state (open/closed/all, default: open)
• -l, --labels <labels> - Filter by comma-separated labels

github test <issueNumber>

Create a QA test job for a GitHub issue.

runhuman github test 123 -r owner/repo -u https://google.com
runhuman github test 123 -u https://google.com --sync

Options:

• -r, --repo <owner/repo> - Repository (or use default from config)
• -u, --url <url> - URL to test (required)
• -t, --template <id> - Template ID to use
• -s, --sync - Wait for result before exiting

github bulk-test

Create QA test jobs for multiple GitHub issues.

runhuman github bulk-test -r owner/repo -u https://google.com
runhuman github bulk-test -r owner/repo -u https://google.com \
  -l "bug,needs-qa" \
  -n 5

Options:

• -r, --repo <owner/repo> - Repository (or use default from config)
• -u, --url <url> - URL to test (required)
• -l, --labels <labels> - Filter issues by comma-separated labels
• -s, --state <state> - Filter by state (open/closed/all, default: open)
• -t, --template <id> - Template ID to use
• -n, --limit <number> - Maximum number of jobs to create (default: 10)

Authentication

login

Login to Runhuman. Opens your browser for OAuth by default.

runhuman login

# Login with API key (skip browser)
runhuman login --token <your-api-key>

# Print auth URL without opening browser
runhuman login --no-browser

logout

Clear saved credentials.

runhuman logout

whoami

Display current user info.

runhuman whoami

Configuration

config get <key>

Get a configuration value.

runhuman config get apiUrl
runhuman config get project

config set <key> <value>

Set a configuration value.

runhuman config set project proj_abc123
runhuman config set color false
runhuman config set project proj_abc123 --global

Options:

• --global - Save to global config instead of project config

config list

List all configuration values.

runhuman config list
runhuman config list --show-secrets

Options:

• --show-secrets - Show API keys (default: masked)

config reset

Reset configuration to defaults.

runhuman config reset --project --force
runhuman config reset --global --force
runhuman config reset --all --force

init

Initialize a new Runhuman project with interactive prompts.

runhuman init
runhuman init --name "My App" --url https://google.com --yes

Creates a .runhumanrc file in the current directory.

Configuration Hierarchy

Configuration is loaded from multiple sources with the following priority (highest to lowest):

1. CLI flags - --api-key, --project, etc.
2. Environment variables - RUNHUMAN_API_KEY, RUNHUMAN_API_URL, etc.
3. Project config - .runhumanrc in current directory
4. Global config - ~/.config/runhuman/config.json
5. Defaults

Environment Variables

RUNHUMAN_API_KEY           # API key for authentication
RUNHUMAN_API_URL           # API base URL (default: https://runhuman.com)
RUNHUMAN_PROJECT_ID        # Default project ID
RUNHUMAN_NO_COLOR=true     # Disable colored output

JSON Output

All commands support -j, --json for machine-readable output:

runhuman create https://google.com -d "Test search" --json
runhuman status job_abc123 --json
runhuman list --json

Exit Codes

• 0 - Success
• 1 - General error / ambiguous ID / full ID required
• 2 - Authentication error
• 3 - Not found / no match for ID prefix
• 4 - Validation error
• 5 - Timeout error

Examples

Basic Workflow

# 1. Login
runhuman login

# 2. Initialize project
runhuman init

# 3. Create a test
runhuman create https://google.com \
  -d "Search for 'recursion' and confirm the tester experiences a joke about recursion"

# 4. Wait for results (truncated ID)
runhuman wait 712e

# 5. View detailed results
runhuman results 712e

CI/CD Integration

runhuman create https://staging.google.com \
  -d "Search for 'recursion' and confirm the joke appears" \
  --sync \
  --json > result.json

How It Works

1. You create a job - Define test instructions and optional output schema
2. Job posted to testers - Request goes to human tester pool
3. Human performs test - Real person tests with video/screenshot recording
4. AI extracts data - AI processes feedback into structured JSON
5. You get results - Clean, typed data ready for automation

Pricing: Pay-per-second ($0.0085/sec of tester time). No monthly fees, no minimums.

Learn More

• Website: runhuman.com
• Documentation: runhuman.com/docs
• GitHub: github.com/volter-ai/runhuman

Support

• Email: [email protected]
• Issues: GitHub Issues

Built by the Volter AI team