@runhuman/mcp

A Model Context Protocol (MCP) server that enables AI agents to orchestrate on-demand human QA testing through the Runhuman service.

Note: This is the MCP server package. For the main CLI tool, see runhuman (coming soon).

What is this?

This MCP server allows AI agents (like Claude, GPT, or any MCP-compatible AI) to request human testing of web applications. The AI defines what to test and what data format it needs back, and real humans perform the testing with structured results extracted automatically.

Perfect for:

• AI coding agents that need human verification of their work
• Automated workflows requiring human-in-the-loop testing
• CI/CD pipelines with human QA gates
• Visual/UX testing that requires real human judgment

Quick Start

Claude Desktop

1. Get your API key at runhuman.com/dashboard

2. Add to your Claude Desktop config:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "runhuman": {
      "command": "npx",
      "args": ["-y", "@runhuman/mcp", "--api-key=rh_xxxxxxxxxxxxx"]
    }
  }
}

3. Restart Claude Desktop

That's it! Claude can now request human testing:

"Can someone test my checkout page at https://myapp.com/checkout and verify the payment flow works?"

Other MCP Clients

For other MCP clients (VS Code extensions, custom agents, etc.):

# Install globally
npm install -g @runhuman/mcp

# Run with your API key
runhuman-mcp --api-key=rh_xxxxxxxxxxxxx

Or use npx without installation:

npx @runhuman/mcp --api-key=rh_xxxxxxxxxxxxx

Features

Available Tools

The MCP server exposes 17 tools for orchestrating human QA testing. The five most commonly used tools are documented in detail below; the rest are summarized in the Additional Tools section. The full reference (with input schemas and examples) lives at runhuman.com/docs/mcp.

create_job

Create a new human QA testing job with custom instructions.

Parameters:

• url (string, optional): The URL to test
• description (string, required if no template): Natural language instructions for the human tester
• template (string, optional): Template name to use as base configuration
• outputSchema (object, optional): JSON schema defining the structured output format
• targetDurationMinutes (number, optional): Time limit for tester (default: 30, range: 1-60)
• Plus additional options (screen size, GitHub repo, etc.)

Example:

{
  url: "https://myapp.com/checkout",
  description: "Test the checkout flow: add item to cart, proceed to checkout, and verify payment options are displayed correctly.",
  outputSchema: {
    type: "object",
    properties: {
      checkout_loaded: { type: "boolean" },
      payment_options_visible: { type: "boolean" },
      any_errors: { type: "string" }
    }
  },
  targetDurationMinutes: 30
}

Returns: jobId for tracking the job

run_template

⭐ Templates are important! Create a job from a pre-configured template.

Templates let you reuse test configurations without writing full descriptions every time. Perfect for common test scenarios.

Parameters:

• template (string, required): Template name (get from list_templates)
• Any parameter from create_job to override template defaults

Example:

{
  template: "checkout-flow",
  url: "https://staging.myapp.com"  // Override template's default URL
}

Returns: jobId for the created job

wait

Idiomatic polling - Wait for a job to complete and return results automatically.

Polls job status every 10 seconds until completion, timeout, or failure. Blocks until terminal state is reached (just like CLI wait command).

Parameters:

• jobId (string, required): Job ID from create_job or run_template
• timeoutSeconds (number, optional): Maximum wait time (default: 600, max: 3600)

Returns when complete:

• result: Structured test results matching your schema
• status: "completed"
• costUsd: Exact cost in USD
• testDurationSeconds: Time spent by tester
• testerResponse: Raw natural language feedback
• testerName/testerAlias: Tester identification
• Full job details

Example workflow:

// Create job
const job = await create_job({
  url: "https://myapp.com",
  description: "Test the login page"
});

// Wait automatically polls until completion
const result = await wait({ jobId: job.jobId });
// Returns immediately when done - no manual polling needed!

get_job

Get current job status without waiting (quick status check).

Use this for:

• Checking if a job is still running
• Getting quick updates
• Manual polling with your own timing
• Checking multiple jobs in parallel

Parameters:

• jobId (string, required): Job ID to check

Returns: Current job status and details (if completed, includes results)

list_templates

List available templates for your project.

Parameters:

• limit (number, optional): Max templates to return (default: 50)

Returns: Array of templates with names, URLs, and descriptions

Additional Tools

These cover discovery, member management, artifact retrieval, and the full template / schedule CRUD surface. See runhuman.com/docs/mcp for input schemas, example payloads, and role/permission requirements.

Discovery:

• list_organizations — List orgs the authenticated user belongs to (no params).
• list_projects — List accessible projects, optionally filtered by organizationId.

Organization member management:

• list_organization_members — List members of an org with role / status / owner flag / join date. Params: organizationId.
• invite_organization_member — Invite by email. Roles: admin / contributor / viewer (default contributor). Requires admin permission.

Artifact retrieval (paid-tier gated):

• get_test_artifacts — Fetch rich session data for a completed job. Artifact types: structured-output, transcript, console-logs, network-requests, conversation, events, key-moments. Use after wait / get_job to investigate why a test passed or failed.

Template CRUD (beyond list_templates):

• create_template — Create a reusable template (params: projectId, name, optional testDescription, url, outputSchema, targetDurationMinutes, deviceClass, GitHub repos, autoCreateGithubIssues).
• update_template — Update any field on an existing template.
• delete_template — Permanently delete a template (existing schedules using it will fail on next execution).

Schedule management:

• list_schedules — List recurring schedules for a project.
• create_schedule — Create a recurring schedule. Required: projectId, name, templateId, scheduledHour, scheduledMinute, timezone. Frequencies: daily / weekly / biweekly / monthly / once.
• update_schedule — Pause/resume (status), retime, swap templates, or change frequency.
• delete_schedule — Permanently delete (use update_schedule with status: "paused" to temporarily stop without deleting).

Configuration

API URL (Optional)

By default, the server connects to the production API at https://runhuman.com. For local development or testing:

{
  "mcpServers": {
    "runhuman": {
      "command": "npx",
      "args": [
        "-y",
        "@runhuman/mcp",
        "--api-key=rh_xxxxx",
        "--api-url=http://localhost:3000"
      ]
    }
  }
}

Environment Variables

Alternatively, use environment variables:

export RUNHUMAN_API_KEY=rh_xxxxxxxxxxxxx
export RUNHUMAN_API_URL=https://runhuman.com  # optional

Pricing

Runhuman uses pay-per-second pricing:

• $0.0085 per second of tester time
• Duration is rounded up to the nearest second
• Example: 220 seconds of testing = $1.87

Costs are calculated exactly and returned in the API response. No monthly fees, no minimums.

How It Works

1. AI creates job - Your AI agent calls create_job or run_template with test instructions
2. Job posted to testers - Request goes to the human tester pool via Slack
3. Human performs test - A real person tests the application (video/screenshots recorded)
4. Human reports findings - Tester describes what they observed in natural language
5. AI extracts data - GPT-4o processes feedback into structured JSON matching your schema
6. AI gets results - wait automatically polls and returns clean, typed data ready for automation

Example Use Cases

AI Coding Agents

User: "Can you update my checkout page and verify it works?"
Agent: [Updates code, deploys, then uses run_template to verify with checkout template]
Agent: "✅ Updated and verified by human tester. Payment flow works correctly."

CI/CD Pipelines

- name: Human QA Gate
  run: |
    # Deploy preview
    # Call Runhuman API via MCP
    # Fail build if test fails

Visual Testing

Agent: "Please verify the new homepage design looks good on mobile"
[Human tester provides UX feedback]
Agent: "Human feedback: Design looks great, minor spacing issue noted..."

Development

Local Development

# Clone the repo
git clone https://github.com/volter-ai/runhuman.git
cd runhuman

# Install dependencies
npm install

# Build the MCP server
npm run build --workspace=@runhuman/mcp

# Run with local API
cd packages/mcp-server
node dist/index.js --api-key=rh_test_key_123 --api-url=http://localhost:3000

Testing

# Run tests
npm run test --workspace=@runhuman/mcp

# Use MCP Inspector (interactive testing)
npm run test:inspector --workspace=@runhuman/mcp

Project Structure

packages/mcp-server/
├── src/
│   ├── index.ts                 # CLI entry point (stdio)
│   ├── mcp-server-factory.ts    # Core server factory
│   ├── lib.ts                   # Library exports
│   ├── types.ts                 # TypeScript types
│   └── tools/                   # Tool implementations
│       ├── create-job.tool.ts
│       ├── run-template.tool.ts
│       ├── wait.tool.ts
│       ├── get-job.tool.ts
│       └── list-templates.tool.ts
├── dist/                        # Compiled output
├── tests/                       # Test suite
├── docs/                        # Developer documentation
├── package.json
├── tsconfig.json
└── README.md                    # This file

Documentation

• Model Context Protocol - Learn about MCP
• Runhuman Website - Product information
• API Documentation - REST API reference
• GitHub Repository - Full source code

Developer Docs

• How Agents Use MCP - Understanding MCP tool discovery
• Tool Response Best Practices - Writing effective tool responses
• API Authentication - Authentication details

Integration Examples

Claude Desktop Usage

Once installed, Claude can naturally use the tools:

User: "Test my app at https://staging.myapp.com and check if the login works"

Claude:

I'll create a QA job to test your login page.
[Calls create_job tool]
Job created! Waiting for a human tester...
[Calls wait tool - automatically polls until complete]
✅ Test complete! The login page works correctly. The tester was able to log in successfully with test credentials.

Programmatic Usage (Custom MCP Client)

import { createMcpServer } from '@runhuman/mcp/factory';

const server = createMcpServer({
  mode: 'direct',
  apiUrl: 'https://runhuman.com',
  apiKey: 'rh_xxxxx'
});

// Use with your MCP client
await server.connect(transport);

Troubleshooting

"Error: API key is required"

Make sure you've:

1. Created an account at runhuman.com
2. Generated an API key in the dashboard
3. Added the key to your config with the correct format (--api-key=rh_...)

"Invalid API key format"

API keys must start with:

• rh_ for production
• rh_ for testing

Get a valid key at runhuman.com/dashboard

Connection Issues

If you're having trouble connecting:

1. Check your API URL is correct (default: https://runhuman.com)
2. Verify your network connection
3. Test the API directly:

curl https://runhuman.com/api/jobs \
  -H "Authorization: Bearer rh_xxxxx" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","description":"test","outputSchema":{}}'

Claude Desktop Not Detecting Server

1. Verify config file location is correct for your OS
2. Check JSON syntax is valid (use a JSON validator)
3. Restart Claude Desktop completely
4. Check Claude's logs for errors

Contributing

This package is part of the Runhuman monorepo. Contributions welcome!

# Clone the full repo
git clone https://github.com/volter-ai/runhuman.git
cd runhuman

# Install dependencies
npm install

# Make changes to packages/mcp-server/

# Build and test
npm run build --workspace=@runhuman/mcp
npm run test --workspace=@runhuman/mcp

# Submit PR

Support

• Email: [email protected]
• Issues: GitHub Issues
• Website: runhuman.com

License

ISC License - See LICENSE for details

About Runhuman

Runhuman is an AI-orchestrated human QA testing service. Part of the Volter AI ecosystem.

Key Features:

• On-demand human testing (no hiring/managing)
• AI-powered orchestration and result extraction
• Pay-per-second pricing ($0.0085/sec)
• Custom output schemas for structured results
• Browser automation with video/screenshot recording
• GitHub Actions integration
• REST API and MCP server for any workflow

Built with ❤️ by the Volter AI team