hypercrawl-mcp

HyperCrawl as an MCP server. Gives any MCP client (Claude Code, Claude Desktop, …) an autonomous browser agent plus crawl / extract / tool-generation tools over the Model Context Protocol.

Install

Add it to your MCP client config — no clone, no build:

{
  "mcpServers": {
    "hypercrawl": {
      "command": "npx",
      "args": ["-y", "hypercrawl-mcp"],
      "env": { "ANTHROPIC_API_KEY": "sk-ant-…" }
    }
  }
}

Or with Claude Code:

claude mcp add hypercrawl -e ANTHROPIC_API_KEY=sk-ant-… -- npx -y hypercrawl-mcp

One-time browser install

The agent / crawl tools drive a real browser, so install Chromium once:

npx playwright install chromium

The server still starts without it (non-browser tools work) and prints this reminder on stderr.

Environment

| Var | Needed for | Notes | |-----|-----------|-------| | ANTHROPIC_API_KEY | hypercrawl_agent, hypercrawl_generate | The autonomous agent + tool generation call Claude. | | HYPERCRAWL_DATA_DIR | optional | Registry + saved sessions dir (default ~/.hypercrawl). |

Tools

| Tool | What it does | |------|--------------| | hypercrawl_agent | Autonomous goal-driven browse-and-act loop (hybrid DOM+vision, allowlist/budget guarded). | | hypercrawl_crawl | Fetch a URL rendered and return LLM-ready markdown (SPA actions + pagination supported). | | hypercrawl_extract | Run a generated site tool to pull structured data. | | hypercrawl_generate | Generate reusable extraction tools for a site from its HTML. | | hypercrawl_tools | List generated site tools in the registry. | | hypercrawl_cookie_sync | Pull cookies for a domain (requires the Collector bridge — use hypercrawl serve). | | hypercrawl_status | Report version + registry tool count. |

Run directly

npx -y hypercrawl-mcp        # stdio MCP server on stdin/stdout

Diagnostics go to stderr; stdout is the JSON-RPC channel.

Part of HyperCrawl. For the HTTP transport (Firebase-gated remote server) and the cookie Collector bridge, use the hypercrawl serve CLI.