erica-crawler-mcp

MCP server for crawling raw HTML from URLs with hash-based authentication.

Installation

npx -y erica-crawler-mcp

Claude Desktop Configuration

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS):

{
  "mcpServers": {
    "erica-crawler": {
      "command": "npx",
      "args": ["-y", "erica-crawler-mcp"],
      "env": {
        "CRAWLER_AUTH_HASH": "your-secret-hash-here",
        "CRAWLER_USER_AGENT": "Mozilla/5.0 (compatible; MyCrawler/1.0)"
      }
    }
  }
}

Authentication

The CRAWLER_AUTH_HASH environment variable is required. Only users with the correct hash can use the crawler. Generate a random hash:

openssl rand -hex 32

Tools Available

| Tool | Description | |------|-------------| | crawler_fetch | Fetch raw HTML content from a URL | | crawler_fetch_text | Fetch URL and extract text (strips HTML tags) |

Usage Examples

Once configured, ask Claude:

• "Crawl https://example.com and show me the HTML"
• "Fetch the text content from https://example.com/page"