crawly-mcp

MCP server for the Crawly web scraping & YouTube transcript API.

2 tools. Future-proof. New endpoints are auto-discovered — you never need to update this package.

Tools

| Tool | Description | | ---------------- | ------------------------------------------------------------------------------ | | crawly_docs | Fetch the live API schema. Returns all endpoints, parameters, and error types. | | crawly_request | Make a request to any Crawly endpoint. Pass endpoint path + params. |

Quick Start

1. Get an API Key

Sign up at crawly.bikal.co and create a key from the API Keys page.

2. Configure Your AI Tool

Claude Desktop — edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "crawly": {
      "command": "npx",
      "args": ["-y", "crawly-mcp", "--api-key", "cr_your_key_here"]
    }
  }
}

Cursor — edit .cursor/mcp.json in your project:

{
  "mcpServers": {
    "crawly": {
      "command": "npx",
      "args": ["-y", "crawly-mcp", "--api-key", "cr_your_key_here"]
    }
  }
}

Windsurf / Other MCP Clients — same pattern, adjust config path per tool.

3. Use It

Once configured, your AI can:

"Scrape https://example.com and summarize it"
→ AI calls crawly_docs, then crawly_request with /v1/scrape

"Get the transcript of this YouTube video"
→ AI calls crawly_request with /v1/transcript

Environment Variable

You can also set the API key via environment variable instead of --api-key:

export CRAWLY_API_KEY=cr_your_key_here

Available Endpoints

The MCP auto-discovers these from the live API:

• /v1/scrape — Extract clean markdown from any URL (1 credit)
• /v1/transcript — Get YouTube transcripts in text, segments, SRT, or VTT (1 credit)

More endpoints coming soon — they'll work automatically without updating.

Pricing

• $1 = 2,000 credits
• 1 credit per request
• Failed requests are refunded
• No subscriptions, credits never expire

Links

• Documentation
• API Keys
• Buy Credits

License

MIT