CommentScraper CLI

Scrape structured comments and reviews from supported discussion platforms for research, customer-voice analysis, and agent workflows.

Install

npm install -g commentscraper
commentscraper login

The CLI requires a paid subscription. Pro supports Reddit. All Access supports every CLI platform.

Commands

commentscraper research "back pain remedies" --quiet
commentscraper search "crm software frustrations" --quiet
commentscraper scrape "https://reddit.com/r/programming/comments/abc123/post/" --format json --quiet
commentscraper mcp                # run as stdio MCP server (see below)
commentscraper whoami
commentscraper platforms

Use --quiet when piping JSON to another process so progress logs stay out of stdout.

Supported Platforms

| Platform | Example | Plan | | --- | --- | --- | | Reddit | https://reddit.com/r/startups/comments/abc123/post/ | Pro+ | | Reddit profile | https://reddit.com/user/spez | Pro+ | | YouTube | https://youtube.com/watch?v=jNQXAC9IVRw | All Access | | Hacker News | https://news.ycombinator.com/item?id=12345 | All Access | | Amazon | https://amazon.com/product-reviews/B08N5WRWNW | All Access | | Steam | https://store.steampowered.com/app/730/ | All Access | | Steam recommendation | https://steamcommunity.com/profiles/76561198000000000/recommended/730/ | All Access | | Trustpilot | https://trustpilot.com/review/www.amazon.com | All Access | | Product Hunt | https://producthunt.com/posts/some-product | All Access | | Product Hunt product | https://producthunt.com/products/some-product | All Access | | Etsy | https://etsy.com/listing/560282270/personalized-custom-mug | All Access | | Quora | https://quora.com/How-do-you-define-product-market-fit | All Access | | Google Maps | https://google.com/maps/place/Blue+Bottle+Coffee/ | All Access | | Notion | https://example.notion.site/page-id | All Access | | Shopify | https://mystore.myshopify.com/products/example-product | All Access | | phpBB forum | https://example.com/phpBB3/viewforum.php?f=20 | All Access | | phpBB thread | https://example.com/phpBB3/viewtopic.php?t=12345 | All Access |

YouTube, Amazon, Trustpilot, Etsy, Quora, Google Maps, and Shopify use a browser-backed Chromium scraper. For pages that need an authenticated browser session, provide one of these optional environment variables before running scrape or mcp:

COMMENTSCRAPER_<PLATFORM>_STORAGE=/path/to/playwright-state.json
COMMENTSCRAPER_<PLATFORM>_USER_DATA_DIR=/path/to/chrome-profile

Output

scrape returns JSON by default:

{
  "scrapedAt": "2026-05-10T00:00:00.000Z",
  "source": "reddit",
  "url": "https://reddit.com/r/startups/comments/abc123/post/",
  "count": 342,
  "post": { "title": "Thread title", "body": "Post body", "url": "..." },
  "comments": [
    {
      "text": "Comment body",
      "author": "username",
      "timestamp": "2026-05-10T00:00:00.000Z",
      "permalink": "https://reddit.com/...",
      "score": 42,
      "depth": 0
    }
  ]
}

CSV and text output are also available:

commentscraper scrape "<url>" --format csv --output comments.csv --quiet
commentscraper scrape "<url>" --format text --quiet
commentscraper scrape "https://example.com/phpBB3/viewforum.php?f=20" --max-threads 20 --quiet

Claude Desktop (MCP)

CommentScraper ships a stdio MCP server so Claude Desktop, Cursor, and other MCP clients can call it directly. Requires an All Access subscription.

One-time setup:

npm install -g commentscraper
commentscraper login                 # opens browser, stores token

Then add to Claude Desktop config at ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "commentscraper": {
      "command": "commentscraper",
      "args": ["mcp"]
    }
  }
}

Restart Claude Desktop. The MCP server reads the token saved by commentscraper login automatically — nothing to copy or paste. If your MCP client runs in a sandbox that can't see ~/.commentscraper/, pass the token via env instead:

{
  "mcpServers": {
    "commentscraper": {
      "command": "npx",
      "args": ["-y", "commentscraper@latest", "mcp"],
      "env": { "COMMENTSCRAPER_CLI_TOKEN": "your-cli-token" }
    }
  }
}

Four tools become available in Claude:

| Tool | Description | | --- | --- | | scrape_url | Scrape comments from any supported URL | | search_reddit | Find Reddit threads matching a keyword | | research_topic | Search + scrape every matching Reddit thread | | whoami | Inspect auth + plan |

Automation

For CI or smoke testing, provide a token through the environment:

COMMENTSCRAPER_CLI_TOKEN=<token> commentscraper scrape "<url>" --format json --quiet
COMMENTSCRAPER_CLI_TOKEN=<token> commentscraper whoami

Without COMMENTSCRAPER_CLI_TOKEN, the CLI uses the token saved by commentscraper login. The CLI does not auto-load a repository .env file; export the variable in your shell, prefix the command, or use commentscraper login.