@cd39390/mcp-web-crawler
v1.0.6
Published
An MCP server plugin to crawl all hyperlinks from a website for AI learning purposes.
Downloads
80
Maintainers
Readme
# mcp-web-crawler
An MCP (Model Context Protocol) server plugin that enables AI models to crawl and extract all hyperlinks from a given website.
## Purpose
This tool is designed for AI learning. For example, you can ask an AI to use this tool to crawl the entire Python documentation site (`https://docs.python.org/`) to gain comprehensive knowledge about its content and structure.
## Installation
This package is intended to be used as an MCP server. You can install it globally or within a specific project.
```bash
npm install -g mcp-web-crawler