can you print the readme as an artifact ???

Web Scrape & Search CLI

A command-line interface tool for web scraping and searching using DuckDuckGo.

Installation

npm install -g web-scrape-search-cli

Features

• Web scraping with multiple modes (static, dynamic, recursive)

• Article content extraction

• DuckDuckGo web search integration

• Customizable output options

• Rate limiting and robots.txt compliance

Usage

Web Scraping

websearch scrape <url> [options]

Options:

  --recursive           Recursively scrape linked pages

  --dynamic            Use dynamic scraping (for JavaScript-rendered content)

  --article            Extract main article content

  --max-pages <num>    Maximum number of pages to scrape

  --delay <ms>         Delay between requests in milliseconds (default: 1000)

  --output <dir>       Custom output directory

  --raw               Save raw HTML without filtering

Web Search

websearch search <query> [options]

Options:

  --max <num>          Maximum number of results (default: 10)

  --save <filepath>    Save results to JSON file

Examples

# Scrape a website recursively

websearch scrape https://example.com --recursive --max-pages 5

# Extract article content

websearch scrape https://example.com/blog --article

# Search with DuckDuckGo

websearch search "nodejs tutorials" --max 5 --save ./results.json

Requirements

• Node.js >= 14.0.0

• npm >= 6.0.0

License

MIT

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.