conscrapurs
v1.0.0
Published
Interactive CLI for monitoring procurement scrapers across multiple government portals
Maintainers
Readme
conscrapurs
Interactive CLI for monitoring procurement scrapers across multiple government portals.
Installation
# Install globally from npm
npm install -g conscrapurs
# Or use npx
npx conscrapursQuick Start
# Launch interactive mode
conscrapurs
# Or use direct commands
conscrapurs status # View all scrapers status
conscrapurs tail # Tail all logs liveCommands
| Command | Description |
|---------|-------------|
| conscrapurs | Launch interactive mode |
| conscrapurs status | Show status dashboard |
| conscrapurs tail [scraper] | Tail scraper logs |
| conscrapurs start <scraper> | Start a scraper |
| conscrapurs stop <scraper> | Stop a scraper |
| conscrapurs output <scraper> | View output summary |
Start Options
conscrapurs start eperolehan # Start with defaults
conscrapurs start zakupsk --details # Fetch tender details
conscrapurs start zakupsk --pages 10 # Limit to 10 pagesSupported Scrapers
| ID | Portal | Country | Source |
|----|--------|---------|--------|
| zakupsk | Zakupsk | 🇰🇿 Kazakhstan | zakup.sk.kz |
| eperolehan | ePerolehan | 🇲🇾 Malaysia | eperolehan.gov.my |
| smartgep | SmartGEP | 🇲🇾 Malaysia | smartgep.gov.my |
Configuration
Configure via environment variables:
export CONSCRAPURS_HOST=your-vps-ip
export CONSCRAPURS_PORT=22
export CONSCRAPURS_USER=rootDefault configuration:
- Host:
45.159.230.42 - Port:
1511 - User:
root - Base Dir:
/opt/scrapers-standalone/
Requirements
- Node.js 16+
- SSH access to VPS with key-based authentication
- Scrapers deployed at configured base directory
Interactive Mode
Running conscrapurs without arguments launches an interactive menu:
___
/ __\___ _ __ ___ ___ _ __ __ _ _ __ _ _ _ __ ___
/ / / _ \| '_ \/ __|/ __| '__/ _` | '_ \| | | | '__/ __|
/ /__| (_) | | | \__ \ (__| | | (_| | |_) | |_| | | \__ \
\____/\___/|_| |_|___/\___|_| \__,_| .__/ \__,_|_| |___/
|_|
? What would you like to do?
❯ 📊 View All Scrapers Status
📋 Tail All Logs (Live)
🔍 Monitor Specific Scraper
🚀 Start a Scraper
🛑 Stop a Scraper
📁 View Output Data
⚙️ Settings
❌ ExitLicense
MIT © ConsurvBL
