CollectionRunner CLI

Enterprise-grade CLI tool to run Postman collections with Newman, collect detailed execution metrics, generate beautiful HTML reports with interactive charts, and send notifications to Telegram, Slack, and Discord.

🔒 Enterprise-Grade Security | ✅ Production Ready | 📊 Advanced Analytics | 🎨 Premium UI Reports | 📈 Execution History

Installation

Global Installation (Recommended)

npm install -g collection-runner-cli

After installation, the collection-runner command will be available globally.

Local Installation

npm install collection-runner-cli

From Source

git clone https://github.com/orewacoder/collection-runner-cli.git
cd collection-runner-cli
npm install
npm link

Quick Start

# Run a Postman collection
collection-runner --collection ./examples/sample_collection.json

# With environment variables
collection-runner --collection ./api.json --env ./prod.json

# Multiple iterations
collection-runner --collection ./api.json --iterations 3

# Custom timeout
collection-runner --collection ./api.json --timeout 60000

# Filter by status or method
collection-runner --collection ./api.json --filter-status failed
collection-runner --collection ./api.json --filter-method POST

Features

🚀 Core Features

• Postman Collection Execution - Run collections via Newman API
• Multiple Iterations - Support for load testing scenarios
• Environment Variables - Use Postman environment files
• Configurable Timeout - Set custom timeout for requests
• Request Filtering - Filter by status code or HTTP method

📊 Advanced Metrics & Analytics

• Per-Request Metrics - Duration, status codes, assertions
• Performance Metrics - Average, median, P95, P99 percentiles
• HTTP Method Statistics - Count, success rate, average duration per method
• Status Code Statistics - Distribution and analysis
• Assertion Tracking - Detailed pass/fail information
• Execution Timeline - Start/end times, total duration

🎨 Premium HTML Reports

Modern Dark Theme (Vercel/Linear Premium UI Style)

• Dark gradient background with glassmorphism effects
• Professional typography (Inter + JetBrains Mono fonts)
• Responsive design for desktop and mobile
• Interactive Chart.js visualizations

Report Sections:

• Overview - KPI cards, status badges, execution timeline
• Test Cases - Detailed request information with assertions
• Timeline - Chronological execution view
• Categories - Grouped by status
• History - Execution history with trend analysis

Interactive Charts:

• Request status distribution (doughnut chart)
• Assertion results (doughnut chart)
• Pass rate trend (line chart)
• Failed requests trend (line chart)
• Duration trend (line chart)

History & Comparison:

• Automatic execution history tracking
• Comparison with previous runs
• Trend visualization over time
• Historical data table

📁 Export Formats

• HTML Reports - Standalone interactive reports
• JSON Export - Machine-readable results with all metrics
• CSV Export - Spreadsheet-compatible format for analysis

💬 Notification Integrations

• Telegram - Send reports via Telegram bot
• Slack - Post results to Slack channels
• Discord - Send notifications to Discord webhooks

🔐 Security & Configuration

• Secure Credential Storage - Tokens stored in ~/.collection-runner/config.json
• Token Masking - Sensitive data masked in logs
• Path Validation - Prevents path traversal attacks
• Environment Variable Support - CI/CD friendly
• Input Validation - Comprehensive parameter validation

📈 Execution History

• Automatic Tracking - Every run is saved to history
• Trend Analysis - Visualize performance over time
• Comparison Reports - Compare current vs previous runs
• Historical Data - Access to past execution results

CLI Options

| Option | Type | Required | Description | |--------|------|----------|-------------| | --collection <path> | string | ✅ Yes | Path to Postman collection JSON file | | --env <path> | string | ❌ No | Path to Postman environment JSON file | | --iterations <number> | number | ❌ No | Number of iterations (default: 1) | | --timeout <ms> | number | ❌ No | Request timeout in milliseconds (default: 30000) | | --filter-status <status> | string | ❌ No | Filter requests by status: passed, failed, or all | | --filter-method <method> | string | ❌ No | Filter requests by HTTP method: GET, POST, PUT, DELETE, etc. | | --telegram-token <token> | string | ❌ No | Telegram bot token (or use config) | | --telegram-chat-id <id> | string | ❌ No | Telegram chat ID (or use config) | | --slack-webhook <url> | string | ❌ No | Slack webhook URL (or use config) | | --discord-webhook <url> | string | ❌ No | Discord webhook URL (or use config) | | --report-dir <path> | string | ❌ No | Report output directory (default: ./reports) | | --report-name <name> | string | ❌ No | Report file base name (default: report-<timestamp>) | | --verbose | flag | ❌ No | Enable verbose logging | | --help | flag | ❌ No | Show help message |

Configuration Management

Save Credentials Securely

# Telegram
collection-runner config set telegram-token YOUR_BOT_TOKEN
collection-runner config set telegram-chat-id YOUR_CHAT_ID

# Slack
collection-runner config set slack-webhook YOUR_WEBHOOK_URL

# Discord
collection-runner config set discord-webhook YOUR_WEBHOOK_URL

# Report settings
collection-runner config set report-dir ./my_reports

View Configuration

# Show all settings (tokens are masked)
collection-runner config show

# Get specific value
collection-runner config get telegram-token

# Clear specific value
collection-runner config clear telegram-token

# Reset all to defaults
collection-runner config reset

Storage Location:

• Linux/Mac: ~/.collection-runner/config.json
• Windows: %USERPROFILE%\.collection-runner\config.json

Notification Setup

Telegram

1. Create a bot via @BotFather
2. Get your bot token
3. Get your chat ID (send message to bot, then visit https://api.telegram.org/bot<TOKEN>/getUpdates)
4. Save credentials:

   collection-runner config set telegram-token YOUR_TOKEN
   collection-runner config set telegram-chat-id YOUR_CHAT_ID

Slack

1. Create a Slack app and webhook
2. Copy webhook URL
3. Save:

   collection-runner config set slack-webhook YOUR_WEBHOOK_URL

Discord

1. Create a Discord webhook in channel settings
2. Copy webhook URL
3. Save:

   collection-runner config set discord-webhook YOUR_WEBHOOK_URL

Report Formats

HTML Report

Standalone interactive report with:

• Dark premium UI design
• Interactive charts and visualizations
• Execution history and trends
• Comparison with previous runs
• Responsive mobile-friendly layout

Location: ./reports/report-<timestamp>.html

JSON Export

Machine-readable format with:

• Complete execution summary
• All request details
• Performance metrics (P95, P99, median)
• HTTP method statistics
• Status code statistics

Location: ./reports/report-<timestamp>.json

CSV Export

Spreadsheet-compatible format:

• Request-level data
• Summary metrics
• Performance statistics
• Easy import to Excel/Google Sheets

Location: ./reports/report-<timestamp>.csv

Execution History

CollectionRunner automatically tracks execution history:

• Storage: ~/.collection-runner/history/ (Linux/Mac) or %USERPROFILE%\.collection-runner\history\ (Windows)
• Automatic Comparison - Each run compares with previous execution
• Trend Visualization - Charts show performance over time
• Historical Reports - Access past execution data in HTML reports

Examples

Basic Usage

# Run collection
collection-runner --collection ./api.json

# With environment
collection-runner --collection ./api.json --env ./prod.json

# Multiple iterations
collection-runner --collection ./api.json --iterations 5

Advanced Usage

# With timeout and filtering
collection-runner --collection ./api.json \
  --timeout 60000 \
  --filter-status failed \
  --filter-method POST

# Custom report location
collection-runner --collection ./api.json \
  --report-dir ./test-results \
  --report-name api-test-run

# With notifications (uses saved config)
collection-runner --collection ./api.json

CI/CD Integration

# Using environment variables
export COLLECTION_RUNNER_TELEGRAM_TOKEN="your_token"
export COLLECTION_RUNNER_TELEGRAM_CHAT_ID="your_chat_id"
collection-runner --collection ./api.json

Project Structure

collection-runner/
├── bin/
│   └── collection-runner.js      # CLI entry point
├── src/
│   ├── index.js                   # Main orchestrator
│   ├── newmanRunner.js            # Newman API wrapper
│   ├── reportGeneratorV2.js       # HTML report generator
│   ├── jsonExporter.js             # JSON export
│   ├── csvExporter.js             # CSV export
│   ├── historyManager.js          # Execution history
│   ├── telegramNotifier.js        # Telegram integration
│   ├── slackNotifier.js           # Slack integration
│   ├── discordNotifier.js         # Discord integration
│   ├── configManager.js           # Configuration management
│   └── utils/
│       ├── argParser.js           # CLI argument parsing
│       ├── logger.js              # Logging utilities
│       ├── fileUtils.js           # File operations
│       ├── statistics.js          # Metrics calculations
│       └── chartGenerator.js      # Chart utilities
├── examples/
│   ├── sample_collection.json     # Example collection
│   └── sample_env.json           # Example environment
└── package.json

Security

✅ Production Ready

• 0 Critical Vulnerabilities
• 0 High Vulnerabilities
• Secure credential storage
• Path traversal prevention
• Token masking in logs
• Input validation

See SECURITY_REPORT.md for detailed security audit.

Dependencies

• newman ^6.0.0 - Postman collection runner
• axios ^1.7.2 - HTTP client
• form-data ^4.0.0 - Form data handling
• canvas ^3.2.0 - Image processing
• sharp ^0.34.5 - Image optimization

Requirements

• Node.js >= 14.0.0
• npm >= 6.0.0

Browser Support

HTML reports work in all modern browsers:

• Chrome/Edge 90+
• Firefox 88+
• Safari 14+

Performance

• Handles collections with 100+ requests
• Report generation < 1 second
• Minimal memory footprint
• CI/CD pipeline ready

Troubleshooting

Collection not found

# Use absolute or relative path
collection-runner --collection /full/path/to/collection.json

Notifications not working

# Verify credentials with verbose mode
collection-runner --collection ./api.json --verbose

Report not generated

# Check directory permissions
collection-runner --collection ./api.json --report-dir ./reports --verbose

License

MIT

Author

Farrukh Kuchkarov

• Email: [email protected]
• GitHub: @orewacoder

Contributing

Contributions welcome! Feel free to submit issues or pull requests.

Support

For issues, questions, or feature requests, please open an issue on GitHub.

Made with ❤️ for API testers and QA engineers