CollectionRunner CLI

CLI tool to run Postman collections with Newman, generate beautiful HTML reports with interactive charts, and send notifications to Telegram, Slack, and Discord.

Installation

npm install -g collection-runner-cli

Quick Start — First Run

On first launch the setup wizard starts automatically and guides you through configuration in 4 steps:

collection-runner

Or run it manually at any time:

collection-runner setup

The wizard will ask you to configure:

1. Collections directory — where your Postman collection .json files live (scanned automatically on every run)
2. Environments directory — folder with Postman environment files (optional)
3. Reports directory — where HTML/JSON/CSV reports are saved
4. Telegram bot token + group/chat ID — for notifications (optional)

After setup, just run:

collection-runner run

If you have one collection in the configured directory it runs immediately. If there are multiple, you get a numbered list to choose from.

Running a Specific Collection

You can always bypass the wizard and point directly to a file:

# Basic run
collection-runner --collection ./api.json

# With environment file
collection-runner --collection ./api.json --env ./prod.json

# Multiple iterations
collection-runner --collection ./api.json --iterations 5

# Custom timeout (ms)
collection-runner --collection ./api.json --timeout 60000

# Custom report location
collection-runner --collection ./api.json --report-dir ./results --report-name my-run

CLI Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | --collection <path> | string | (from config) | Path to Postman collection JSON | | --env <path> | string | (from config) | Path to Postman environment JSON | | --iterations <n> | number | 1 | Number of iterations (max 10000) | | --timeout <ms> | number | 30000 | Request timeout (1000–300000 ms) | | --report-dir <path> | string | ./reports | Report output directory | | --report-name <name> | string | report-<timestamp> | Report file base name | | --telegram-token <t> | string | (from config) | Telegram bot token | | --telegram-chat-id <id> | string | (from config) | Telegram group/chat ID | | --slack-webhook-url <url> | string | (from config) | Slack webhook URL | | --discord-webhook-url <url> | string | (from config) | Discord webhook URL | | --filter-status <s> | string | — | Filter by passed or failed | | --filter-method <m> | string | — | Filter by HTTP method (GET, POST, …) | | --verbose | flag | — | Enable debug logging | | --help | flag | — | Show help |

Configuration

Settings are saved to ~/.collection-runner/config.json (Windows: %USERPROFILE%\.collection-runner\config.json).

Available Config Keys

| Key | Description | |-----|-------------| | collections-dir | Directory scanned for collection files on each run | | env-dir | Directory scanned for environment files | | report-dir | Default report output directory | | telegram-token | Telegram bot token | | telegram-chat-id | Telegram group or chat ID (numeric) | | slack-webhook-url | Slack incoming webhook URL | | discord-webhook-url | Discord webhook URL |

Config Commands

# Set a value
collection-runner config set telegram-token YOUR_TOKEN
collection-runner config set telegram-chat-id -100XXXXXXXXX
collection-runner config set collections-dir /path/to/collections
collection-runner config set env-dir /path/to/envs
collection-runner config set report-dir ./reports

# View all settings (tokens are masked)
collection-runner config show

# Get a single value
collection-runner config get telegram-token

# Clear a value
collection-runner config clear telegram-token

# Reset everything to defaults
collection-runner config reset

Environment Variables (CI/CD)

export COLLECTION_RUNNER_TELEGRAM_TOKEN="your_token"
export COLLECTION_RUNNER_TELEGRAM_CHAT_ID="your_chat_id"
export COLLECTION_RUNNER_SLACK_WEBHOOK_URL="https://hooks.slack.com/..."
export COLLECTION_RUNNER_DISCORD_WEBHOOK_URL="https://discord.com/api/webhooks/..."
export COLLECTION_RUNNER_REPORT_DIR="./reports"

Telegram Setup

1. Open Telegram → message @BotFather → /newbot
2. Copy the bot token
3. Add your bot to the group/channel
4. Send any message in the group
5. Open https://api.telegram.org/bot<TOKEN>/getUpdates → find "chat":{"id": -100XXXXXXXXX}
6. Save both:

   collection-runner config set telegram-token YOUR_TOKEN
   collection-runner config set telegram-chat-id -100XXXXXXXXX

Each test run sends a summary message with an ASCII chart and attaches the full HTML report as a document.

Reports

Three files are generated per run:

| File | Description | |------|-------------| | report-<timestamp>.html | Interactive dark-theme report (Charts, Timeline, History) | | report-<timestamp>.json | Machine-readable full results with performance metrics | | report-<timestamp>.csv | Spreadsheet-compatible per-request data |

Report sections:

• Overview — KPI cards, pass rate, duration, assertion summary
• Test Cases — each request with status, timing, and assertion details
• Timeline — chronological execution view
• Categories — grouped by passed/failed
• History — trend charts comparing with previous runs

Features

• Run Postman collections via Newman (supports pm.test(), pm.expect(), pre-request scripts)
• Interactive setup wizard on first run
• Smart collection picker — scans configured directory, no path required
• Per-request metrics: duration, status codes, assertions
• Performance percentiles: average, median, P95, P99
• Execution history with trend visualization
• Notifications: Telegram (with HTML report attachment), Slack, Discord
• Export: HTML, JSON, CSV
• CI/CD friendly — environment variables, exit codes (0 = pass, 1 = fail)
• Zero native binary dependencies (canvas, sharp, axios removed)

Project Structure

collection-runner-cli/
├── bin/
│   └── collection-runner.js       # CLI entry point
├── src/
│   ├── index.js                   # Main orchestrator + collection picker
│   ├── setup.js                   # Interactive first-run wizard
│   ├── newmanRunner.js            # Newman wrapper
│   ├── configManager.js           # Config file management
│   ├── reportGeneratorV2.js       # HTML report generator
│   ├── telegramNotifier.js        # Telegram integration
│   ├── slackNotifier.js           # Slack integration
│   ├── discordNotifier.js         # Discord integration
│   ├── historyManager.js          # Execution history
│   ├── jsonExporter.js            # JSON export
│   ├── csvExporter.js             # CSV export
│   └── utils/
│       ├── http.js                # Built-in HTTPS wrapper (no axios)
│       ├── argParser.js           # CLI argument parsing
│       ├── chartGenerator.js      # ASCII chart for Telegram
│       ├── fileUtils.js           # File operations
│       ├── logger.js              # Colored console logger
│       └── statistics.js          # Metrics calculations
└── examples/
    ├── sample_collection.json
    └── sample_env.json

Dependencies

| Package | Purpose | |---------|---------| | newman 6.2.2 | Postman collection runner (required for pm.test/pm.expect scripts) |

All HTTP requests (Telegram, Slack, Discord APIs) use Node.js built-in https module — no axios, no form-data, no native binaries.

Security note: Newman 6.x contains known vulnerabilities in its transitive dependencies (handlebars, lodash, node-forge). These are only exploitable via malicious collection files or untrusted server responses. Only run collections you trust. Track upstream: newman issues.

Requirements

• Node.js >= 16.0.0
• npm >= 7.0.0

Troubleshooting

--collection file not found

# Use an absolute path or configure collections-dir
collection-runner config set collections-dir /full/path/to/collections
collection-runner run

Telegram notifications not working

# Check saved config
collection-runner config show

# Test with verbose output
collection-runner --collection ./api.json --verbose

Report directory permission error

collection-runner config set report-dir ./reports
# or pass directly:
collection-runner --collection ./api.json --report-dir /writable/path

License

MIT

Author

Farrukh Kuchkarov

• Email: [email protected]
• GitHub: @orewacoder

For bugs and feature requests → GitHub Issues