Proxy Test Suite — Multi-Provider Edition

Automated performance and reliability test suite for proxy services. Supports multiple providers (SimplyNode, BrightData, etc.), cross-provider comparison, and detailed reporting.

Evaluates HTTP, HTTPS, and SOCKS5 proxies across connectivity, latency, geo-targeting accuracy, IP rotation, sticky sessions, success rate, and concurrent load.

Quick Start

# 1. Install dependencies
npm install

# 2a. Single-provider mode (legacy .env)
cp .env.example .env
# Edit .env with your proxy username and password

# 2b. Multi-provider mode (YAML config)
cp proxy-config.example.yaml proxy-config.yaml
# Edit proxy-config.yaml with your provider credentials

# 3. Run all tests (single provider)
npx tsx src/index.ts run --all

# 4. Cross-provider comparison
npx tsx src/index.ts compare --config proxy-config.yaml

Test Suites

| Suite | Key | Description | Pass Criteria | |-------|-----|-------------|---------------| | Connectivity | connectivity | HTTP, HTTPS, SOCKS5 basic connection | Status 200 from ipify.org | | Latency | latency | Response time benchmarks (residential + mobile, US + ID) | Requests complete successfully | | Geo-Targeting | geo_targeting | IP location matches requested country/city | Country code match, city match if specified | | IP Rotation | ip_rotation | Consecutive requests get different IPs | >70% unique IPs across N requests | | Sticky Sessions | sticky_sessions | Same port maintains same IP | ≤2 unique IPs across N sequential requests | | Success Rate | success_rate | 100+ requests per target, measure ratio | >95% success rate | | Comparison | comparison | Side-by-side residential vs mobile benchmarks | Both types produce data | | Concurrent | concurrent | 10/50/100 parallel requests | >90% success rate under load |

CLI Usage

run — Single-Provider Test Run

# Run everything
npx tsx src/index.ts run --all

# Run specific suites
npx tsx src/index.ts run --suite connectivity geo_targeting

# Custom output directory
npx tsx src/index.ts run --all --output ./my-reports

# Dry run (show what would run)
npx tsx src/index.ts run --all --dry-run

# List available suites
npx tsx src/index.ts list

# Show help
npx tsx src/index.ts --help

compare — Cross-Provider Comparison

# Compare all providers from config
npx tsx src/index.ts compare --config proxy-config.yaml

# Compare specific providers
npx tsx src/index.ts compare --config proxy-config.yaml --providers simplynode brightdata

# Compare specific accounts
npx tsx src/index.ts compare --config proxy-config.yaml --accounts residential-us mobile-id

# Run only specific suites during comparison
npx tsx src/index.ts compare --config proxy-config.yaml --suite connectivity latency

# Dry run (show comparison jobs without executing)
npx tsx src/index.ts compare --config proxy-config.yaml --dry-run

# Custom output directory
npx tsx src/index.ts compare --config proxy-config.yaml --output ./comparison-reports

Comparison generates:

• comparison-{timestamp}.json — Structured results with rankings
• comparison-{timestamp}.md — Markdown table with verdict

providers — Provider Management

# List registered provider adapters
npx tsx src/index.ts providers list

Configuration

Single-Provider Mode (.env file)

| Variable | Required | Default | Description | |----------|----------|---------|-------------| | SIMPLYNODE_USERNAME | Yes | — | Your SimplyNode dashboard username | | SIMPLYNODE_PASSWORD | Yes | — | Your SimplyNode dashboard password | | DEFAULT_PORT | No | 9100 | Proxy port (valid range: 9000-19999) | | REQUESTS_PER_TEST | No | 50 | Number of requests per test case | | CONCURRENCY | No | 10 | Max concurrent requests | | TIMEOUT_MS | No | 30000 | Request timeout in milliseconds |

Multi-Provider Mode (proxy-config.yaml)

schemaVersion: 1

providers:
  - name: simplynode
    adapter: simplynode
    accounts:
      - name: residential-us
        credentials:
          username: ${SIMPLYNODE_USER}
          password: ${SIMPLYNODE_PASS}

  - name: brightdata
    adapter: generic
    accounts:
      - name: datacenter-us
        credentials:
          host: brd.superproxy.io
          port: "22225"
          username: ${BRD_USER}
          password: ${BRD_PASS}

defaults:
  protocol: http
  timeout: 30000
  retries: 3
  concurrency: 10

reporting:
  outputDir: ./reports
  formats:
    - markdown
    - json

Environment variables in ${...} are resolved from .env or system env.

Project Structure

src/
├── index.ts               # CLI entry point (run, compare, providers, list)
├── types.ts               # Shared TypeScript types
├── comparison.ts          # Cross-provider comparison runner
├── config.ts              # Proxy endpoint config, auth encoding, .env loader
├── client.ts              # HTTP/HTTPS/SOCKS5 proxy client with retry
├── stats.ts               # Timing metrics, percentile calculations
├── report.ts              # JSON + Markdown report generator
├── providers/
│   ├── types.ts           # Provider adapter interface, SuiteContext
│   ├── registry.ts        # Adapter registration and resolution
│   ├── context.ts         # SuiteContext factory, CapabilityError
│   └── adapters/
│       ├── simplynode.ts  # SimplyNode adapter
│       └── generic.ts     # Generic HTTP/SOCKS5 adapter
├── config/
│   ├── schema.ts          # YAML config validation schema
│   └── load-config.ts     # Config loading + env var resolution
└── suites/
    ├── index.ts              # Barrel export
    ├── suite-helpers.ts      # Bridge helpers (ctx + legacy paths)
    ├── connectivity.ts       # Suite 1: Protocol connectivity
    ├── latency.ts            # Suite 2: Response time benchmarks
    ├── geo-targeting.ts      # Suite 3: Location accuracy
    ├── ip-rotation.ts        # Suite 4: IP rotation uniqueness
    ├── sticky-sessions.ts    # Suite 5: Session IP stability
    ├── success-rate.ts       # Suite 6: Bulk success rate
    ├── comparison.ts         # Suite 7: Residential vs Mobile
    └── concurrent.ts         # Suite 8: Parallel load testing

How Targeting Works (SimplyNode)

SimplyNode encodes proxy targeting in the password field:

Format: {connectionType};{country};{region};{city};{isp}

Examples:
  wifi;us;;;            → US residential
  cell;us;;;            → US mobile (4G/5G)
  wifi;id;jakarta;;     → Indonesia, Jakarta residential
  wifi;gb;england;london;virgin  → UK London, Virgin ISP

Report Output

Reports are saved to ./reports/ (or custom --output dir) with timestamp:

• report-{timestamp}.json — Full structured results
• report-{timestamp}.md — Human-readable Markdown with metrics table and recommendations
• comparison-{timestamp}.json — Cross-provider comparison with rankings (compare command)
• comparison-{timestamp}.md — Markdown comparison table with verdict (compare command)

Interpreting Results

• Pass Rate ≥ 95%: Production-ready. Proxy meets SLA requirements.
• Pass Rate 80-95%: Usable with caveats. Check specific failure suites for patterns.
• Pass Rate < 80%: Not recommended for production. Investigate error types.

Key metrics to watch:

• P95 latency: Should be < 2000ms for residential, < 3000ms for mobile
• Geo accuracy: Should be 100% for country-level targeting
• Rotation uniqueness: >70% means healthy IP pool diversity
• Concurrent success rate: >90% under 50+ concurrent requests

Requirements

• Node.js ≥ 20.18.1
• Active proxy account(s) with traffic balance

License

MIT