⚡ Tempest

High-Performance LLM & Streaming API Load Tester with Traffic Replay and CI/CD Performance Regression Checks

╔══════════════════════════════════════════════════════════════════════╗
║  ████████╗███████╗███╗   ███╗██████╗ ███████╗███████╗████████╗       ║
║  ╚══██╔══╝██╔════╝████╗ ████║██╔══██╗██╔════╝██╔════╝╚══██╔══╝       ║
║     ██║   █████╗  ██╔████╔██║██████╔╝█████╗  ███████╗   ██║          ║
║     ██║   ██╔══╝  ██║╚██╔╝██║██╔═══╝ ██╔══╝  ╚════██║   ██║          ║
║     ██║   ███████╗██║ ╚═╝ ██║██║     ███████╗███████║   ██║          ║
║     ╚═╝   ╚══════╝╚═╝     ╚═╝╚═╝     ╚══════╝╚══════╝   ╚═╝          ║
║      High-Performance LLM & Streaming API Load & Regression Tester   ║
╚══════════════════════════════════════════════════════════════════════╝

Traditional load testing tools (like JMeter or ApacheBench) only measure static response duration. Tempest is designed specifically for modern AI, high-concurrency (1,000+ users), and streaming architectures:

• ⚡ Interactive Terminal Wizard: Just run npx tempest-load to launch a step-by-step interactive benchmark.
• 🔍 Zero-Config Auto-Detect (tempest start): Automatically scans for active Next.js (port 3000), Ollama (port 11434), FastAPI (port 8000), or Express (port 8080) servers and starts load testing instantly.
• 🚀 1,000+ Concurrent Virtual Users: High-throughput async connection pool capable of firing heavy workloads from a single machine.
• 📊 Real-time SSE Streaming Metrics: Measures TTFT (Time to First Token), ITL (Inter-Token Latency), and TPS (Tokens Per Second) with microsecond precision.
• 🔄 HAR & Traffic Replay: Ingests recorded browser sessions (.har files), strips static assets, and replays authentic API sequences across thousands of simulated users.
• 🛡️ CI/CD Performance Regression: Compares test runs against a baseline JSON, fails PRs if latency degrades (e.g. p99_ttft > +15%), and renders rich GitHub PR comment diff tables.

⚠️ Responsible Use & Disclaimer

IMPORTANT: Tempest is intended strictly for authorized benchmarking, development profiling, and performance testing of systems you own or have explicit permission to test.

1. Authorized Testing Only: Do not perform load testing against third-party endpoints or infrastructure without authorization. Unauthorized high-volume requests may violate Terms of Service and local laws regarding Denial-of-Service (DoS).
2. Third-Party API & Cloud Costs: When benchmarking paid cloud endpoints (e.g. OpenAI, Anthropic, AWS, Vercel, Supabase), high-throughput load tests will consume tokens, bandwidth, and compute units. You are solely responsible for any costs incurred.
3. Limitation of Liability: The maintainers and contributors of this project assume no liability and are not responsible for any misuse, server downtime, data loss, or financial expenses resulting from the use of this software.

🚀 Instant Usage (Zero Install)

1. Interactive Wizard Mode (Easiest)

Run in any repository terminal:

npx tempest-load

Follow the interactive arrow-key prompts to pick your target and choose your load level (Light, Medium, Heavy, 1,000 Users, or Custom).

2. Auto-Detect Mode (tempest start)

Scans your open local ports and fires immediately with zero questions:

npx tempest-load start

3. Custom CLI Commands

# Benchmark local Ollama / AI endpoint with 100 concurrent streams
npx tempest-load bench --target http://localhost:11434/v1/chat/completions --model llama3 -c 100 -n 500

# High-concurrency stress test with 1,000 concurrent users
npx tempest-load bench --target http://localhost:3000/api/search -c 1000 -n 5000 --no-stream

# Replay a recorded browser .har session across 50 users
npx tempest-load replay --har session.har -c 50

# Save and compare CI/CD baseline performance
npx tempest-load bench --target http://localhost:3000/api/chat --save-baseline baseline.json
npx tempest-load bench --target http://localhost:3000/api/chat --baseline baseline.json --fail-on "p99_ttft > +15%"

🛠️ CLI Reference

| Command | Description | | :--- | :--- | | npx tempest-load | Interactive arrow-key menu wizard | | npx tempest-load start | Zero-config: auto-detects running local servers and tests immediately | | npx tempest-load init | Drops a tempest.config.json for one-command execution | | npx tempest-load bench | Custom synthetic load test with custom flags | | npx tempest-load replay | Replay captured .har traffic under concurrent load | | npx tempest-load diff | Compare two benchmark JSON files and evaluate SLA rules |

🧪 Testing

pnpm test

📄 License

MIT © 2026 Abhinav Jain