Artillery Performance Testing MCP Server

A Model Context Protocol (MCP) server for generating, running, and analyzing performance tests using Artillery.io.

Disclaimer!

⚠️ This software is provided "as is" without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and non-infringement. By using this software, you acknowledge and agree that any performance testing activities are conducted at your own risk and discretion. The author explicitly disclaims all liability for any licensing requirements, direct, indirect, incidental, special, consequential or exemplary damages resulting from the use, misuse, or inability to use this software, whether based on warranty, contract, tort, or any other legal theory.

⚠️ Important: This MCP server is intended for ease of performance test creation and learning Artillery.io on local development machines only. For production use and licensing requirements, please consult with the Artillery team directly.

Overview

This server implements the Model Context Protocol to enable LLMs to create and run performance tests. It provides:

• Resources for Artillery YAML configuration templates and documentation
• Tools for generating, running, and analyzing performance tests
• Prompts for common load testing tasks

Features

• Five comprehensive tools for complete Artillery testing workflow
• Create performance tests from templates or custom configurations
• Enhanced HTML Reports: Comprehensive reports with stakeholder summaries and technical analysis
• Stakeholder-Friendly Summaries: Business impact assessments and actionable recommendations
• Modular Report Generation: Clean separation of HTML templates, stakeholder summaries, and technical details
• Run Artillery tests with automatic JSON and enhanced HTML report generation
• Analyze test results with performance insights and improvement suggestions
• Support for HTTP GET, POST, and complex flow testing scenarios
• Enhanced AI agent integration with detailed tool descriptions
• Local file management in ./artillery-tests/ directory (no temp files)
• Simple HTTP test creation tool for quick performance testing
• Built-in performance analysis with detailed metrics and recommendations
• Supports both STDIO and HTTP/SSE transports for MCP client communication

Prerequisites

Before using this package, ensure you have the following installed:

• Node.js: v16.x or higher
• npm: v8.x or higher
• Artillery: This package requires Artillery.io to be installed. You can install it globally:

  npm install -g artillery

System Requirements

• Minimum 16GB RAM recommended
• 1 GB free disk space
• Operating Systems supported:
  • Windows 10/11
  • macOS 10.15 or later
  • Linux (Ubuntu 20.04 or later recommended)

Required Permissions

• Network access for HTTP/SSE transport (default ports 3000)
• File system permissions for:
  • Writing test files
  • Creating log files
  • Generating reports

Optional but Recommended

• Git (for version control)
• A modern code editor (VS Code, Cursor, etc.) with MCP support
• Basic understanding of:
  • RESTful APIs
  • Performance testing concepts
  • YAML configuration files

Quick Start - STDIO Mode (Recommended)

Install and Start STDIO Server

1. Install the server globally:

   npm install -g artillery-performance-mcp-server

2. Start in STDIO mode:

   artillery-performance-mcp-server-stdio

3. Optional: Specify a working directory for test files:

   artillery-performance-mcp-server-stdio /path/to/your/project

   This will create all Artillery test files, results, and reports in the specified directory:

   /path/to/your/project/
   ├── test-configs/     # Artillery test configuration files (.yml)
   ├── test-results/     # Test results (.json)
   └── test-reports/     # HTML reports with stakeholder summaries

Configure in VS Code with GitHub Copilot

1. Open VS Code Settings (Cmd+, on Mac, Ctrl+, on Windows/Linux)

2. Search for "mcp" and find "Mcp Servers" setting

3. Click "Edit in settings.json" and add the Artillery MCP server configuration:

   {
     "mcp.servers": {
       "artillery-mcp": {
         "command": "artillery-performance-mcp-server-stdio",
         "args": []
       }
     }
   }

   To specify a custom working directory for test files:

   {
     "mcp.servers": {
       "artillery-mcp": {
         "command": "artillery-performance-mcp-server-stdio", 
         "args": ["/path/to/your/project"]
       }
     }
   }

4. Restart VS Code to apply the configuration

5. Verify the server is working by opening GitHub Copilot Chat and typing:

   @artillery-mcp Create a simple HTTP performance test for https://jsonplaceholder.typicode.com/posts

Configure in Cursor IDE

1. Open Cursor Settings and navigate to MCP Servers

2. Add new MCP server with the following configuration:

   {
     "mcpServers": {
       "artillery-mcp": {
         "type": "stdio",
         "command": "artillery-performance-mcp-server-stdio",
         "args": []
       }
     }
   }

3. Restart Cursor and verify by using the AI chat with:

   Use artillery-mcp tools to create a basic HTTP test for an API endpoint

Configure in Other MCP Clients

For other MCP-compatible clients (Claude Desktop, etc.), add to your MCP configuration:

{
  "mcpServers": {
    "artillery-mcp": {
      "command": "artillery-performance-mcp-server-stdio",
      "args": []
    }
  }
}

HTTP/SSE Transport (Alternative)

If you prefer HTTP transport over STDIO, you can run the server in HTTP mode:

# Start HTTP server (default port 3000)
artillery-performance-mcp-server

# Or specify custom port
PORT=4001 artillery-performance-mcp-server

This starts:

• HTTP Transport: http://localhost:3000/mcp
• SSE Transport: http://localhost:3000/sse

Configure in your MCP client using the HTTP URL. Note: STDIO mode is recommended for better performance and reliability.

Additional Client Configuration

For other MCP-compatible clients, refer to their documentation for adding MCP servers. Use the STDIO configuration shown above for the best experience:

• Command: artillery-performance-mcp-server
• Args: ["--stdio"]
• Type: stdio

Most clients support both STDIO and HTTP transports. STDIO is recommended for better performance and reliability.

MCP Resources

The server provides the following resources:

• artillery://templates - List all available test templates
• artillery://templates/{filename} - Get a specific test template
• artillery://examples/http-get - Example HTTP GET test template
• artillery://examples/http-post - Example HTTP POST test template
• artillery://examples/complex-flow - Example complex flow test template

MCP Tools

The server provides five comprehensive tools for Artillery performance testing:

Generate_Artillery_Test

Create Artillery performance tests from predefined templates with comprehensive configuration options.

Enhanced for AI Agents: This tool creates Artillery.io performance test configurations from templates. When specifying targets, provide the actual endpoint URL you want to test (e.g., "https://api.example.com" not just "Artillery.io").

Parameters:

• templateType: Test template (http-get, http-post, complex-flow)
• target: Target URL/hostname for testing
• endpoint: Specific endpoint path (default: "/")
• duration: Test duration in seconds (default: 30)
• arrivalRate: Requests per second during normal phase (default: 5)
• peakRate: Peak requests per second during ramp-up (optional)
• timeout: Request timeout in seconds (default: 10)
• scenarioName: Descriptive name for the test scenario (optional)
• auth: Authentication token for requests (optional)
• additionalConfig: Extra Artillery configuration options (optional)

File Output: Saves test files to ./artillery-tests/ directory with timestamped names.

Run_Artillery_Test

Execute Artillery performance tests and generate comprehensive reports with stakeholder summaries.

Enhanced for AI Agents: Runs Artillery.io performance tests and generates both JSON results and enhanced HTML reports with stakeholder summaries for analysis. Files are stored locally in the artillery-tests directory.

Parameters:

• testFilePath: Path to the Artillery test YAML file
• generateReport: Whether to create enhanced HTML report (default: true)
• reportType: Report type - 'enhanced' (with stakeholder summary) or 'simple' (basic metrics only, default: 'enhanced')

Enhanced Features:

• Stakeholder Summary: Executive overview with business impact assessment
• Technical Analysis: Detailed performance metrics and bottleneck identification
• Visual Charts: Response time distributions and performance indicators
• Actionable Recommendations: Business-friendly improvement suggestions
• Risk Assessment: Performance risk levels for stakeholder decision-making

Output: Creates JSON results file and comprehensive HTML report with both technical details and stakeholder-friendly summaries.

Analyze_Artillery_Test_Results

Analyze Artillery test results and provide performance insights and improvement recommendations.

Enhanced for AI Agents: Analyzes Artillery.io test results from JSON files and provides detailed performance insights, bottleneck identification, and actionable improvement recommendations.

Parameters:

• resultsFilePath: Path to the JSON results file from Artillery test
• suggestImprovements: Whether to provide improvement suggestions (default: true)

Analysis Includes: Response times, throughput, error rates, performance bottlenecks, and optimization recommendations.

Create_Custom_Artillery_Test

Create custom Artillery tests with full YAML configuration control.

Enhanced for AI Agents: Creates fully customized Artillery.io test configurations when templates don't meet specific requirements. Accepts complete YAML content for advanced test scenarios.

Parameters:

• name: Descriptive name for the custom test
• content: Complete Artillery YAML configuration content

Use Case: For complex test scenarios requiring custom phases, multiple targets, or advanced Artillery features.

Create_Simple_HTTP_Test

Quick creation of basic HTTP performance tests for immediate testing needs.

Enhanced for AI Agents: Simplified tool for creating basic HTTP performance tests against single endpoints. Perfect for quick performance checks or simple load testing scenarios.

Parameters:

• targetUrl: Complete URL to test (e.g., "https://api.example.com/health")
• method: HTTP method (default: "GET")
• duration: Test duration in seconds (default: 30)
• requestsPerSecond: Request rate (default: 5)
• scenarioName: Test description (optional)

Best For: Quick API endpoint testing, health check load testing, simple performance validation.

MCP Prompts

The server provides the following prompts:

generate-basic-http-test

Generate a basic HTTP performance test.

Parameters:

• targetUrl: Target URL for the test
• endpoint: Specific endpoint to test (default: "/")
• requestsPerSecond: Request rate (default: 5)
• testDuration: Test duration in seconds (default: 60)
• authToken: Authentication token (optional)

generate-api-flow-test

Generate a complex API flow test.

Parameters:

• targetUrl: Target URL for the test
• flows: Array of API calls in the flow
  • method: HTTP method (GET, POST, PUT, DELETE)
  • endpoint: Endpoint for the API call
  • payload: JSON payload (optional)
• requestsPerSecond: Request rate (default: 5)
• testDuration: Test duration in seconds (default: 60)
• authToken: Authentication token (optional)

analyze-test-results

Analyze performance test results.

Parameters:

• resultsOutput: Test results output
• performanceTargets: Performance targets (optional)
  • maxResponseTime: Maximum acceptable response time in ms
  • maxErrorRate: Maximum acceptable error rate percentage

create-load-test-plan

Create a comprehensive load testing plan.

Parameters:

• applicationDescription: Description of the application being tested
• endpoints: Array of endpoints to test
• userScenarios: Array of user scenarios (optional)
• performanceGoals: Array of performance goals (optional)

AI Agent Usage Guide

This Artillery MCP server is optimized for AI agent integration. Here's how to use it effectively:

Important: Avoid Endpoint Confusion

When working with AI agents, be specific about target URLs to avoid confusion:

✅ Correct: "Test performance of https://api.myservice.com/health" ❌ Incorrect: "Test performance of Artillery.io" (AI might interpret this as testing Artillery's website)

Recommended Prompts for AI Agents

For Quick Testing:

Create a simple HTTP performance test for https://jsonplaceholder.typicode.com/posts with 10 requests per second for 30 seconds

For API Flow Testing:

Generate an Artillery test for https://api.example.com that tests the login flow: POST /auth/login, then GET /user/profile, running for 60 seconds at 5 RPS

For Analysis:

Run the test and analyze the results, suggesting performance improvements

File Management

• All test files are stored in ./artillery-tests/ directory
• Test files use timestamped names for easy identification
• JSON results and HTML reports are generated in the same directory
• No temporary files - everything is kept locally for analysis

🚀 End-to-End Sample Prompts with Public APIs

Ready to test the Artillery MCP server? Use these complete prompts with public endpoints that you can try immediately:

📋 Complete Workflow Examples

Example 1: Basic API Performance Test (JSONPlaceholder)

Step 1: Create and Run Test

Use the Artillery MCP server tools to:
1. Create a basic HTTP GET performance test for https://jsonplaceholder.typicode.com/posts 
2. Configure it for 10 requests per second for 30 seconds
3. Run the test immediately after creation
4. Generate an enhanced HTML report with stakeholder summary

Step 2: Analyze Results

Analyze the Artillery test results from the JSON file generated in the previous test. Provide detailed performance insights and recommendations for the JSONPlaceholder API performance.

Example 2: REST API CRUD Operations Test

Complete Workflow Prompt:

Create a comprehensive Artillery performance test for https://jsonplaceholder.typicode.com with the following scenarios:
1. GET /posts (retrieve posts)
2. POST /posts (create new post with sample data)
3. GET /posts/1 (retrieve specific post)
4. PUT /posts/1 (update post)

Configure the test for:
- Duration: 10 seconds
- Arrival rate: 5 requests per second
- Include authentication headers if needed

After creating the test:
1. Run the Artillery test with enhanced reporting
2. Analyze the results and provide stakeholder-friendly recommendations

Example 3: API Health Check Performance

Single Comprehensive Prompt:

Perform a complete performance analysis of the GitHub API health endpoint:

1. Create a simple HTTP GET test for https://api.github.com/zen
2. Configure for 15 requests per second for 15 seconds
3. Run the test with enhanced HTML reporting
4. Analyze the JSON results and provide:
   - Performance summary for stakeholders
   - Technical recommendations for developers
   - Business impact assessment
   - Risk level evaluation

Generate both technical and business-friendly insights from the test results.

Example 4: E-commerce API Simulation

Advanced Workflow Prompt:

Create and execute a realistic e-commerce API performance test using https://fakestoreapi.com:

Test Scenarios:
1. Browse products: GET /products
2. View product details: GET /products/1
3. Get categories: GET /products/categories
4. Filter by category: GET /products/category/electronics

Configuration:
- Test duration: 10 seconds
- Start with 5 RPS, peak at 15 RPS
- Generate enhanced report with stakeholder summary

After execution:
1. Run the test and generate comprehensive reports
2. Analyze performance from both technical and business perspectives
3. Provide actionable recommendations for production deployment

🎯 Quick Start Prompts (Copy & Paste Ready)

For VS Code/Cursor/Claude Users:

Basic Performance Test:

@artillery-mcp Create a simple HTTP performance test for https://httpbin.org/get with 8 requests per second for 20 seconds, then run it with enhanced reporting and analyze the results.

API Load Testing:

@artillery-mcp Generate a performance test for https://reqres.in/api/users that tests both GET /api/users and POST /api/users endpoints, run for 30 seconds at 6 RPS, execute the test, and provide stakeholder-friendly analysis.

Health Check Monitoring:

@artillery-mcp Test the performance of https://httpbin.org/status/200 endpoint with 12 requests per second for 15 seconds, generate enhanced HTML report, and analyze for production readiness.

🔧 Troubleshooting Test Prompts

When Tests Fail:

My Artillery test failed with errors. Please:
1. Analyze the error output
2. Suggest configuration changes
3. Create a modified test with better error handling
4. Re-run with conservative settings (2 RPS for 10 seconds)

For Custom Endpoints:

Create a performance test for [YOUR_API_ENDPOINT_HERE]:
- Method: GET/POST (specify your method)
- Requests per second: 5
- Duration: 30 seconds
- Include appropriate headers if needed
- Run with enhanced reporting
- Provide business impact analysis

📊 Expected Outputs

After using these prompts, you'll get:

1. Test Configuration File: artillery-tests/test-name-timestamp.yml
2. JSON Results: artillery-tests/artillery-results-timestamp.json
3. Enhanced HTML Report: artillery-tests/artillery-report-timestamp.html
4. Analysis Summary: Performance insights and recommendations

💡 Pro Tips for Better Results

• Be Specific: Include exact URLs, request rates, and duration
• Use Public APIs: Start with reliable public endpoints for testing
• Request Analysis: Always ask for result analysis after running tests
• Stakeholder Reports: Use "enhanced" report type for business-friendly summaries
• Error Handling: If a test fails, ask for troubleshooting and retry with lower load

Enhanced HTML Reports

The Run_Artillery_Test tool now generates comprehensive HTML reports that include both technical analysis and stakeholder-friendly summaries.

Report Features

Executive Summary Section:

• Overall Performance Status: Clear indicators (Excellent, Good, Fair, Poor, Critical)
• Key Metrics Display: Average response time, success rate, throughput
• Business Impact Assessment: Risk levels and performance implications
• Actionable Recommendations: Prioritized improvement suggestions

Technical Details Section:

• Response Time Charts: Visual distribution of response times (min, avg, P95, max)
• Detailed Statistics Table: Comprehensive metrics with performance status indicators
• Error Analysis: Detailed breakdown of any errors encountered
• Test Timeline: Start time, end time, and duration tracking

Stakeholder Benefits:

• Business-Friendly Language: Technical metrics translated to business impact
• Risk Assessment: Clear risk levels for decision-making
• Next Steps Guidance: Specific actions based on performance results
• Visual Indicators: Color-coded status indicators for quick assessment

Report Types

1. Enhanced Report (default): Full stakeholder summary + technical details
2. Simple Report: Basic metrics with minimal business context

Examples

💡 Looking for complete workflow examples? Check out the End-to-End Sample Prompts section above for ready-to-use prompts with public APIs that you can try immediately!

Tool-Specific Examples

Example: Generate enhanced report

Using the Run_Artillery_Test tool with enhanced reporting:

{
  "testFilePath": "./artillery-tests/api-test-20240101-120000.yml",
  "generateReport": true,
  "reportType": "enhanced"
}

This generates:

• JSON results file with raw metrics
• Enhanced HTML report with stakeholder summary
• Technical analysis with visual charts
• Business impact assessment

Example: Generate simple report

{
  "testFilePath": "./artillery-tests/api-test-20240101-120000.yml",
  "generateReport": true,
  "reportType": "simple"
}

Example: Generate a basic HTTP test

Using the Generate_Artillery_Test tool:

{
  "templateType": "http-get",
  "target": "https://jsonplaceholder.typicode.com",
  "endpoint": "/posts",
  "duration": 30,
  "arrivalRate": 10,
  "scenarioName": "Posts API Load Test"
}

Example: Create a simple HTTP test

Using the Create_Simple_HTTP_Test tool:

{
  "targetUrl": "https://api.example.com/health",
  "method": "GET",
  "duration": 60,
  "requestsPerSecond": 15,
  "scenarioName": "Health Check Load Test"
}

Example: Run a test

Using the Run_Artillery_Test tool:

{
  "testFilePath": "./artillery-tests/posts-api-load-test-20240101-120000.yml",
  "generateReport": true
}

Example: Analyze test results

Using the Analyze_Artillery_Test_Results tool:

{
  "resultsFilePath": "./artillery-tests/posts-api-load-test-20240101-120000-results.json",
  "suggestImprovements": true
}

Example: Custom test creation

Using the Create_Custom_Artillery_Test tool:

{
  "name": "Complex API Flow Test",
  "content": "config:\n  target: https://api.example.com\n  phases:\n    - duration: 60\n      arrivalRate: 10\nscenarios:\n  - name: Login and Browse\n    flow:\n      - post:\n          url: /auth/login\n          json:\n            username: testuser\n            password: testpass\n      - get:\n          url: /dashboard"
}

File Structure

After running tests, your directory structure will look like:

your-project/
├── artillery-tests/
│   ├── api-test-20240101-120000.yml           # Generated test file
│   ├── api-test-20240101-120000-results.json  # Test results (JSON)
│   └── api-test-20240101-120000-report.html   # Test report (HTML)
└── ... (your other files)

Recent Improvements

Recent Improvements

Latest Features (August 2025)

• ✅ End-to-End Sample Prompts: Complete workflow examples with public APIs for immediate testing
• ✅ Enhanced HTML Reports: Comprehensive reports with stakeholder summaries and technical analysis
• ✅ Stakeholder-Friendly Summaries: Business impact assessments with actionable recommendations
• ✅ Modular Report Architecture: Clean separation of HTML templates, summaries, and technical details
• ✅ Visual Performance Charts: Response time distributions and performance indicators
• ✅ Risk Assessment Framework: Performance risk levels for business decision-making
• ✅ Public API Examples: Ready-to-use prompts with JSONPlaceholder, GitHub API, HTTPBin, and more

Previous Improvements

• ✅ Fixed zod compatibility: Downgraded from v4 to v3.25.x for MCP SDK compatibility
• ✅ Enhanced development environment: Replaced ts-node with tsx for better ES module support
• ✅ Improved file management: Changed from temp directory to local ./artillery-tests/ directory
• ✅ Enhanced AI agent integration: Added detailed tool descriptions and usage guidance
• ✅ Added Create_Simple_HTTP_Test: New tool for quick HTTP endpoint testing
• ✅ Fixed ENAMETOOLONG errors: Resolved file path issues in test analysis
• ✅ Improved error handling: Better error messages and debugging information

Development Setup

# Development with auto-restart
npm run dev

# Production build and start
npm run build
npm start

Known Issues & Limitations

• Data-driven testing: Advanced data-driven test features are work in progress
• Complex scenarios: For highly complex test scenarios, use Create_Custom_Artillery_Test with custom YAML
• Authentication: Currently supports token-based auth; OAuth flows require custom configuration
• Monitoring: Real-time monitoring during test execution is limited to Artillery's built-in output

For complex requirements not covered by the templates, use the Create_Custom_Artillery_Test tool with your complete Artillery configuration.

Uninstallation

npm uninstall artillery-performance-mcp-server

License

MIT

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Development Guidelines

• Follow TypeScript best practices
• Ensure MCP SDK compatibility (currently requires zod v3.25.x)
• Test both stdio and HTTP transports
• Update tool descriptions for AI agent clarity
• Maintain file organization in ./artillery-tests/ directory