⚡ fast-api-tester

A lightweight, zero-overhead API testing engine built for high-performance Node.js environments.

fast-api-tester provides a clean, framework-agnostic client to run fast HTTP assertions against REST APIs. Designed with connection pooling and low-latency execution in mind, it works seamlessly with Vitest, Jest, or Node's native node:assert.

📦 Installation

Install fast-api-tester using your preferred package manager:

# npm
npm install fast-api-tester

# pnpm
pnpm add fast-api-tester

# yarn
yarn add fast-api-tester

For TypeScript projects, also install execution tools or test runners as development dependencies:

npm install -D vitest typescript @types/node

🚀 Quick Start

Option A: Using Vitest (Recommended)

1. Ensure "type": "module" is in your package.json:

{
  "type": "module",
  "scripts": {
    "test": "vitest run"
  }
}

2. Create a test file at tests/api.test.ts:

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { ApiEngine } from 'fast-api-tester';

describe('JSONPlaceholder API Test Suite', () => {
  let api: ApiEngine;

  beforeAll(() => {
    api = new ApiEngine();
    api.setBaseUrl('https://jsonplaceholder.typicode.com');
  });

  afterAll(async () => {
    // Clean up socket connection pools
    await api.destroy();
  });

  it('GET /posts/1 - should fetch post details', async () => {
    const res = await api.get('/posts/1');

    expect(res.statusCode).toBe(200);
    expect(res.body).toHaveProperty('id', 1);
    expect(res.latencyMs).toBeGreaterThan(0);
  });

  it('POST /posts - should create a new post', async () => {
    const payload = {
      title: 'Fast API Testing',
      body: 'Testing with fast-api-tester',
      userId: 1,
    };

    const res = await api.post('/posts', payload);

    expect(res.statusCode).toBe(201);
    expect(res.body).toMatchObject(payload);
  });
});

3. Run your tests:

npm test

Option B: Standalone Script (Zero Runner Setup)

If you don't want to use a runner like Vitest or Jest, run a script directly using tsx and Node's native assert:

// run-tests.ts
import assert from 'node:assert/strict';
import { ApiEngine } from 'fast-api-tester';

const api = new ApiEngine();
api.setBaseUrl('https://jsonplaceholder.typicode.com');

async function run() {
  try {
    console.log('⚡ Running API assertions...');

    const res = await api.get('/posts/1');
    assert.equal(res.statusCode, 200, 'Expected status code 200');

    console.log(`✅ Success! Response status: ${res.statusCode} (${res.latencyMs}ms)`);
  } catch (err) {
    console.error('❌ Test failed:', err);
  } finally {
    await api.destroy();
  }
}

run();

Run it instantly:

npx tsx run-tests.ts

🛠️ API Reference

new ApiEngine(options?)

Creates a new instance of the HTTP engine.

• setBaseUrl(url: string): Sets the default host URL for all relative request paths.
• get(path, headers?): Executes an HTTP GET request.
• post(path, body?, headers?): Executes an HTTP POST request.
• put(path, body?, headers?): Executes an HTTP PUT request.
• delete(path, headers?): Executes an HTTP DELETE request.
• destroy(): Gracefully closes active network sockets and releases resources.

Response Object Structure

All HTTP methods return a uniform response structure:

{
  statusCode: number;   // e.g., 200, 201, 404
  body: any;            // Parsed JSON response payload
  headers: Record<string, string | string[]>; // Response headers
  latencyMs: number;    // Round-trip duration in milliseconds
}

📡 Advanced Features

Standalone Monitoring Setup (startHeartbeat)

For continuous monitoring — as opposed to a single-run test suite — use startHeartbeat to run checks on a recurring interval. This is kept separate from your Vitest/Jest suites so that test runners don't auto-terminate the infinite monitoring loop.

import { ApiEngine } from './index';

const api = new ApiEngine().setBaseUrl('https://api.example.com');

// Continuous heartbeat monitoring loop
api.startHeartbeat(async () => {
  const res = await api.get('/health');
  res.expectLatencyUnder(200);
});

📊 Interactive HTML Analytics & Historical Tracking

Every execution run updates test-report.html and persists historical execution data to .test-history.json (retaining the last 10 runs).

• Current Run Breakdown: Visual status pie chart for fast status verification.
• Historical Execution Trend: Stacked bar chart visualizing test pass/fail trends over time across consecutive heartbeats.
• Detailed Table View: Complete summary including HTTP Method, Endpoint, Status Code, Measured Latency, SLA Target, and exact error stack/messages for failing requests.

You can also opt in to a plain JSON export of a run alongside the HTML report — see Executive Summary Generation below.

🐌 Bandwidth Throttling Simulator

Test your API's resilience and SLA assertions under real-world network conditions. The bandwidth simulator introduces precise artificial latency to mimic poor connections, ensuring your timeouts and SLA limits are truly battle-tested before hitting production.

Available Profiles:

• Slow-3G: 500ms delay
• 3G: 300ms delay
• 4G: 100ms delay
• None: 0ms delay (Default)
• Custom: Pass any raw number to simulate exact millisecond latency.

Example Usage:

import { ApiEngine } from './index';

// Initialize the engine with a 3G network profile (adds 300ms artificial delay)
const api = new ApiEngine()
  .setBaseUrl('https://api.example.com')
  .simulateNetwork('3G');

api.startHeartbeat(async () => {
  const res = await api.get('/health');

  // If the server takes 250ms to respond, total measured latency will be 550ms.
  // This helps ensure your application handles degraded network states gracefully.
  res.expectLatencyUnder(500);
});

⚖️ Assertion Modes (Hard vs. Soft)

The framework supports two distinct validation behaviors to give you maximum flexibility when testing APIs:

• Hard Assertions (Default): Designed for strict control flows where subsequent steps depend entirely on the success of the current step. When a hard assertion fails (e.g., unexpected status code or SLA violation), it immediately records the failure, updates the test execution record, and throws a fatal Error that aborts the test suite execution instantly.
• Soft Assertions: Designed for comprehensive test reporting where you want to execute an entire suite or multiple checks without stopping at the first error. When a soft assertion fails, it logs a warning, flags the failure in the test record, and continues test execution. This ensures all endpoints are tested and included in the final HTML report and telemetry history, rather than failing fast.

How the Implementation Works

1. Configuration & Mode State (AssertionMode)

• Global Level: You can configure the engine-wide default behavior using engine.setAssertionMode('soft') or 'hard'. Every request made through the engine inherits this baseline mode.
• Inline Level: You can override the mode on a per-response chain using .soft() or .hard() right before your assertions.

2. Centralized Error Handling (handleAssertionFailure)

Instead of directly throwing errors inside every expectation method, assertions pass their failure message to a centralized handler:

• It updates the TestExecutionRecord by setting passed = false and appends the error message.
• If the mode is 'hard', it throws an Error immediately.
• If the mode is 'soft', it catches/bypasses the throw, logs a warning message to the console (console.warn), and allows the script execution to proceed normally.

const engine = new ApiEngine()
  .setBaseUrl('https://api.example.com')
  .setAssertionMode('soft'); // Make all requests log failures but continue

await engine.startHeartbeat(async () => {
  // Even if this fails (e.g., latency > 50ms), execution continues!
  await engine.get('/users').then(res => res.expectStatusOk().expectLatencyUnder(50));

  // This request STILL executes and gets recorded in the HTML report
  await engine.get('/orders').then(res => res.expectStatus(200));

  // Abort suite hard AT THE END if any soft assertions failed above
  engine.assertAll();
});

⚡ Key Visualizations & Diagnostic Features

🌊 1. 500 Cascade Waterfall Timeline (Floating Execution Window)

• What it does: Renders a floating horizontal bar chart displaying the exact timing, duration, and execution sequence of every HTTP request in your test workflow.
• Why it matters: In sequential API chains (e.g., Auth → Cart → Checkout), a single 500 Internal Server Error can stall downstream services. The waterfall chart visually pinpoints the precise endpoint and millisecond offset where the execution chain collapsed.
• Color Coding:
  • 🟢 Green: Successful execution (2xx / 3xx).
  • 🟠 Orange: Client-side failure (4xx).
  • 🔴 Red Alert: Critical backend server fault (500 / 5xx).

🥧 2. Status Code Distribution Matrix

• What it does: Categorizes responses into high-level HTTP status families (2xx Success, 4xx Client Errors, 5xx Server Failures).
• Why it matters: Quickly exposes whether test failures are caused by client assertion mismatches (404 / 422) or infrastructure/service outages (500 / 502 / 503).

📈 3. Historical Run Trend Analysis

• What it does: Tracks pass/fail counts across the last 10 test execution cycles (persisted automatically via local execution history).
• Why it matters: Detects flaky endpoints, intermittent microservice degradation, and build stability trends over time when running continuous heartbeat monitoring.

📄 4. One-Click Executive PDF Export

• What it does: Embedded browser-side PDF generation via client-side streaming (PDFKit & Blob Stream).
• Why it matters: Instantly creates a clean, C-suite ready PDF summary containing SLA metrics, overall pass rates, and detailed execution logs — no external server-side PDF generator binaries required.

🚀 Semantic Status Code Helpers

To make test assertions more expressive, readable, and maintainable, a comprehensive suite of semantic status code helpers has been integrated into the ApiResponse class.

These new methods seamlessly wrap around the existing .expectStatus() mechanism and error-handling flow, keeping the core engine intact while drastically improving test script readability.

Summary of Available Helpers

| Category | Method | Expected Status Code / Range | |---|---|---| | 2xx Success | .expectStatusCreated() | 201 Created | | 2xx Success | .expectStatusAccepted() | 202 Accepted | | 2xx Success | .expectStatusNoContent() | 204 No Content | | 2xx Success | .expectStatus2xx() | Any 200–299 status code | | 4xx Client Errors | .expectStatusBadRequest() | 400 Bad Request | | 4xx Client Errors | .expectStatusUnauthorized() | 401 Unauthorized | | 4xx Client Errors | .expectStatusForbidden() | 403 Forbidden | | 4xx Client Errors | .expectStatusNotFound() | 404 Not Found | | 4xx Client Errors | .expectStatusUnprocessableEntity() | 422 Unprocessable Entity | | 4xx Client Errors | .expectStatus4xx() | Any 400–499 status code | | 5xx Server Errors | .expectStatusInternalServerError() | 500 Internal Server Error | | 5xx Server Errors | .expectStatusBadGateway() | 502 Bad Gateway | | 5xx Server Errors | .expectStatusServiceUnavailable() | 503 Service Unavailable | | 5xx Server Errors | .expectStatus5xx() | Any 500–599 status code |

🤖 AI Swagger Generation

This project utilizes AI to automatically generate and maintain our OpenAPI/Swagger documentation. This ensures our API documentation stays up-to-date with our codebase with minimal manual effort.

Prerequisites

To run the AI generation script, you must have a valid API key configured in your environment.

For macOS/Linux (Bash/Zsh):

export OPENROUTER_API_KEY="your-api-key-here"

For Windows (PowerShell):

$env:OPENROUTER_API_KEY="your-api-key-here"

Note: Do not commit your .env files or API keys to version control.

Usage

To trigger the AI Swagger generation, run the following command:

# Generate the swagger documentation
npx tsx tests/generate.ts

# OR if you have a script defined in package.json:
npm run generate:docs

Generator Script (tests/generate.ts)

import { ApiEngine } from 'fast-api-tester';

async function runGenerator() {
  const engine = new ApiEngine();
  await engine.generateAITests('aiPrompt.txt');
}

runGenerator().catch(console.error);

aiPrompt.txt File Structure

#prompt 1. from https://petstore.swagger.io/#/ create fast-api-tester test for GET call store/inventory
#prompt 2. from https://petstore.swagger.io/#/ create fast-api-tester test for GET call pet/findByStatus

How it Works

1. The script parses the target source files or endpoint definitions.
2. The AI model analyzes the expected inputs, outputs, and logic.
3. A standardized swagger.json (or swagger.yaml) file is generated/updated in the project directory.

📦 Postman Collection to Fast API Tester Conversion

Description

The Postman Collection Conversion Engine automatically transforms exported Postman Collections into executable, native fast-api-tester TypeScript test suites. Powered by OpenRouter AI (nvidia/nemotron-3-nano-30b-a3b:free), it parses Postman requests, environment variables, and payload structures, translating them into individual test blocks utilizing fast-api-tester's chainable assertions.

If no API key is present or if the AI service call fails, the converter automatically falls back to a deterministic local code generator to guarantee reliable test generation.

Key Features

• Flexible Sources: Ingests Postman collection files directly via local file paths, online URLs, or raw JSON string content.
• Environment & Variable Support: Accepts Postman environment files to substitute variables like {{baseUrl}} or {{token}} throughout generated scripts.
• Data-Driven Testing: Supports external JSON or CSV data files to structure iterative testing over dynamic datasets.
• Automatic Assertion Mapping: Maps collection operations to native fast-api-tester chainable methods such as .expectStatus2xx(), .expectHeaderJson(), and .expectJsonPathExists().
• Fallback Generator: Features a local deterministic spec generator when offline or when OPENROUTER_API_KEY is omitted.

How To Use It

You can generate test scripts directly using static utilities, ApiEngine instance methods, or the standalone converter class.

Method 1: Using ApiEngine Utility Methods

import { ApiEngine } from 'fast-api-tester';

async function generateTests() {
  // Generate test suite directly from a local or remote Postman collection
  const tsCode = await ApiEngine.generateSuiteFromPostman('./collections/my_api.json', {
    baseUrl: 'http://localhost:3000',
    outputPath: './tests/generated_postman.spec.ts',
    environment: './env/dev.postman_environment.json', // Optional Postman environment
    dataFile: './data/test_cases.json',                 // Optional data-driven test dataset
    assertionMode: 'hard',                              // 'hard' or 'soft'
  });

  console.log('Generated Test Code:\n', tsCode);
}

generateTests();

Method 2: Using an Active ApiEngine Instance

import { ApiEngine } from 'fast-api-tester';

const api = new ApiEngine();
api.setBaseUrl('http://localhost:3000');

// Inherits configured instance defaults (baseUrl, assertionMode)
await api.importPostmanAndRun('https://example.com/api-collection.json', {
  outputPath: './tests/imported_postman.spec.ts',
});

Method 3: Direct Usage via PostmanToFastApiTesterConverter

import { PostmanToFastApiTesterConverter } from 'fast-api-tester';

await PostmanToFastApiTesterConverter.convert('./postman_collection.json', {
  apiKey: process.env.OPENROUTER_API_KEY, // Optional override (defaults to process.env.OPENROUTER_API_KEY)
  baseUrl: 'http://localhost:3000',
  outputPath: './tests/converted.spec.ts',
  useFallbackOnAiError: true,             // Fall back to local generator if AI call fails
});

Conversion Options Reference

| Option | Type | Default | Description | | :--- | :--- | :--- | :--- | | apiKey | string | process.env.OPENROUTER_API_KEY | OpenRouter API Key for AI conversion. | | baseUrl | string | 'http://localhost:3000' | Target base URL for the generated API test suite. | | outputPath | string | undefined | Optional file path where generated TypeScript code will be saved. | | assertionMode | 'hard' \| 'soft' | 'hard' | Default assertion behavior (hard aborts test on failure; soft logs and continues). | | environment | string | undefined | Path or raw JSON string for a Postman environment file. | | dataFile | string | undefined | Path or raw text for CSV/JSON files used in data-driven tests. | | useFallbackOnAiError | boolean | true | Whether to fall back to the local deterministic generator if the AI call fails. |

🎭 Playwright to Fast API Tester Conversion

You can generate test scripts directly using static utilities, fast-api-tester instance methods, or the standalone converter class.

Method 1: Using ApiEngine Utility Methods

import { ApiEngine } from 'fast-api-tester';

async function generateTests() {
  // Generate a robust TypeScript test suite directly from a local or remote Playwright API test file
  const tsCode = await ApiEngine.generateSuiteFromPlaywright('./tests/playwright_api.spec.ts', {
    baseUrl: 'http://localhost:3000',
    outputPath: './tests/generated_playwright.spec.ts',
    assertionMode: 'hard', // Configures test assertions as 'hard' (abort on fail) or 'soft' (log and continue)
  });

  console.log('Generated Test Code:\n', tsCode);
}

generateTests();

Method 2: Using an Active ApiEngine Instance

import { ApiEngine } from 'fast-api-tester';

const api = new ApiEngine();
api.setBaseUrl('http://localhost:3000');
api.setAssertionMode('hard');

// Inherits configured instance defaults (baseUrl, assertionMode) automatically while converting
await api.importPlaywrightAndRun('https://example.com/tests/api.spec.ts', {
  outputPath: './tests/imported_playwright.spec.ts',
});

Detailed Overview of Features

• AI-Powered Translation: Leverages advanced language models via OpenRouter (nvidia/nemotron-3-nano-30b-a3b:free) to intelligently parse and transform Playwright API test syntax into idiomatic fast-api-tester equivalents.
• Flexible Ingestion Sources: Supports reading target test scripts from local file paths, remote URLs, or directly passed raw code strings.
• Intelligent Method Mapping: Automatically converts Playwright request contexts (request.get(), request.post(), etc.) and native assertions into chainable fast-api-tester response validators (e.g., .expectStatusOk(), .expectHeaderJson(), .expectBodyMatchObject()).
• Automatic Fallback Mechanism: Features an intelligent deterministic parser that generates standard test files locally if the AI service encounters connectivity issues or if an OpenRouter API key is missing.
• Complete Test Scaffolding: Automatically wires up required framework hooks including Vitest imports (describe, it, beforeAll, afterAll), proper ApiEngine setup, and cleanup routines.

🛡️ AI Security Test Generation (generateSecurityTests)

generateSecurityTests is an automated security test generation function built into ApiEngine. It reads natural language security prompts from a specification file (by default securityPrompt.txt), sends them to OpenRouter (using nvidia/nemotron-3-nano-30b-a3b:free), and dynamically outputs executable TypeScript security test suites (.spec.ts) targeting OWASP API Top 10 vulnerabilities.

The generated test suites utilize fast-api-tester alongside Vitest test runners to send adversarial payloads (e.g., SQL injections, XSS scripts, invalid UUIDs) and assert proper application defense, error handling, and input validation.

Key Features

• OWASP API Vulnerability Coverage: Automatically crafts tests targeting common vulnerabilities like BOLA, SSRF, BOPLA, SQL Injections, and XSS.
• Adversarial Payload Generation: Tests system resilience using invalid data, malicious code injection strings, and unauthorized request structures.
• OpenRouter AI Integration: Powered by OpenRouter using the nvidia/nemotron-3-nano-30b-a3b:free model.
• Automated File Output: Automatically creates executable .spec.ts files inside a designated tests/ directory.
• Native Assertion Integration: Generates tests using chainable fast-api-tester response assertions (e.g., .expectStatus4xx(), .expectStatus5xx(), .expectBodyContains()).

Method Signature

public async generateSecurityTests(promptFileName: string = 'securityPrompt.txt'): Promise<void>

Parameters

• promptFileName (string, optional): The path or name of the prompt text file relative to the root directory. Defaults to 'securityPrompt.txt'.

Prerequisites

• An OPENROUTER_API_KEY environment variable must be set.
• The targeted prompt text file must exist in the root directory and contain lines prefixed with #prompt or # prompt.

Usage Example

1. Create securityPrompt.txt

Create a file named securityPrompt.txt in your project root containing your test prompts:

#prompt Test GET /api/users/{id} for Broken Object Level Authorization (BOLA) by requesting unauthorized IDs.
#prompt Test POST /api/comments payload for stored and reflected XSS vulnerabilities.
#prompt Test POST /api/login for SQL Injection resilience in email and password fields.

2. Execute generateSecurityTests in Code

import { ApiEngine } from 'fast-api-tester';

async function runSecurityGenerator() {
  const engine = new ApiEngine();

  // Triggers AI generation from securityPrompt.txt
  await engine.generateSecurityTests('securityPrompt.txt');
}

runSecurityGenerator();

🧠 AI Assertion Synthesis (autoSynthesizeAssertions)

It uses the OpenRouter API to automatically inspect HTTP responses (status codes, headers, response body/payload) and infer precise assertion criteria (expected status ranges, structural schemas, data types, required fields, and value pattern checks) without requiring manual schema definitions.

const engine = new APIEngine(process.env.OPENROUTER_API_KEY!);

// 1. Make API request using your existing engine functionality
const response = await engine.executeRequest({ url: 'https://api.example.com/users/1', method: 'GET' });

// 2. Automatically synthesize assertions from the response payload
const synthesized = await engine.autoSynthesizeAssertions({
  status: response.status,
  headers: response.headers,
  body: response.data,
});

console.log('Inferred Success Criteria:', JSON.stringify(synthesized, null, 2));

🧪 What The QA Engineer Agent Automatically Tests

When configured with fast-api-tester and OpenRouter, The QA Engineer automatically generates and executes payloads covering:

Tested Mutation Categories

• Boundary Limits:
  • Strings: "" (empty), 10,000+ character strings, SQL/script injection payloads.
  • Numbers: 0, -1, MAX_SAFE_INTEGER, floating points when integers are expected.
  • Arrays: Empty lists [], single item, 1000+ items.
• Null & Missing Values:
  • Key set to null ({"username": null}).
  • Key completely omitted.
  • Keys with whitespace or "".
• Assertion Synthesis:
  • Inferred HTTP status expectations (400 Bad Request or 422 Unprocessable Entity for invalid payloads vs. 200/201 for valid ones).
  • Auto-generated schema validation rules checking field types and nested structure integrity.

🕵️ The QA Engineer Agent

What The QA Engineer Agent Automatically Tests

When configured with fast-api-tester and OpenRouter, The QA Engineer automatically generates and executes payloads covering:

🧪 Tested Mutation Categories

• Boundary Limits:
  • Strings: "" (empty), 10,000+ character strings, SQL/script injection payloads.
  • Numbers: 0, -1, MAX_SAFE_INTEGER, floating points when integers are expected.
  • Arrays: Empty lists [], single item, 1000+ items.
• Null & Missing Values:
  • Key set to null ({"username": null}).
  • Key completely omitted.
  • Keys with whitespace or "".
• Assertion Synthesis:
  • Inferred HTTP status expectations (400 Bad Request or 422 Unprocessable Entity for invalid payloads vs. 200/201 for valid ones).
  • Auto-generated schema validation rules checking field types and nested structure integrity.

📋 Features

• QA Persona Prompts: Instruct AI models using custom persona prompts to catch edge cases typical human testers might miss.
• Boundary & Limit Testing: Automated test generation for max integers, payload field lengths, off-by-one errors, and empty strings.
• Resilience & Null Safety: Tests missing parameters, null/undefined fields, negative values, and special character injection.
• Seamless Integration: Built on vitest for fast execution and simple TypeScript assertions.

🛠️ Step-by-Step Setup Guide

Step 1: Install Dependencies & Configure API Keys

Install fast-api-tester alongside vitest and undici in your project development dependencies:

npm install -D fast-api-tester vitest undici

Set your OpenRouter API key in your environment so fast-api-tester can communicate with OpenRouter models:

• macOS / Linux:

export OPENROUTER_API_KEY="your-openrouter-api-key"

• Windows (PowerShell):

$env:OPENROUTER_API_KEY="your-openrouter-api-key"

Step 2: Define QA Persona Prompts in aiPrompt.txt

Create a file named aiPrompt.txt in your project root directory. Use the #prompt directive to instruct the AI persona on the specific boundary limit and edge case scenarios you want to cover.

aiPrompt.txt

#prompt Act as a Senior QA Automation Engineer. Generate boundary limit test cases for POST /api/v1/users (test zero, max integer, 10k character strings, empty strings, and off-by-one payload field lengths).
#prompt Act as a Senior QA Automation Engineer. Generate edge case and null value test cases for PUT /api/v1/orders/123 (test missing required fields, null payloads, unexpected data types, negative quantities, and special characters/SQL injection patterns).

Step 3: Create the Generation Script

Create a script named generate-tests.ts. This script instantiates ApiEngine, sets your base target URL, and executes generateAITests(), which reads aiPrompt.txt and outputs generated TypeScript .spec.ts files.

generate-tests.ts

import { ApiEngine } from 'fast-api-tester';

async function generateQAEastCases() {
  const api = new ApiEngine();

  // Set target base URL for the API under test
  api.setBaseUrl('https://api.yourdomain.com');

  console.log('Generating boundary & null-value test suites...');

  // Reads aiPrompt.txt and generates test files under ./tests
  await api.generateAITests('aiPrompt.txt');

  console.log('Test generation complete! Check the ./tests directory.');
}

generateQAEastCases().catch(console.error);

👵 The End-User Persona

In fast-api-tester, the 👵 The End-User persona is a pre-configured AI directive/prompt strategy designed for End-to-End (E2E) workflow and stateful journey testing.

Unlike personas focused on breaking an application (like the QA Engineer persona for nulls/boundaries or the Penetration Tester persona for security), The End-User persona models normal, realistic human behavior across multi-step transactions.

🎯 What Does "The End-User" Persona Do?

When you pass an End-User prompt into fast-api-tester's AI generator (e.g., via generateAITests('aiPrompt.txt')), OpenRouter AI acts as a customer interacting with your application.

It generates test suites that:

1. Maintain Context & State Across Calls:
   • It takes data returned from Step A (e.g., user_id, auth_token, cart_id) and injects it into Step B and Step C.
2. Simulate Business Processes:
   • It creates multi-endpoint sequences like:
     • POST /api/v1/auth/login → (saves JWT token)
     • POST /api/v1/cart/items → (adds product)
     • POST /api/v1/checkout → (completes order)
     • GET /api/v1/orders/status → (verifies purchase)
3. Assert Expected Happy-Path Outcomes:
   • Checks that status codes (e.g., 200 OK, 201 Created), response structures, and payload values match an end user's expected flow.

💻 How to Use It in Your Project

1. Define the End-User Prompt in aiPrompt.txt

In your aiPrompt.txt file, use the #prompt directive and explicitly assign The End-User persona:

#prompt Act as 👵 The End-User. Generate a stateful end-to-end user workflow test for an e-commerce platform. Start by authenticating at POST /api/v1/login, take the returned token to add an item to POST /api/v1/cart, proceed to POST /api/v1/checkout, and finally confirm the order status with GET /api/v1/orders/{orderId}.

2. Generate the End-to-End Test Suite

Run your generation script:

import { ApiEngine } from 'fast-api-tester';

async function generateEndUserFlows() {
  const api = new ApiEngine();
  api.setBaseUrl('https://api.yourdomain.com');

  // Generates workflow test specs based on the 👵 End-User persona prompt
  await api.generateAITests('aiPrompt.txt');
}

generateEndUserFlows().catch(console.error);

📚 The Documenter

The Documenter refers to the dedicated utility (or middleware module) responsible for automatically capturing, formatting, and generating structured documentation for API requests and responses on the fly.

Here is a breakdown of its role, architecture, and behavior within the apiEngine.ts context.

What "The Documenter" Does

The Documenter acts as an inline observer inside the API execution engine. Rather than requiring us to write static OpenAPI/Swagger specs by hand, it dynamically analyzes real traffic passing through the engine to build and maintain up-to-date API documentation.

Core Responsibilities

• Payload & Schema Extraction: It intercepts incoming request payloads (query params, headers, body) and outgoing responses, automatically inferring data types and structural schemas.
• Endpoint Cataloging: It automatically registers active routes, HTTP methods (GET, POST, PUT, etc.), and execution paths handled by apiEngine.ts.
• Metadata Attachment: It enriches bare route definitions with contextual metadata — such as authentication requirements, rate-limiting rules, expected status codes, and error response shapes.
• Doc Generation: It formats the captured metadata into standardized formats (e.g., JSON schema, OpenAPI/Swagger specs, or Markdown) for local preview or public developer portals.

Create a Documentation Script (generate-docs.ts)

Create a new file named generate-docs.ts in your workspace to execute The Documenter:

import { ApiEngine } from 'fast-api-tester';
import path from 'node:path';

async function main() {
  console.log('📚 Starting The Documenter...');

  // 1. You can pass a local file path, a raw Swagger string, or a live URL
  const specSource = 'https://petstore.swagger.io/v2/swagger.json';

  // 2. Define your output Markdown file location
  const outputPath = path.resolve('./docs/API_REFERENCE.md');

  try {
    // Call the static Documenter method directly
    const markdownDocs = await ApiEngine.documentOpenApi(specSource, {
      baseUrl: 'https://petstore.swagger.io/v2',
      outputPath: outputPath,
      useFallbackOnAiError: true, // Gracefully falls back to deterministic markdown if AI fails
    });

    console.log(`✅ Successfully generated Markdown documentation!`);
    console.log(`📄 Saved to: ${outputPath}`);
    console.log('\n--- Preview of Generated Docs ---\n');
    console.log(markdownDocs.substring(0, 500) + '\n...\n');
  } catch (error) {
    console.error('❌ Documentation generation failed:', error);
  }
}

main();

Run The Documenter via tsx or ts-node

Execute the TypeScript script directly in your terminal using a runner like tsx (recommended) or ts-node:

# Using npx tsx (fastest, no extra global installation required)
npx tsx generate-docs.ts

Expected Terminal Output:

📚 Starting The Documenter...
✅ Successfully generated Markdown documentation!
📄 Saved to: /your-project/docs/API_REFERENCE.md

--- Preview of Generated Docs ---

# Swagger Petstore (v1.0.6)
This is a sample Petstore server. You can find out more about Swagger at http://swagger.io...

**Base URL:** `https://petstore.swagger.io/v2`

---

## Endpoints

### `POST` /pet
Add a new pet to the store...

🕳️ The BOLA Hunter

The BOLA Hunter (Broken Object Level Authorization): Swaps user IDs in payloads to see if they can access another user's data.

import { ApiEngine } from 'fast-api-tester';

async function runBolaAudit() {
  const api = new ApiEngine();
  api.setBaseUrl('https://api.example.com');

  const bolaResult = await ApiEngine.testBola(api, {
    method: 'GET',
    endpoint: '/api/v1/accounts?userId=123',
    originalId: 123,
    targetId: 456,
    headers: {
      'x-api-key': 'user-123-api-key',
    },
  });

  if (bolaResult.vulnerable) {
    console.error('🚨 BOLA Vulnerability Found:', bolaResult.details);
  } else {
    console.log('✅ Endpoint passed BOLA checks:', bolaResult.details);
  }

  await api.destroy();
}

runBolaAudit();

💉 The SQL Injector

SQL Injector: Attempts to inject modern and classic SQL/NoSQL payloads into every string field.

import { ApiEngine } from 'fast-api-tester';

async function testSqlSecurity() {
  const api = new ApiEngine();
  api.setBaseUrl('https://petstore.swagger.io/#/store/getOrderById');

  console.log('🔍 Running SQL / NoSQL Injection Security Tests...');

  const result = await api.testSqlInjection({
    method: 'POST',
    endpoint: '/api/v1/users/search',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'Bearer YOUR_TEST_TOKEN'
    },
    // The engine automatically detects and injects every string property inside body & query
    body: {
      username: 'john_doe',
      role: 'admin',
      filter: {
        category: 'electronics'
      }
    },
    query: {
      searchQuery: 'laptop'
    }
  });

  if (result.vulnerable) {
    console.error('🚨 VULNERABILITY FOUND!');
    console.error('Vulnerable fields:', result.vulnerableFields);
    console.error('Detailed results:', result.results.filter(r => r.vulnerable));
  } else {
    console.log(`✅ Passed! Tested ${result.testedPayloadsCount} injection payloads with zero vulnerabilities found.`);
  }

  await api.destroy();
}

testSqlSecurity();

🎯 The Rate Limit Sniper

The Rate Limit Sniper: Probes headers and carefully skirts the edge of rate limits to see how quickly the server recovers.

1. Prerequisites & Environment Setup

Set your OpenRouter API key in environment variables to enable AI performance analysis:

export OPENROUTER_API_KEY="your_openrouter_api_key_here"

2. Standard Usage Script (rateLimitTest.ts)

import { ApiEngine } from 'fast-api-tester';

async function runRateLimitSniper() {
  const api = new ApiEngine();
  api.setBaseUrl('https://api.example.com');

  // Run Rate Limit Sniper on a target endpoint
  const result = await api.testRateLimit({
    method: 'GET',
    endpoint: '/v1/data',
    headers: {
      Authorization: 'Bearer YOUR_AUTH_TOKEN',
    },
    maxProbes: 30,         // Probe endpoint up to 30 times rapidly
    probeDelayMs: 20,       // 20ms delay between probing bursts
    useAiAnalysis: true,    // Enable OpenRouter AI analysis of server recovery
  });

  console.log('--- 🎯 Rate Limit Sniper Results ---');
  console.log(`Endpoint: ${result.endpoint}`);
  console.log(`Rate Limit Detected: ${result.rateLimitDetected}`);
  console.log(`HTTP 429 Triggered: ${result.triggered429}`);
  console.log(`Probes Sent: ${result.totalProbesSent}`);
  console.log(`Server Recovery Time: ${result.recoveryTimeMs ? `${result.recoveryTimeMs}ms` : 'N/A'}`);
  console.log(`Detected Headers:`, result.limitHeaderInfo);
  console.log(`Details: ${result.details}`);

  if (result.aiAnalysis) {
    console.log(`\n🤖 OpenRouter AI Analysis:\n${result.aiAnalysis}`);
  }

  await api.destroy();
}

runRateLimitSniper().catch(console.error);

🚀 The Contract Lawyer

The Contract Lawyer: Strictly validates every single response against the OpenAPI/Swagger specification and flags any deviation.

Once users install fast-api-tester from npm (npm install fast-api-tester), they can use The Contract Lawyer in their test scripts in two convenient ways.

1. Set your OpenRouter API Key

Make sure OPENROUTER_API_KEY is exported in your environment:

export OPENROUTER_API_KEY="your-openrouter-api-key"

2. Method 1: Direct Method Chaining (.expectContractCompliance())

Users can chain .expectContractCompliance() on any API request response object:

import { ApiEngine } from 'fast-api-tester';

async function testApiWithContractLawyer() {
  const api = new ApiEngine();
  api.setBaseUrl('http://localhost:3000');

  // Make an API call
  const response = await api.get('/api/users/123');

  // ⚖️ The Contract Lawyer strictly validates the response against openapi.json
  await response.expectContractCompliance('./openapi.json', {
    strictMode: true, // Flag unmapped or extra undocumented properties
  });

  console.log('✅ Response strictly complies with OpenAPI contract!');
  await api.destroy();
}

testApiWithContractLawyer().catch(console.error);

📈 The Load Spiker

The Load Spiker is a load-testing feature that automatically finds an API's exact breaking point instead of you guessing a concurrency number to test.

Here's how it works, briefly:

• Ramps up gradually — starts at a low concurrency (default 5 parallel requests) and multiplies it upward each step (5 → 9 → 16 → 28 → ...) rather than jumping straight to a big number.
• Watches for failure signals — at each step it measures the error rate and p95 latency across a batch of real concurrent requests.
• Confirms the breaking point — when a step crosses your thresholds (default: 20% errors or 5s p95 latency) for a couple of consecutive steps, it locks that in as the breaking concurrency, rather than reacting to a single fluke.
• Backs off and verifies — once it finds the breaking point, it drops back to a safer concurrency (default 70% of the breaking point) and fires a stabilization batch there to confirm the API is actually healthy at that level.
• Optional AI analysis — if you enable it, it sends the full ramp history to OpenRouter for a short natural-language read on the likely bottleneck (connection pool, DB, rate limiter, etc.) and a capacity recommendation.

The end result is a concrete number: "your API breaks around X concurrent requests, and Y is a confirmed-safe ceiling" — which is the kind of thing you'd otherwise get from a much heavier load-testing tool, but built right into fast-api-tester.

import { ApiEngine } from 'fast-api-tester';

async function main() {
  const engine = new ApiEngine().setBaseUrl('https://api.example.com');

  const result = await engine.testLoadSpike({
    method: 'GET',
    endpoint: '/products',
    headers: { authorization: 'Bearer <token>' },

    startConcurrency: 5,        // ramp starts here
    maxConcurrency: 500,        // never exceed this
    rampMultiplier: 1.75,       // 5 -> 9 -> 16 -> 28 -> ...
    errorRateThreshold: 0.2,    // 20% errors = broken
    latencyThresholdMs: 5000,   // p95 > 5s = broken
    confirmationStreak: 2,      // must break twice in a row to confirm
    backoffFactor: 0.7,         // back off to 70% of breaking point
    stabilizationRequests: 15,

    useAiAnalysis: true,        // set OPENROUTER_API_KEY in env
  });

  console.log(result.details);
  console.log('Breaking point:', result.breakingConcurrency);
  console.log('Safe concurrency:', result.recommendedSafeConcurrency);
  console.log(result.steps); // full step-by-step ramp history

  await engine.destroy(); // writes HTML report + closes connection pools
}

main();

Example Output:

📈 [Load Spiker] Ramping up requests on "/products" starting at concurrency=5 (max=500)...
  ⏫ Step 1: firing 10 requests @ concurrency=5...
  🔥 [Load Spiker] Step 1 breached thresholds (errorRate=100.0%, p95=5000ms) — confirming (1/2)...
  ⏫ Step 2: firing 10 requests @ concurrency=9...
  🔥 [Load Spiker] Step 2 breached thresholds (errorRate=100.0%, p95=5000ms) — confirming (2/2)...
⬇️ [Load Spiker] Breaking point ≈ 9 concurrent requests. Backing off to 6 to verify stability...
⚠️ [Load Spiker] Concurrency=6 still unstable — consider a lower ceiling manually.
🔥 Breaking point found at ~9 concurrent requests. Recommended safe operating concurrency: ~6 (not fully confirmed stable — verify manually).
Breaking point: 9
Safe concurrency: 6
[
  {
    step: 1,
    concurrency: 5,
    requestsSent: 10,
    errors: 10,
    errorRate: 1,
    avgLatencyMs: 5000,
    p95LatencyMs: 5000,
    minLatencyMs: 5000,
    maxLatencyMs: 5000,
    broke: true
  },
  {
    step: 2,
    concurrency: 9,
    requestsSent: 10,
    errors: 10,
    errorRate: 1,
    avgLatencyMs: 5000,
    p95LatencyMs: 5000,
    minLatencyMs: 5000,
    maxLatencyMs: 5000,
    broke: true
  }
]

🧬 Schema Drift Detection

APIs change. When an endpoint that used to work suddenly starts returning 400 Bad Request because a new field became required, your test suite doesn't just fail — it heals itself.

Schema Drift Detection watches for 400 responses, sends the error body to an AI model (via OpenRouter) to pinpoint exactly which field the API newly requires, and automatically rewrites your test script's request body to include it — so the next run passes without you touching a line of code.

How it works

1. Detects an HTTP 400 on a request that includes a request body.
2. Sends the error response to OpenRouter, which identifies the missing/invalid required field(s) and suggests a plausible sample value for each.
3. Falls back to deterministic pattern matching (e.g. "role" is required) if no OPENROUTER_API_KEY is set or the AI call fails.
4. Locates the failing request in your script and inserts the missing field(s) directly into its body: { ... } object.
5. Backs up the original file to <filename>.bak before writing, so you can always revert.

Usage

import { ApiEngine } from 'fast-api-tester';

const engine = new ApiEngine().setBaseUrl('https://api.example.com');

const res = await engine.post('/users', {
  email: '[email protected]',
  name: 'Jane Doe',
});

if (res.statusCode === 400) {
  const heal = await res.autoHealSchemaDrift({
    scriptPath: __filename, // or fileURLToPath(import.meta.url) for ESM
    dryRun: false,          // preview only with `true`, no file is written
    backup: true,           // writes users-test.ts.bak before patching
  });

  console.log(heal.details);
  // 🔧 Schema drift healed: added role to the request body for POST /users
  //    in users-test.ts. Re-run your suite to confirm.
}

Options

| Option | Default | Description | | :--- | :--- | :--- | | scriptPath | process.argv[1] | The file to patch. Defaults to the script currently running. | | useAiAnalysis | true (if OPENROUTER_API_KEY is set) | Use AI to identify the missing field(s); falls back to regex matching otherwise. | | dryRun | false | Compute the patch without writing to disk. | | backup | true | Save a .bak copy of the script before patching. | | occurrenceIndex | 0 | Which occurrence of the endpoint to patch, if it appears more than once in the file. | | fieldValueOverrides | undefined | Manually specify sample values instead of relying on AI/heuristic guesses. |

⚠️ This feature rewrites source files on disk. Run with dryRun: true first, and keep your changes under version control so you can always diff or revert an automated patch.

Requires an OpenRouter API key set as OPENROUTER_API_KEY for AI-powered field detection (optional — a regex-based fallback works without one, with reduced accuracy).

🩺 Root Cause Analysis (RCA)

When a test fails, you shouldn't have to dig through raw headers and response dumps to figure out why. The Root Cause Analyzer does that for you: it feeds the full HTTP trace — request headers, request body, response headers, response body, latency, and the assertion failure message — to an AI model (via OpenRouter) and gives you back a plain-English explanation of what went wrong and how to fix it.

How it works

1. Triggers automatically on any failed test (an assertion failure, or an HTTP status ≥ 400) — or force it on a passing response with { force: true }.
2. Sends the complete trace to OpenRouter and asks for a failure category, a summary, a likely cause, and a recommended fix.
3. Falls back to a deterministic heuristic (based on status code and error message — 401/403 → auth failure, 404 → not found, 429 → rate limited, 5xx → server error, SLA breach → latency, etc.) if no OPENROUTER_API_KEY is set or the AI call fails.
4. Prints a formatted summary straight to the console by default, so it's useful even without reading the return value.

Usage

import { ApiEngine } from 'fast-api-tester';

const engine = new ApiEngine().setBaseUrl('https://api.example.com');

const res = await engine.post('/orders', {
  productId: 'abc123',
  quantity: 2,
});

try {
  res.expectStatus(201).expectLatencyUnder(500);
} catch {
  const rca = await res.analyzeRootCause();

  console.log(rca.details);
  // 🩺 Root Cause: [Authentication/Authorization Failure] The API rejected the
  //    request with HTTP 403, indicating insufficient permissions for this account.

  console.log(rca.recommendedFix);
}

Options

| Option | Default | Description | | :--- | :--- | :--- | | useAiAnalysis | true (if OPENROUTER_API_KEY is set) | Use AI to analyze the trace; falls back to a heuristic summary otherwise. | | force | false | Run the analysis even on a passing test. | | maxBodyChars | 2000 | Truncates large request/response bodies before sending them to the AI prompt. | | logToConsole | true | Print the human-readable summary to the console. |

Also available as engine.analyzeRootCause(res, options) / ApiEngine.analyzeRootCause(res, options) if you'd rather call it off the engine than the response.

Requires an OpenRouter API key set as OPENROUTER_API_KEY for AI-powered analysis (optional — a status-code-based heuristic works without one, with reduced accuracy and detail).

📣 Executive Summary Generation

After a test run, fast-api-tester can read the JSON output of that run and use AI (via OpenRouter) to write a concise Slack message and an email-ready executive summary — so you don't have to translate a wall of pass/fail records into something a teammate or manager can skim in ten seconds.

No extra packages required. Posting to Slack is optional and only happens if you provide a webhook URL; the email content is returned to you as plain text so you can send it through whatever mail provider you already use.

How It Works

1. Your tests run as normal using ApiEngine.
2. Call api.generateExecutiveSummary() at the end of the run (alongside api.generateReport()).
3. Under the hood, the engine exports the current run as structured JSON (exportJsonReport() — total/passed/failed/avg latency + every request record + recent run history).
4. That JSON is either sent to OpenRouter (nvidia/nemotron-3-nano-30b-a3b:free) to write a Slack message and an email subject/body, or — if no key is configured or the AI call fails — a clean deterministic template is generated locally instead.
5. If slackWebhookUrl is set, the Slack message is POSTed there directly. Everything can also be written to disk as JSON.

Setup

export OPENROUTER_API_KEY="sk-or-..."   # https://openrouter.ai/keys — free tier available

For Slack delivery, create an Incoming Webhook and pass its URL as slackWebhookUrl:

export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T.../B.../..."

Usage

Instance method — summarize the current run

Call this after your tests, alongside api.generateReport():

import { ApiEngine } from 'fast-api-tester';

const api = new ApiEngine().setBaseUrl('https://petstore.swagger.io/v2');

// Run your tests
await api.get('/pet/findByStatus', { query: { status: 'available' } })
  .then(r => r.expectStatus2xx());

await api.post('/pet', { id: 987654321, name: 'Rex', status: 'available' })
  .then(r => r.expectStatus2xx());

await api.get('/pet/000000000'); // intentional 404 — will appear in the summary

await api.get('/store/inventory')
  .then(r => r.expectStatus2xx());

// Standard HTML report (existing feature, unchanged)
api.generateReport();

// Executive summary
const summary = await api.generateExecutiveSummary({
  apiKey: process.env.OPENROUTER_API_KEY,          // omit to use the deterministic fallback
  companyName: 'Petstore QA',
  audience: 'leadership',                           // 'engineering' | 'leadership' | 'general'
  slackWebhookUrl: process.env.SLACK_WEBHOOK_URL,    // optional — omit to skip posting
  outputPath: 'reports/executive-summary.json',      // optional local copy
});

console.log(summary.slackMessage);
console.log(summary.emailSubject, '\n', summary.emailBody);
console.log('Posted to Slack:', summary.postedToSlack);

await api.destroy();

Static method — summarize a saved JSON file

You don't need a live ApiEngine instance for this — point it at any JSON file, a raw JSON string, or an in-memory object (e.g. one previously written by exportJsonReport()):

import { ApiEngine } from 'fast-api-tester';

// Read a JSON file written by a previous run (e.g. from a CI artifact) and post it to Slack
const summary = await ApiEngine.generateExecutiveSummary('reports/test-run.json', {
  slackWebhookUrl: process.env.SLACK_WEBHOOK_URL,
});

Exporting the JSON test run yourself

generateExecutiveSummary() reads its input from exportJsonReport() under the hood, but you can also export or auto-save that JSON independently — e.g. to hand off to a separate CI step, or to feed the static method above:

// Auto-write reports/test-run.json alongside the existing HTML report + history file
api.configureReport({ jsonOutputPath: 'reports/test-run.json' });

// ...run your tests...

api.generateReport(); // now also writes reports/test-run.json

// or export/write it explicitly at any point without waiting for generateReport():
const runData = api.exportJsonReport('reports/test-run.json');

Sample Output

Slack message (summary.slackMessage, Slack mrkdwn):

⚠️ *Petstore QA Test Run Summary — #4821*
*Pass Rate:* 75.0% (3/4)
*Failed:* 1
*Avg Latency:* 747ms
*Base URL:* https://petstore.swagger.io/v2
*Run Time:* 8/5/2026, 2:14:03 PM

*Top Failures:*
- GET /pet/000000000 → HTTP 404

Email (summary.emailSubject / summary.emailBody):

Subject: ⚠️ Petstore QA API Test Report — #4821 (3/4 passed)

Hi team,

Here is the executive summary for the latest Petstore QA API test run.

Run ID: #4821
Run Time: 8/5/2026, 2:14:03 PM
Base URL: https://petstore.swagger.io/v2

Total Requests: 4
Passed: 3
Failed: 1
Pass Rate: 75.0%
Average Latency: 747ms

Top Failures:
- GET /pet/000000000 → HTTP 404

This summary was generated automatically by Petstore QA.

Regards,
Petstore QA Automated Reporting

If OPENROUTER_API_KEY is not set (or the AI call fails and useFallbackOnAiError is not false), the deterministic template above is generated automatically — no errors, no extra configuration needed.

Options (ExecutiveSummaryOptions)

| Option | Type | Default | Description | | :--- | :--- | :--- | :--- | | apiKey | string | process.env.OPENROUTER_API_KEY | OpenRouter API key for AI generation | | outputPath | string | undefined | Local path to write the generated summary as JSON ({ slackMessage, emailSubject, emailBody, postedToSlack }) | | slackWebhookUrl | string | undefined | Slack Incoming Webhook URL — if set, slackMessage is POSTed there directly | | companyName | string | 'fast-api-tester' | Name used in the summary headline/signature | | audience | 'engineering' \| 'leadership' \| 'general' | 'general' | Tone/detail level of the AI-generated summary | | useFallbackOnAiError | boolean | true | Whether to fall back to the deterministic template if the AI call fails or no key is set |

Return Value (ExecutiveSummaryResult)

{
  slackMessage: string;    // Slack mrkdwn-formatted summary
  emailSubject: string;    // Email subject line
  emailBody:    string;    // Plain-text email body
  postedToSlack: boolean;  // true if slackWebhookUrl was set and the POST succeeded
}

Also available as ApiEngine.generateExecutiveSummary(jsonSource, options) (static) and ExecutiveSummaryGenerator.generateSummary(source, options) if you'd rather call it directly.

Requires an OpenRouter API key set as OPENROUTER_API_KEY for AI-written summaries (optional — a deterministic template works without one, with less narrative detail).

📖 Automated Runbook Generation

When a test run surfaces a failing request — a 503 from a database timeout, a 504 gateway hang, a 401 from an expired token — fast-api-tester automatically drafts a Markdown runbook for each failure. The AI reads the full HTTP trace (method, endpoint, status code, latency, request headers and body) and writes a structured, developer-ready document that explains what likely went wrong and exactly how to fix it.

No extra packages required. If no OpenRouter key is configured, a rich deterministic runbook is generated instead — no network call, no errors.

ℹ️ Status: documented here for reference, but generateRunbooks() is not yet implemented in this copy of apiEngine.ts — only generateExecutiveSummary() / exportJsonReport() above are live. Let me know if you'd like this wired up the same way.

What each runbook contains

Every generated runbook has five sections:

• Error Summary — a table of method, endpoint, status code, latency, and error message
• Likely Root Causes — tailored to the specific error (e.g. connection pool exhaustion for a 503, missing index for a 504, expired credential for a 401)
• Immediate Diagnostic Steps — bash commands the developer can run right now (kubectl logs, psql, redis-cli ping, curl -v, etc.)
• Resolution Steps — numbered, actionable instructions to restore the service
• Prevention — what to add (health checks, circuit breakers, SLA alerts, regression tests) so the failure does not recur

Each runbook is written to a separate .md file in the runbooks/ directory, named after the method, endpoint, and status code — for example runbook-GET--pet-000000000-404.md.

Setup

export OPENROUTER_API_KEY="sk-or-..."   # https://openrouter.ai/keys — free tier available

Usage

Instance method — after a live test run

import { ApiEngine } from 'fast-api-tester';

const api = new ApiEngine().setBaseUrl('https://petstore.swagger.io/v2');

await api.get('/pet/findByStatus', { query: { status: 'available' } })
  .then(r => r.expectStatus2xx());

await api.get('/pet/000000000');          // 404 — will produce a runbook
await api.get('/store/nonexistent');      // 404 — will produce a runbook

api.generateReport();                     // existing HTML report, unchanged

const result = await api.generateRunbooks({
  apiKey:             process.env.OPENROUTER_API_KEY,
  outputDir:          'runbooks',
  environmentContext: 'Node.js 20, PostgreSQL 15 on AWS ECS, Redis 7',
  filterStatusCodes:  [[400, 599]],       // default — all 4xx and 5xx
  useFallbackOnAiError: true,
});

console.log(`${result.totalGenerated} runbook(s) written to runbooks/`);

Static method — against a saved JSON file

// Run this in CI after tests have already completed and saved their records
await ApiEngine.generateRunbooks('.test-history.json', {
  outputDir:          'ci-runbooks',
  environmentContext: 'Staging on GCP Cloud Run',
});

Options

| Option | Type | Default | Description | | :--- | :--- | :--- | :--- | | apiKey | string | process.env.OPENROUTER_API_KEY | OpenRouter API key for AI generation | | model | string | nvidia/nemotron-3-nano-30b-a3b:free | Any OpenRouter model ID | | useFallbackOnAiError | boolean | true | Use deterministic template if AI fails | | outputDir | string | runbooks/ | Directory for generated .md files | | filterStatusCodes | Array<number \| [number, number]> | [[400, 599]] | Status codes or ranges to generate runbooks for | | environmentContext | string | — | Stack description injected into the AI prompt (e.g. "Node.js 20, PostgreSQL 15 on AWS ECS") | | maxBodyChars | number | 2000 | Max request/response body characters sent to the AI |

Return value

{
  totalGenerated: number;      // runbooks generated
  filesWritten:   number;      // files successfully written to disk
  runbooks: Array<{
    method:          HttpMethod;
    endpoint:        string;
    statusCode:      number;
    severity:        'low' | 'medium' | 'high' | 'critical';
    runbookMarkdown: string;   // full Markdown content
    filePath?:       string;   // absolute path of the written file
    aiGenerated:     boolean;  // false = deterministic fallback was used
  }>;
}

📋 HTML Failure Log

fast-api-tester can capture assertion failures and display them directly inside the generated HTML report.

Instead of relying only on terminal warnings or test-runner output, failed assertions can be collected into a dedicated Failure Log panel inside test-report.html.

The panel helps developers and QA engineers quickly review:

• Soft assertion failures
• Hard assertion failures
• Failure timestamps
• Warning and error severity
• The exact validation message
• All failures from the completed API test run

How It Works

When an assertion fails, the related test execution record is marked as failed and its error message is stored for reporting.

Soft assertion failures are automatically passed to the report through an internal failure callback while still being printed to the terminal.

Hard assertions throw an error by default. They can be added to the Failure Log using engine.logFailure() when handled inside a try/catch block.

The generated report displays:

• ⚠️ WARN entries for soft assertion failures
• ❌ ERROR entries for hard assertion failures
• A timestamp for each captured failure
• A dedicated Failure Log panel above the test results

The panel is hidden automatically when the test run contains no captured failures.

Installation

npm install fast-api-tester

Soft Assertion Failures

Soft assertions do not stop the test execution. Their failures are captured automatically and displayed in the HTML report.

import { ApiEngine } from 'fast-api-tester';

const engine = new ApiEngine();

engine.setBaseUrl('https://jsonplaceholder.typicode.com');

const response = await engine.get('/posts/1');

response
  .soft()
  .expectStatus(999)
  .expectBodyToBeArray();

await engine.destroy();

In this example:

• expectStatus(999) fails because the endpoint does not return status 999.
• expectBodyToBeArray() fails because the response body is an object.
• Both failures are recorded.
• Test execution continues because soft assertion mode is enabled.
• Both warnings appear in the Failure Log panel.

After the test finishes, open:

test-report.html

The report will contain a 📋 Failure Log section with both failed assertions.

Engine-Wide Soft Assertion Mode

You can enable soft assertions for every response created by the engine.

import { ApiEngine } from 'fast-api-tester';

const engine = new ApiEngine();

engine
  .setBaseUrl('https://jsonplaceholder.typicode.com')
  .setAssertionMode('soft');

const postResponse = await engine.get('/posts/1');
postResponse.expectStatus(500);

const userResponse = await engine.get('/users/1');
userResponse.expectBodyToBeArray();

const postsResponse = await engine.get('/posts');
postsResponse.expectStatus2xx();

await engine.destroy();

All failed assertions are collected without immediately terminating the test workflow.

Hard Assertion Failures

Hard assertions remain the default behavior. A hard assertion throws an error immediately.

To include a handled hard assertion failure in the HTML Failure Log, catch the error and pass its message to engine.logFailure().

import { ApiEngine } from 'fast-api-tester';

const engine = new ApiEngine();

engine.setBaseUrl('https://jsonplaceholder.typicode.com');

try {
  const response = await engine.get('/posts/1');

  response.expectStatus(500);
} catch (error: unknown) {
  const message =
    error instanceof Error
      ? error.message
      : String(error);

  engine.logFailure(message);
}

await engine.destroy();

The captured failure appears as a red ERROR entry in the HTML report.

Combining Hard and Soft Failures

A realistic test suite can contain both hard and soft validation behavior.

import { ApiEngine } from 'fast-api-tester';

async function runApiTests(): Promise<void> {
  const engine = new ApiEngine();

  engine.setBaseUrl('https://jsonplaceholder.typicode.com');

  try {
    const postResponse = await engine.get('/posts/1');

    postResponse.expectStatus(500);
  } catch (error: unknown) {
    const message =
      error instanceof Error
        ? error.message
        : String(error);

    engine.logFailure(message);
  }

  const userResponse = await engine.get('/users/1');

  userResponse
    .soft()
    .expectStatus(418)
    .expectBodyToBeArray();

  const postsResponse = await engine.get('/posts');

  postsResponse
    .expectStatus2xx()
    .expectBodyToBeArray();

  await engine.destroy();
}

runApiTests().catch((error: unknown) => {
  console.error('API test execution failed:', error);
  process.exitCode = 1;
});

Expected Report Results

| Request | Result | Failure Log | | -------------- | -----: | --------------------------- | | `GET /posts