E2E Test Runner

A powerful, flexible end-to-end testing framework for API and database testing. Write tests in YAML or TypeScript, validate against multiple data stores, and generate comprehensive reports.

Features

• Multi-format tests: Write tests in YAML for simplicity or TypeScript for full control
• Database adapters: Built-in support for PostgreSQL, MongoDB, Redis, and Azure EventHub
• HTTP testing: Native HTTP adapter for REST API testing with JSONPath assertions
• Parallel execution: Run tests concurrently with configurable parallelism
• Multiple reporters: Console, JUnit XML, HTML, and JSON output formats
• Variable interpolation: Dynamic values with built-in functions (${uuid()}, ${now()}, etc.)
• Flexible filtering: Filter tests by tags, priority, or name patterns

Installation

Global Installation (recommended for CLI usage)

npm install -g @liemle3893/e2e-runner

Local Installation (for project integration)

npm install --save-dev @liemle3893/e2e-runner

Using npx (no installation)

npx @liemle3893/e2e-runner --help
npx @liemle3893/e2e-runner run --config ./e2e.config.yaml

Optional Adapters

Install database adapters as needed:

# PostgreSQL
npm install pg

# MongoDB
npm install mongodb

# Redis
npm install ioredis

# Azure EventHub
npm install @azure/event-hubs

Quick Start

1. Initialize Configuration

e2e init

This creates e2e.config.yaml in your current directory.

2. Create a Test

Create my-api.test.yaml:

name: Get Users API
description: Test the users endpoint
tags: [smoke, api]
priority: P1

execute:
  - id: get-users
    adapter: http
    action: GET
    params:
      url: /api/users
    assertions:
      - path: $.status
        operator: equals
        expected: 200
      - path: $.body.data
        operator: isArray

3. Run Tests

e2e run

CLI Commands

e2e run

Execute E2E tests.

# Run all tests
e2e run

# Run with specific config
e2e run -c ./config/e2e.config.yaml

# Run tests from a specific directory
e2e run -d ./tests/integration

# Filter by tags
e2e run --tag smoke --tag regression

# Filter by priority
e2e run --priority P0 --priority P1

# Filter by name pattern
e2e run -g "user*"

# Parallel execution
e2e run -p 4

# Stop on first failure
e2e run --bail

# Dry run (show what would run)
e2e run --dry-run

e2e validate

Validate test file syntax without running.

e2e validate
e2e validate -d ./tests

e2e list

List discovered tests.

e2e list
e2e list --tag smoke

e2e health

Check adapter connectivity.

e2e health
e2e health --adapter postgresql

e2e init

Initialize configuration file.

e2e init

CLI Options

| Option | Short | Description | Default | |--------|-------|-------------|---------| | --config | -c | Config file path | e2e.config.yaml | | --env | -e | Environment name | local | | --test-dir | -d | Test directory | . (current) | | --report-dir | | Report output directory | ./reports | | --verbose | -v | Verbose output | false | | --quiet | -q | Errors only | false | | --parallel | -p | Parallel test count | 1 | | --timeout | -t | Timeout in ms | 30000 | | --retries | -r | Retry count | 0 | | --bail | | Stop on first failure | false | | --grep | -g | Filter by name pattern | | | --tag | | Filter by tag (repeatable) | | | --priority | | Filter by P0/P1/P2/P3 | | | --reporter | | Reporter type (repeatable) | console | | --output | -o | Report output path | | | --dry-run | | Show tests without running | false |

Environment Variables

| Variable | Description | |----------|-------------| | E2E_CONFIG | Config file path | | E2E_ENV | Environment name | | E2E_TEST_DIR | Test directory | | E2E_REPORT_DIR | Report output directory | | E2E_VERBOSE | Enable verbose output (1 or true) | | NO_COLOR | Disable colored output (1 or true) |

Configuration File

version: "1.0"

environments:
  local:
    baseUrl: "http://localhost:3000"
    adapters:
      postgresql:
        connectionString: "${POSTGRESQL_CONNECTION_STRING}"
      mongodb:
        connectionString: "${MONGODB_CONNECTION_STRING}"
      redis:
        connectionString: "${REDIS_CONNECTION_STRING}"

defaults:
  timeout: 30000
  retries: 1
  retryDelay: 1000
  parallel: 4

variables:
  testPrefix: "e2e_test_"

reporters:
  - type: console
    verbose: true
  - type: junit
    output: "./reports/junit.xml"

Writing Tests

YAML Format

name: Create User Flow
description: Test user creation and retrieval
tags: [integration, users]
priority: P1
timeout: 60000

variables:
  testEmail: "test-${uuid()}@example.com"

setup:
  - id: cleanup-existing
    adapter: postgresql
    action: exec
    params:
      query: "DELETE FROM users WHERE email LIKE 'test-%@example.com'"

execute:
  - id: create-user
    adapter: http
    action: POST
    params:
      url: /api/users
      body:
        email: "${testEmail}"
        name: "Test User"
    capture:
      userId: $.body.id
    assertions:
      - path: $.status
        operator: equals
        expected: 201

  - id: get-user
    adapter: http
    action: GET
    params:
      url: "/api/users/${captured.userId}"
    assertions:
      - path: $.body.email
        operator: equals
        expected: "${testEmail}"

verify:
  - id: check-database
    adapter: postgresql
    action: query
    params:
      query: "SELECT * FROM users WHERE id = $1"
      params: ["${captured.userId}"]
    assertions:
      - path: $.rows[0].email
        operator: equals
        expected: "${testEmail}"

teardown:
  - id: delete-user
    adapter: postgresql
    action: exec
    params:
      query: "DELETE FROM users WHERE id = $1"
      params: ["${captured.userId}"]

TypeScript Format

// users.test.ts
import type { TestDefinition } from '@liemle3893/e2e-runner';

export default {
  name: 'Create User Flow',
  tags: ['integration', 'users'],
  priority: 'P1',

  async execute(ctx) {
    // Create user via API
    const createRes = await ctx.http.post('/api/users', {
      email: `test-${ctx.uuid()}@example.com`,
      name: 'Test User',
    });

    ctx.expect(createRes.status).toBe(201);
    ctx.capture('userId', createRes.body.id);

    // Verify in database
    const dbResult = await ctx.postgresql.query(
      'SELECT * FROM users WHERE id = $1',
      [ctx.captured.userId]
    );

    ctx.expect(dbResult.rows).toHaveLength(1);
    ctx.expect(dbResult.rows[0].email).toContain('test-');
  },

  async teardown(ctx) {
    await ctx.postgresql.exec(
      'DELETE FROM users WHERE id = $1',
      [ctx.captured.userId]
    );
  },
} as TestDefinition;

Variable Interpolation

Built-in Functions

| Function | Description | Example | |----------|-------------|---------| | ${uuid()} | Generate UUID v4 | 550e8400-e29b-41d4-a716-446655440000 | | ${now()} | Current timestamp (ISO) | 2024-01-15T10:30:00.000Z | | ${timestamp()} | Unix timestamp (ms) | 1705315800000 | | ${random()} | Random number 0-1 | 0.7234 | | ${randomInt(min, max)} | Random integer | 42 |

Environment Variables

Access environment variables with ${env.VAR_NAME}:

params:
  apiKey: "${env.API_KEY}"

Captured Values

Access values captured from previous steps:

params:
  url: "/api/users/${captured.userId}"

Adapters

HTTP Adapter

- adapter: http
  action: POST  # GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS
  params:
    url: /api/endpoint
    headers:
      Authorization: "Bearer ${token}"
    body:
      key: value
    timeout: 5000

PostgreSQL Adapter

- adapter: postgresql
  action: query  # or 'exec'
  params:
    query: "SELECT * FROM users WHERE id = $1"
    params: [123]

MongoDB Adapter

- adapter: mongodb
  action: find  # insertOne, updateOne, deleteOne, find, etc.
  params:
    collection: users
    filter:
      email: "[email protected]"

Redis Adapter

- adapter: redis
  action: get  # set, del, hgetall, etc.
  params:
    key: "user:123"

Assertions

| Operator | Description | |----------|-------------| | equals | Strict equality | | notEquals | Not equal | | contains | String/array contains | | notContains | Does not contain | | greaterThan | Numeric > | | lessThan | Numeric < | | matches | Regex match | | isArray | Is array type | | isObject | Is object type | | hasLength | Array/string length | | hasProperty | Object has property |

Reporters

Console Reporter

Default output to terminal with colors and progress.

JUnit Reporter

XML output for CI/CD integration:

reporters:
  - type: junit
    output: "./reports/junit.xml"

HTML Reporter

Interactive HTML report:

reporters:
  - type: html
    output: "./reports/report.html"

JSON Reporter

Machine-readable JSON:

reporters:
  - type: json
    output: "./reports/results.json"

Programmatic API

import { runTests, validateTests, listTests, checkHealth } from '@liemle3893/e2e-runner';

// Run tests
const result = await runTests({
  config: './e2e.config.yaml',
  testDir: './tests',
  tag: ['smoke'],
  parallel: 4,
});

console.log(`Passed: ${result.passed}, Failed: ${result.failed}`);

// Validate test syntax
const validation = await validateTests({ testDir: './tests' });
if (!validation.valid) {
  console.error('Validation errors:', validation.errors);
}

// List available tests
const tests = await listTests({ tag: ['integration'] });
tests.forEach(t => console.log(`- ${t.name} (${t.tags.join(', ')})`));

// Check adapter health
const health = await checkHealth();
console.log('All adapters healthy:', health.healthy);

License

MIT