@gherkle/runner

Test runner agent for Gherkle. Connects to the Gherkle platform over WebSocket, receives Gherkin feature files, executes them against a configured test framework, and streams results back in real time.

Installation

The runner supports two frameworks today. Install the one that matches your test config:

For API tests (framework: 'supertest')

Supertest is bundled with the runner — no extra deps, no browsers:

npm install -g @gherkle/runner
gherkle-runner start --token ghr_xxxxx --url https://runner.gherkle.com

For browser tests (framework: 'playwright')

Playwright is an optional peer dep — install it alongside the runner, then download chromium once:

npm install -g @gherkle/runner @playwright/test playwright-core
npx playwright install chromium
gherkle-runner start --token ghr_xxxxx --url https://runner.gherkle.com

One-shot via npx

# supertest
npx @gherkle/runner start --token ghr_xxxxx --url https://runner.gherkle.com

# playwright (still need peer deps + chromium locally)
npm install @playwright/test playwright-core
npx playwright install chromium
npx @gherkle/runner start --token ghr_xxxxx --url https://runner.gherkle.com

Docker

To run the agent in a container, drop the CLI into any Node 22 image:

FROM mcr.microsoft.com/playwright:v1.58.2-jammy
RUN npm install -g @gherkle/runner
ENTRYPOINT ["gherkle-runner", "start"]

docker build -t my-gherkle-runner .
docker run --rm \
  -e GHERKLE_API_URL=https://runner.gherkle.com \
  -e GHERKLE_TOKEN=ghr_xxxxx \
  my-gherkle-runner

How runs work

When you click Run in Gherkle the server pushes the feature file AND the saved AI-generated step definitions (step_definition_files table) to your runner over the existing WebSocket. The runner materializes a self-contained temp project on disk, forks @cucumber/cucumber, and streams pass/fail results back. No step-definition code needs to exist on the runner machine.

For the BYO workflow (your own step-defs already committed to disk), set stepDefinitionsPath in gherkle-runner.config.js and dispatch with a project that has no saved step-defs — the runner falls back to the legacy Playwright executor against your local files.

Secrets & environment variables

Step definitions often need credentials — a login password, an API key, a token for the app under test. Gherkle never stores these. They live only on the runner host. Set them as environment variables before (or when) you start the runner, and AI-generated step defs read them off the Cucumber world as this.env.YOUR_VAR:

# Shell / secret manager
export LOGIN_PASSWORD=...
export API_KEY=...
gherkle-runner start --token ghr_xxxxx

# Docker
docker run --rm \
  -e GHERKLE_TOKEN=ghr_xxxxx \
  -e LOGIN_PASSWORD=... \
  -e API_KEY=... \
  gherkle-runner

In an AI-generated (cucumber) step def:

When("I sign in", async function () {
  await this.page.fill("#password", this.env.LOGIN_PASSWORD);
});

The runner forwards its host environment into the forked test process, so any variable you export is visible as this.env.*. The only value Gherkle itself injects is BASE_URL (from the project's test config).

The config-file env: {} block is NOT read on this path. It applies only to the legacy BYO-Playwright executor (see Writing Step Definitions). For AI-generated step defs, use host environment variables as shown above.

Leak surface: a secret your test uses can still appear in a failure screenshot or an error message, both of which stream back to Gherkle and are saved in the run history. Avoid logging secret values, and prefer dedicated test accounts over real production credentials.

Quick Start

1. Generate a config file

gherkle-runner init

This creates gherkle-runner.config.js in the current directory.

2. Set your runner token

export GHERKLE_TOKEN=ghr_xxxxx

3. Start the runner

gherkle-runner start

The runner connects to Gherkle and waits for test runs. It automatically reconnects if the connection drops.

CLI Reference

gherkle-runner <command> [options]

start

Start the runner agent and connect to Gherkle.

| Option | Description | | ----------------------------- | ------------------------------------------------------ | | -t, --token <token> | Runner token (overrides GHERKLE_TOKEN env var) | | -n, --name <name> | Runner name | | -u, --url <url> | Gherkle API URL | | -f, --framework <framework> | Test framework: playwright, cypress, or cucumber | | --headless | Run browsers in headless mode (default: true) | | --no-headless | Run browsers with visible UI |

gherkle-runner start --token ghr_xxxxx --name ci-runner --framework playwright

init

Generate a sample gherkle-runner.config.js in the current directory. Fails if one already exists.

gherkle-runner init

status

Print runner version, config file detection, and token status.

gherkle-runner status

Configuration

Configuration is resolved by merging sources in this order (later wins):

1. Built-in defaults
2. Config file
3. Environment variables
4. CLI options

Config File

The runner looks for config files in the working directory, checking these names in order:

• gherkle-runner.config.js
• gherkle-runner.config.mjs
• gherkle-runner.config.ts
• .gherklerc.js
• .gherklerc.json

Example config:

// gherkle-runner.config.js
module.exports = {
  // Connection (required)
  token: process.env.GHERKLE_TOKEN,

  // Runner identification
  name: "my-runner",
  labels: ["local", "dev"],

  // Test framework
  framework: "playwright", // 'playwright' | 'cypress' | 'cucumber'

  // Paths
  featuresPath: "./features",
  stepDefinitionsPath: "./steps",
  supportPath: "./support",

  // Browser configuration (E2E only)
  browsers: ["chromium"],
  headless: true,

  // Timeouts
  testTimeout: 30000,
  stepTimeout: 10000,

  // Artifacts
  screenshots: "on-failure", // 'always' | 'on-failure' | 'never'
  video: "never", // 'always' | 'on-failure' | 'never'

  // Environment
  baseUrl: process.env.BASE_URL || "http://localhost:3000",
  // NOTE: `env` is read only by the legacy BYO-Playwright executor (context.env).
  // AI-generated cucumber step defs read host env vars as this.env.* instead.
  env: {
    API_URL: process.env.API_URL,
  },
};

Environment Variables

| Variable | Maps to | | --------------------- | -------- | | GHERKLE_TOKEN | token | | GHERKLE_API_URL | apiUrl | | GHERKLE_RUNNER_NAME | name |

Config Options Reference

| Option | Type | Default | Description | | --------------------- | ---------- | ------------------------- | ------------------------------------------------------------ | | token | string | required | Runner authentication token | | apiUrl | string | https://app.gherkle.com | Gherkle platform URL | | name | string | gherkle-runner | Display name for this runner | | labels | string[] | [] | Labels for filtering/routing runs to this runner | | framework | string | playwright | Test framework: playwright, cypress, or cucumber | | featuresPath | string | ./features | Directory containing .feature files | | stepDefinitionsPath | string | ./steps | Directory containing step definition files | | supportPath | string | ./support | Directory containing support/helper files | | browsers | string[] | ['chromium'] | Browsers to use: chromium, firefox, webkit | | headless | boolean | true | Run browsers without UI | | testTimeout | number | 30000 | Max duration per test scenario (ms) | | stepTimeout | number | 10000 | Max duration per step (ms) | | screenshots | string | on-failure | When to capture screenshots: always, on-failure, never | | video | string | never | When to record video: always, on-failure, never | | baseUrl | string | undefined | Base URL for the application under test | | env | object | {} | BYO-Playwright executor only — key-value pairs passed as context.env. AI-generated (cucumber) step defs ignore this; set host env vars instead (see Secrets & environment variables). |

Writing Step Definitions

Gherkle has two executors, each with its own step format:

• AI-generated step defs (the default path, dispatched from the app) run through @cucumber/cucumber. They use the Cucumber world — this.page, this.env, this.request — and need no local files. See Cucumber World Format below.
• BYO step defs on local disk run through the legacy Playwright executor. They use the context-argument format documented immediately below, and read env from your config file.

Step definition files (BYO path) are loaded from stepDefinitionsPath. The runner scans recursively for files matching:

• *.steps.ts / *.steps.js
• *.step.ts / *.step.js
• *_steps.ts / *_steps.js

Step File Format

A step file exports a function that receives registration helpers:

// steps/login.steps.js
module.exports = function ({ Given, When, Then }) {
  Given(/the user is on the login page/, async (context) => {
    await context.page.goto(`${context.baseUrl}/login`);
  });

  When(
    /the user enters "(.*)" and "(.*)"/,
    async (context, username, password) => {
      await context.page.fill("#username", username);
      await context.page.fill("#password", password);
      await context.page.click('button[type="submit"]');
    },
  );

  Then(/the user sees the dashboard/, async (context) => {
    await context.page.waitForURL("**/dashboard");
  });
};

Step Context

Every step function receives a StepContext object as its first argument:

| Property | Type | Description | | ----------- | --------------------------- | -------------------------------------------------- | | page | playwright.Page | Playwright Page instance for the current scenario | | context | playwright.BrowserContext | Playwright BrowserContext for the current scenario | | baseUrl | string \| undefined | The configured baseUrl | | env | Record<string, string> | Environment variables from config | | dataTable | string[][] \| undefined | Data table attached to the current step | | docString | string \| undefined | Doc string attached to the current step |

Capture groups from the step pattern regex are passed as additional string arguments after the context.

Cucumber World Format

AI-generated step definitions (the default dispatch path) use the @cucumber/cucumber world instead of a context argument. State lives on this:

const { When } = require("@cucumber/cucumber");

When("I sign in", async function () {
  await this.page.goto(`${this.env.BASE_URL}/login`);
  await this.page.fill("#password", this.env.LOGIN_PASSWORD);
});

| World property | Type | Description | | -------------- | ------------------- | ----------------------------------------------------- | | this.page | playwright.Page | Page instance (playwright framework) | | this.request | supertest agent | HTTP agent bound to BASE_URL (supertest framework) | | this.env | NodeJS.ProcessEnv | The runner host environment — see Secrets & environment variables |

Secrets reach this.env as host environment variables, not through the config-file env: {} block (that block is read only by the BYO Playwright executor above).

Keyword Matching

Steps registered with Given, When, or Then are matched by keyword first. Steps using And or But in the feature file inherit their effective keyword from the preceding step. If no keyword-specific match is found, the runner falls back to matching against all registered steps regardless of keyword.

Gherkin Parser

The runner includes a built-in Gherkin parser (@cucumber/gherkin) that supports:

• Feature descriptions and tags
• Scenarios with Given/When/Then/And/But steps
• Background sections (run before each scenario)
• Scenario Outlines with Examples tables (expanded into individual scenarios)
• Data Tables on steps
• Doc Strings on steps
• Multiple Examples blocks per outline
• Tags at feature, scenario, and examples level

Architecture

bin/gherkle-runner.js     CLI entrypoint (commander)
src/
  index.ts                Public API exports
  types.ts                Type definitions and WebSocket message types
  config.ts               Config loading, merging, validation
  agent.ts                GherkleAgent - WebSocket client, run orchestration
  step-registry.ts        Step definition registry and matching
  gherkin-parser.ts       Gherkin AST parsing
  frameworks/
    playwright.ts         Playwright executor (browser launch, scenario execution)
    cucumber-subprocess.ts  Cucumber.js executor: forks @cucumber/cucumber and maps
                            its result envelopes (gherkinDocument/pickle/testCase)
                            into step text, keyword, and pass/fail results
  workers/
    cucumber-worker.ts      Worker entrypoint run inside the forked cucumber subprocess

How It Works

1. Connect -- GherkleAgent opens a WebSocket connection to the Gherkle platform, authenticating with the runner token.
2. Wait -- The agent sends heartbeats every 30 seconds and waits for an execute message.
3. Execute -- When a run is dispatched, the agent parses the received Gherkin feature files, loads step definitions, launches a browser, and runs each scenario.
4. Stream -- Step and scenario results are sent back to Gherkle in real time over the WebSocket (scenario:started, step:completed, scenario:completed).
5. Artifacts -- On step failure (when screenshots is not never), a full-page screenshot is captured and reported as an artifact. Screenshots are saved to .gherkle-artifacts/ in the working directory.
6. Complete -- A run:completed message with the full summary is sent when all features finish.

The agent automatically reconnects after 5 seconds if the WebSocket connection drops.

Programmatic API

The package exports the agent and config loader for use in custom tooling:

import { GherkleAgent, loadConfig } from "@gherkle/runner";

const config = loadConfig({
  token: "ghr_xxxxx",
  framework: "playwright",
});

const agent = new GherkleAgent(config);

// Handle shutdown
process.on("SIGINT", () => agent.stop());

await agent.start();

Exports

| Export | Description | | ------------------------ | ------------------------------------------------------------------------------------ | | GherkleAgent | Main agent class. Call start() to connect, stop() to disconnect. | | loadConfig(overrides?) | Load and merge config from file, env, and provided overrides. Returns AgentConfig. | | AgentConfig | Configuration interface | | RunnerStatus | 'online' \| 'offline' \| 'busy' \| 'error' | | TestStatus | 'pending' \| 'running' \| 'passed' \| 'failed' \| 'skipped' | | Framework | 'playwright' \| 'cypress' \| 'cucumber' | | StepResult | Result of a single step execution | | ScenarioResult | Result of a scenario including all steps and artifacts | | RunSummary | Aggregate pass/fail/skip counts and duration | | FeatureContent | Feature file id, name, path, and raw content | | TestConfig | Framework and path configuration for a test run | | ArtifactInfo | Metadata for a captured artifact (screenshot, video, trace, log) | | RunnerMessage | Union type of all messages sent from runner to server | | ServerMessage | Union type of all messages sent from server to runner |

Docker

Two Dockerfiles are provided depending on your workload:

| File | Base image | Size | Use case | | --------------------- | ------------------------------ | ------- | ----------------------------------------------------------------- | | Dockerfile | node:22-slim | ~200 MB | Step definitions, assertions, API tests, any non-browser workload | | Dockerfile.browsers | mcr.microsoft.com/playwright | ~2 GB | E2E tests requiring Chromium, Firefox, or WebKit |

Use the default Dockerfile unless your step definitions launch a browser.

Build

Both Dockerfiles must be built from the monorepo root so the workspace lockfile is available:

# Standard image (step definitions + assertions, no browsers)
docker build -t gherkle-runner -f packages/runner/Dockerfile .

# With Playwright browsers for E2E
docker build -t gherkle-runner-browsers -f packages/runner/Dockerfile.browsers .

Run

docker run --rm \
  -e GHERKLE_TOKEN=ghr_xxxxx \
  gherkle-runner

Mount step definitions and features

Bind-mount your project directories so the runner can find your test code:

docker run --rm \
  -e GHERKLE_TOKEN=ghr_xxxxx \
  -v $(pwd)/features:/app/features \
  -v $(pwd)/steps:/app/steps \
  -v $(pwd)/support:/app/support \
  gherkle-runner

Pass CLI options

Arguments after the image name are forwarded to gherkle-runner start:

docker run --rm \
  -e GHERKLE_TOKEN=ghr_xxxxx \
  gherkle-runner start --name ci-runner --framework cucumber

Use a config file

Mount a config file into /app:

docker run --rm \
  -v $(pwd)/gherkle-runner.config.js:/app/gherkle-runner.config.js \
  -v $(pwd)/features:/app/features \
  -v $(pwd)/steps:/app/steps \
  gherkle-runner

Docker Compose

services:
  # Headless step execution (API tests, assertions, etc.)
  gherkle-runner:
    build:
      context: .
      dockerfile: packages/runner/Dockerfile
    environment:
      - GHERKLE_TOKEN=${GHERKLE_TOKEN}
      - GHERKLE_RUNNER_NAME=docker-runner
    volumes:
      - ./features:/app/features
      - ./steps:/app/steps
      - ./support:/app/support
    restart: unless-stopped

  # E2E variant with Playwright browsers
  gherkle-runner-e2e:
    build:
      context: .
      dockerfile: packages/runner/Dockerfile.browsers
    environment:
      - GHERKLE_TOKEN=${GHERKLE_TOKEN}
      - GHERKLE_RUNNER_NAME=docker-e2e-runner
    volumes:
      - ./features:/app/features
      - ./steps:/app/steps
      - ./support:/app/support
    restart: unless-stopped

Environment variables

All configuration environment variables work inside the container. Both images run as a non-root user (gherkle / pwuser) by default.

Requirements

• Node.js >= 18
• Playwright browsers installed (for E2E execution): npx playwright install

Development

# Build
npm run build

# Watch mode
npm run dev

# Run tests
npm test

# Run tests in watch mode
npm run test:watch

# Lint
npm run lint

License

MIT