SmokeQA

A minimal HTTP load tester driven by JSON plans, similar in spirit to JMeter-style configuration. It allows declaring behavior, load shape, captures, assertions, and report formats in JSON files without changing the runner's code for each new site.

Features

• JSON-based Plans: Define complex load testing scenarios using a simple JSON structure.
• Load Shaping: Configure the number of threads, ramp-up time, and test duration or number of iterations.
• HTTP Flows: Define a sequence of HTTP steps, including captures and optional proxies.
• Assertions: Validate response bodies using regex and group comparisons.
• Flexible Reporting: Specify which artifacts to generate, including JTL, JSON, and HTML reports.

Setup

Local Installation

1. Clone the repository.
2. Install dependencies:

   npm install

3. Build the project:

   npm run build

Requires Node.js 20+. After building, you can run the tool using node dist/cli.js ... or npm run smokeqa -- ....

Global Installation

You can install SmokeQA globally from npm or from a local tarball.

From NPM

To install from the npm registry, run:

npm install -g smokeqa

This will make the smokeqa executable available in your system's PATH.

From a Tarball

To create a tarball and install from it:

npm pack
npm install -g ./smokeqa-1.0.0.tgz

Publishing

To publish the package to the npm registry:

1. Make sure you are logged in to npm (npm login).
2. Ensure the package name in package.json is unique.
3. Run the following commands:

   npm run build
   npm publish

The prepublishOnly script will compile the TypeScript code before publishing.

Usage

To run a load test, use the run command with a JSON plan:

npm run smokeqa -- run --config ./examples/sample-plan.json --out-dir ./reports/run1

CLI Reference

| Flag | Description | | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --config path | Required. Path to the JSON plan. | | --out-dir dir | Output directory for report files. Default: ./reports/run-<ISO-timestamp>/. If dir already exists, it writes to dir-1, dir-2, etc. | | --verbose | Prints one line to stderr per HTTP sample (OK / FAIL). Disables the live spinner / multi-line progress UI. | | --plain | No live progress widgets on stderr; emits a textual status line about every three seconds unless --verbose is on. Useful when stderr is not a TTY or the spinner breaks your terminal. | | --http-trace-log [file] | Troubleshooting: per-hop plaintext log (manual redirect = one hop): cookie jar snapshot, Cookie: as sent, headers + body, Set-Cookie, response body (capped), bash + PowerShell curl replays. Includes wall-clock request/response timestamps per hop. | | --http-trace-mode combined|per-thread | Combined (default): one log file shared by all threads (writes serialized). Per-thread: one log file per SmokeQA thread index (http-trace-1.log, …). Overrides plan httpTrace.fileMode. | | --http-trace-timezone IANA | Wall-clock timestamps in traces (e.g. Europe/Madrid). Overrides httpTrace.timeZone in JSON. Default UTC if unset. | | --http-trace-timestamp-style iso_offset|epoch_ms | iso_offset: local ISO-like wall time plus offset suffix. epoch_ms: numeric Unix ms only. Overrides httpTrace.timestampStyle. Ignored when a pattern is set. | | --http-trace-timestamp-pattern str | Strftime-lite pattern (CLI wins over httpTrace.timestampPattern). Tokens supported: %% %Y %y %m %d %H %M %S %3f (ms)** %f %z (+/-offset). | | --html-report-mode single|split | single (default): one HTML file. split: three files: report-head.html, report-errors.html, and report-asserts.html. Overrides plan report.htmlReportMode. | | --theme name | Color theme for the live TUI: opencode, tokyonight, dracula, monokai. Default: opencode. |

TUI Keyboard Controls

When running in an interactive terminal (TTY), you can control the test live:

| Key | Action | | --------- | ------------------------------------------- | | p / | Pause / Resume the test | | q | Quit (abort the test) | | t | Cycle to the next color theme |

The UI shows the available keys at the bottom when you press t to change themes.

Environment Variables

| Variable | Effect | | ------------------------------ | ---------------------------------------------------------------------------------------------------------------- | | ${UPPER_SNAKE_CASE} in JSON | Substituted from process.env when the plan is loaded (e.g., for credentials in CI). | | SMOKEQA_LIVE_SINGLE_LINE=1 | Single-line progress bar instead of the multi-line block. | | SMOKEQA_DEBUG=1 | Extra stderr debug from the fetch layer (not a full HTTP trace; use --http-trace-log for that). | | SMOKEQA_THEME name | Set the default color theme: opencode, tokyonight, dracula, monokai. |

JSON Plan Structure

A JSON plan defines the entire load test. Here are the main sections:

Root Fields

| Field | Notes | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------- | | name | Shown on reports and aggregates. | | baseUrl, steps[] | Each step: label, method, path (/ fragment or absolute URL). | | userAgent | Optional User-Agent string for every step. | | defaultHeaders | Merged before each step’s headers. | | threads, rampUpSeconds | Concurrency and stagger between threads. | | durationMinutes or iterations | Exactly one must be provided — a duration-based run vs. fixed loops per thread. | | timeoutMs | Per-request timeout (default is large). | | proxyHttp | Optional proxy URL for fetch. | | httpTrace | Optional defaults for --http-trace-log. |

report field

The report object defines the output contract.

| Property | Description | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | report.formats | Non-empty array of identifiers: jtl, json, html. | | report.files | Optional basename overrides per format. Defaults: results.jtl, report.json, report.html. | | report.streamingFiles | Optional NDJSON basenames for streaming samples. Defaults: failures.ndjsonl, session-checks.ndjsonl. | | report.correlation | Optional tracing ID on every sample. Use exactly one of sourceVar or cookieName. | | report.htmlReportMode | Optional report mode. single (default) or split. |

users and expect

| Field | Notes | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | username, password | Credentials, interpolated as ${username} and ${password} in step bodies. | | expect | Optional object { "yourKey": "expectedValue" }. Keys are arbitrary; assertions reference them with compareCapturedGroupToExpectKey. |

Captures and Assertions

• Captures: On a step, use captures with a regex on the response body. Capturing group 1 is stored as vars[name] for later use.
• Assertions: Steps may include an assert array. Each assertion has a type, target, and pattern. You can also compare a captured group to a value in the expect object.

Runner Behaviour

• Iterations: Each iteration starts with a fresh cookie jar. Steps within an iteration share the same cookie jar.
• Threads: Threads iterate through the plan.users array.
• Stopping: The test stops when durationMinutes elapses or all threads complete their iterations.
• Failures: A failure in any step skips the rest of that iteration.
• Exit Code: The process exits with a non-zero code if any sample is unsuccessful.

Sample Plans

• examples/sample-plan.json: A generic minimal plan included in the repo.
• configs/: This directory is in .gitignore and is a good place to store your own plans.

| Pattern | Typical Use | | ---------------------------- | ---------------------------------------------------------------------------------------------------- | | *-basic.json | A login journey (capture CSRF, POST credentials, etc.) with optional assertions. | | *-session-validation.json | The same flow with regex assertions that compare captured values against users[].expect. |

Agent Skills

This repository includes agent skills to help automate tasks. The main skill is smokeqa-cli, located at .cursor/skills/smokeqa-cli/SKILL.md. This skill is useful for running smokeqa, authoring JSON plans, and other related tasks.

For more details on how to use and configure agent skills for different tools like Cursor, GitHub Copilot, Claude Code, and OpenCode, please refer to the SKILL.md file and the documentation of the respective tools.

Contributing

| Area | Location | | --------------------------- | ---------------------------------------------------------------------- | | Agent Skills | .cursor/skills/ | | CLI and Tracing | src/cli.ts, src/engine/http-trace-run.ts, src/engine/http-trace.ts | | Configuration Loading | src/config/load-config.ts | | Plan Validation | src/config/validate-config.ts | | Reporting | src/reporting/ | | Cookies and Redirects | src/engine/cookies.ts | | HTTP Steps and Assertions | src/engine/exec-steps.ts | | Scheduler | src/engine/runner.ts |