playwright-openapi-contract

Generates Playwright API contract tests from an OpenAPI specification, so the tests that prove your API still matches its spec are derived from the spec instead of hand written and left to rot.

API specs and real API behaviour drift apart. Writing contract tests by hand for hundreds of endpoints, across several environments and API versions, is slow, and nobody keeps them up to date. This tool generates them from the spec you already have, and each generated test calls the endpoint and checks the response status, content type, and body against the schema the spec declares.

Install

npm install --save-dev playwright-openapi-contract @playwright/test

Requires Node 20 or later. @playwright/test is a peer dependency.

The generated tests are ES modules, so your project needs "type": "module" in its package.json. contract generate warns if it does not.

Quickstart

1. Create a config.

npx contract init

That writes contract.config.json. Point specs[].file at your OpenAPI document and set a baseUrl for each environment you test against:

{
  "outDir": "tests/contract",
  "specs": [{ "name": "v1", "file": "./openapi.yaml" }],
  "environments": {
    "dev": { "baseUrl": "https://dev.api.example.com" }
  },
  "auth": { "type": "none" },
  "fixtures": {},
  "skip": []
}

2. Generate the tests.

npx contract generate

v1  5 files, 6 tests, 3 skipped
  tests/contract/v1/__config__.json
  tests/contract/v1/__spec__.json
  tests/contract/v1/default.spec.ts
  tests/contract/v1/pets.spec.ts
  tests/contract/v1/stores.spec.ts
  skipped  createPet POST /pets            (no fixture)
  skipped  deletePet DELETE /pets/{petId}  (no fixture)
  skipped  getPetById GET /pets/{petId}    (missing path parameter)

Wrote 5 files to tests/contract

3. Fill in fixtures for anything that was skipped.

A POST needs a body, and a path parameter needs a value. Both come from fixtures, keyed by operationId:

{
  "fixtures": {
    "createPet": { "body": { "name": "fixture-pet", "tag": "dog" } },
    "getPetById": { "pathParams": { "petId": "1" } },
    "deletePet": { "pathParams": { "petId": "1" } }
  }
}

Run npx contract generate again and those tests become live.

4. Run them with Playwright.

CONTRACT_ENV=dev npx playwright test tests/contract

Before and after

Hand written, for one endpoint:

import { test, expect } from "@playwright/test";

test("GET /pets returns a valid pet list", async ({ request }) => {
  const response = await request.get("https://dev.api.example.com/pets?limit=10", {
    headers: { authorization: `Bearer ${process.env.API_TOKEN}` },
  });

  expect(response.status()).toBe(200);
  expect(response.headers()["content-type"]).toContain("application/json");

  const body = await response.json();
  expect(Array.isArray(body.data)).toBe(true);
  for (const pet of body.data) {
    expect(typeof pet.id).toBe("number");
    expect(typeof pet.name).toBe("string");
    expect(pet.nickname === null || typeof pet.nickname === "string").toBe(true);
    expect(Date.parse(pet.createdAt)).not.toBeNaN();
  }
});

That checks a handful of fields, hardcodes one environment, and has to be updated by hand whenever the schema changes. The generated equivalent validates the whole schema, every field, every nested object, for every endpoint:

// Generated by playwright-openapi-contract. Do not edit.
import { test } from "@playwright/test";
import { contract, type ContractConfig } from "playwright-openapi-contract/runtime";
import config from "./__config__.json" with { type: "json" };
import spec from "./__spec__.json" with { type: "json" };

const api = contract(spec, config as ContractConfig);

test.describe("pets", () => {
  test("listPets GET /pets", async ({ request }) => {
    const response = await api.call(request, "listPets");
    await api.expectValid(response, "listPets");
  });
});

Configuration

contract.config.json, found by walking up from the current directory.

| Key | Type | Required | Description | | --- | --- | --- | --- | | outDir | string | yes | Where generated tests go, relative to the config file. | | specs | array | yes | The specs to generate from. At least one. | | specs[].name | string | yes | Output subdirectory name, and the value for --spec. Must be unique. | | specs[].file | string | yes | Path to the OpenAPI document, relative to the config file. YAML or JSON. | | environments | object | yes | Named environments. At least one. | | environments.<name>.baseUrl | string | yes | Origin that requests are sent to. | | auth | object | no | Defaults to { "type": "none" }. | | auth.type | "none" | "bearer" | "header" | no | How the credential is attached. | | auth.tokenEnvVar | string | for bearer and header | Environment variable holding the token. Read at run time, never stored. | | auth.headerName | string | for header | Header the token is sent in, for example X-Api-Key. | | fixtures | object | no | Request data, keyed by operationId. | | fixtures.<id>.body | any | no | JSON request body, sent with the content type the spec declares. | | fixtures.<id>.pathParams | object of strings | no | Values for {placeholders} in the path. | | fixtures.<id>.queryParams | object of strings | no | Query string values. Overrides any example in the spec. | | skip | array of strings | no | operationIds to emit as skipped. |

Which operations get a live test

The generator never emits a test that would fail for a reason unrelated to the contract. A suite that is red because of setup is worthless.

| Case | Result | | --- | --- | | get, head, options | Live test. | | post, put, patch, delete with a fixtures entry | Live test. | | post, put, patch, delete without one | test.skip, titled (no fixture). | | Listed in skip | test.skip, titled (skipped by config). | | Required path parameter with no fixture and no example in the spec | test.skip, titled (missing path parameter). |

Path and query parameter values come from fixtures first, then from an example in the spec, and are otherwise left out.

CLI

contract init                                     write a starter config, failing if one exists
contract generate [--config <path>] [--spec <name>] [--dry-run]

--spec limits generation to one spec by name. --dry-run prints the files that would be written without writing them. Any error exits 1 with a plain message and no stack trace.

Multiple environments and multiple versions

One generation run serves every environment. The base URL is resolved at run time from CONTRACT_ENV, which defaults to the first environment declared, so the same generated files run against dev, sit, and prod without regenerating:

{
  "environments": {
    "dev": { "baseUrl": "https://dev.api.example.com" },
    "sit": { "baseUrl": "https://sit.api.example.com" },
    "prod": { "baseUrl": "https://api.example.com" }
  }
}

CONTRACT_ENV=sit npx playwright test tests/contract

Multiple API versions are separate entries in specs, each generated into its own subdirectory of outDir:

{
  "outDir": "tests/contract",
  "specs": [
    { "name": "v3", "file": "./specs/api-v3.yaml" },
    { "name": "v4", "file": "./specs/api-v4.yaml" }
  ]
}

tests/contract/v3/pets.spec.ts
tests/contract/v4/pets.spec.ts

Both OpenAPI 3.0 and 3.1 are supported, and they can be mixed in one config. The version is read from the openapi field, 3.0 schemas are normalized to JSON Schema draft-07, and 3.1 schemas are validated as 2020-12.

When a contract breaks

Contract violation: listPets GET /pets
  Environment: sit
  Status: 200
  /data/0/createdAt  must match format "date-time"  (got "not-a-date")
  /data/1            must have required property 'id'

Every error in the response is listed, not just the first, with the JSON pointer to the field, what was expected, and what arrived.

Programmatic API

The package entry point exports the pieces the CLI is built from, for anyone who wants to generate tests from a script rather than the command line:

import { buildOperations, loadSpec, renderSpec } from "playwright-openapi-contract";

Generated tests import from playwright-openapi-contract/runtime, which exports contract(spec, config). The returned object has three methods:

• call(request, operationId) builds the URL from the base URL, path parameters, and query parameters, attaches auth headers and the fixture body, sends the request, and returns the Playwright APIResponse.
• expectValid(response, operationId) asserts the status is declared in the spec, the content type matches, and the body validates against the schema for that status. It is async, so await it.
• url(operationId) returns the resolved URL, which is useful in hand written tests alongside the generated ones.

Non-goals

This tool does one thing. It will not grow to cover:

• Contract testing for GraphQL or gRPC
• Mock server generation
• Spec diffing or breaking change detection
• Load or performance testing
• Generating test data from schemas, beyond the fixtures declared in config
• A web UI or dashboard
• Swagger 2.0
• Test frameworks other than Playwright

License

MIT