ai-testcase-generator

Generate Playwright or Vitest API test skeletons from an OpenAPI spec — offline, template-driven, no API key required.

Writing the first draft of an API test suite is repetitive: read the spec, enumerate every endpoint, stub a happy-path test, then stub the obvious negative cases (missing required params, missing body). ai-testcase-generator (atg) does that mechanical first pass for you — deterministically, in milliseconds, with no network calls and no LLM credentials.

You get a runnable skeleton with TODOs where judgement is needed; you fill in the real data and assertions. It is a starting point, not an oracle — and it is honest about that.

Why "AI" with no API key? The v0.1 engine is a deterministic, rule-based generator — it always runs offline and is fully reproducible. An optional LLM mode (to suggest realistic test data and richer assertions) is on the roadmap as an enhancement, never a requirement. The core tool will always work without any key.

Features

• OpenAPI 3 in, tests out — reads JSON or YAML specs.
• Two formats — --format playwright (API request fixture) or --format vitest (fetch).
• Positive + negative cases — a happy path per operation, plus a negative case for every required parameter and required body.
• list command — quickly enumerate the operations in a spec.
• Offline & deterministic — same spec in, same tests out. No network, no key.
• One small dependency (yaml); everything else is standard library.

Installation

# Global CLI
npm install -g ai-testcase-generator

# Or run once
npx ai-testcase-generator list examples/petstore.yaml

Requires Node.js >= 18.

Usage

atg <command> [options]

Commands
  list      <spec>            List the operations found in the spec
  generate  <spec>            Generate test skeletons for every operation

Options
  --format <playwright|vitest>   Output format (default: playwright)
  --out <file>                   Write to a file instead of stdout
  -h, --help                     Show help
  -v, --version                  Show version

Examples

List operations:

$ atg list examples/petstore.yaml
GET    /pets  (listPets)
POST   /pets  (createPet)
GET    /pets/{petId}  (getPet)

3 operation(s).

Generate Playwright tests:

$ atg generate examples/petstore.yaml

import { test, expect } from '@playwright/test';

const BASE_URL = process.env.BASE_URL ?? 'http://localhost:3000';

test.describe('GET /pets/{petId}', () => {
    test('getPet returns success for a valid request', async ({ request }) => {
        // Send a well-formed request with all required inputs.
        // TODO: fill in path/query params, headers, and body as needed
        const response = await request.get(`${BASE_URL}/pets/{petId}`);
        expect(response.status()).toBeGreaterThanOrEqual(200);
        expect(response.status()).toBeLessThanOrEqual(299);
    });

    test('getPet rejects a request missing required path parameter "petId"', async ({ request }) => {
        // Omit the required path parameter "petId".
        // TODO: fill in path/query params, headers, and body as needed
        const response = await request.get(`${BASE_URL}/pets/{petId}`);
        expect(response.status()).toBeGreaterThanOrEqual(400);
        expect(response.status()).toBeLessThanOrEqual(499);
    });
});

Write Vitest tests to a file:

$ atg generate examples/petstore.yaml --format vitest --out tests/api.spec.ts

Use it as a library:

import { parseSpec, extractOperations, deriveCases, renderSuite } from 'ai-testcase-generator';

const ops = extractOperations(parseSpec(specText));
const code = renderSuite(ops.map((op) => ({ op, cases: deriveCases(op) })), 'playwright');

Roadmap

• [ ] v0.2 — generate realistic request bodies from the schema (example/enum aware)
• [ ] v0.3 — boundary cases for typed/constrained parameters (min/max, length, enum)
• [ ] v0.4 — --out <dir> mode: one test file per tag/resource
• [ ] v0.5 — optional --llm enhancement to suggest test data and assertions (key-gated, never required)
• [ ] v1.0 — stable library API + Postman collection input

Contributing

See CONTRIBUTING.md. The whole pipeline is pure functions (parseSpec → extractOperations → deriveCases → renderSuite), so contributions come with fast, deterministic unit tests.

git clone https://github.com/fkhb90/ai-testcase-generator.git
cd ai-testcase-generator
npm install
npm test
npm run dev -- generate examples/petstore.yaml

Related projects

• playwright-test-utils — utilities for the Playwright tests this generates.
• url-filter-analyzer — lint and test ad-block / URL filter lists.

License

MIT © fkhb90