shiplightai

AI-powered end-to-end testing for Playwright. Write tests as YAML with natural-language steps, run them alongside your existing .test.ts files, get self-healing locators and a visual debugger. One package ships both the shiplight CLI and the library you wire into playwright.config.ts.

Full documentation: docs.shiplight.ai

Quick start

Scaffold a new test project in under a minute:

npx shiplightai@latest create ./my-tests
cd my-tests
cp .env.example .env            # then set one of GOOGLE_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY
npm install
npx playwright install chromium
npx shiplight test              # runs the scaffolded starter test

The scaffolder writes package.json, playwright.config.ts, .env.example, .gitignore, and a runnable tests/example.test.yaml that exercises a live site. Open shiplight-report/index.html after the run to see per-step screenshots, videos, and traces.

Or add Shiplight to an existing Playwright project:

npm install shiplightai

CLI

Every command is invoked via npx shiplight <command> from inside a project that has shiplightai installed.

| Command | Purpose | |---|---| | shiplight create <path> | Scaffold a new test project | | shiplight test [args] | Run your YAML and Playwright test suite (forwards all Playwright flags) | | shiplight debug <file> | Launch the interactive visual debugger for a YAML test. The server picks a free port automatically; the chosen URL is printed on startup. Pass --port N for a stable URL (CI, bookmarks). Multiple concurrent shiplight debug invocations work without any port coordination. | | shiplight report [folder] | Regenerate or merge HTML reports | | shiplight transpile [glob] | Transpile YAML tests to .yaml.spec.ts (usually automatic) |

Full reference with flags, options, and troubleshooting: docs.shiplight.ai/local/cli-reference.

Library

Wire up YAML support and the Shiplight reporter in playwright.config.ts:

import { defineConfig, shiplightConfig } from 'shiplightai';

export default defineConfig({
  ...shiplightConfig(),

  testDir: './tests',
  use: {
    headless: true,
    viewport: { width: 1280, height: 720 },
  },
});

shiplightConfig() returns a partial Playwright config that wires in YAML transpilation, auto-discovered .env files, and the HTML reporter. Spread it into your own defineConfig and override whatever you need.

For the YAML statement language, authentication, variables, templates, and custom functions: docs.shiplight.ai/local.

Environment variables

Set at least one LLM provider API key in .env before running tests:

GOOGLE_API_KEY=...
# or ANTHROPIC_API_KEY=sk-ant-...
# or OPENAI_API_KEY=sk-...

The AI model is auto-selected from the first key set. Override with WEB_AGENT_MODEL=<model>.

Shiplight uses an explicit env var allowlist: only the vars it forwards into its internal SDK config are accessible to the agent. Any env var not on the list is invisible to the SDK by design. Full list (OPENAI_BASE_URL, Vertex AI routing, Mailgun keys, etc.): docs.shiplight.ai/local/cli-reference#environment-variables.

Requirements

• Node.js >= 22
• @playwright/test 1.60.0 — declared as a peer dependency and installed automatically by npm install
• Chromium — install on demand with npx playwright install chromium

Links

• Documentation: docs.shiplight.ai
• VS Code extension: github.com/ShiplightAI/vscode-extension
• Cloud platform: shiplight.ai