Appliqation Scriptgen

Drafts and verifies an enterprise-grade Playwright script for one test case — and never claims it passes without actually running it.

Point it at a test case that has no canonical automation yet, and it investigates the surrounding context (scenario intent, sibling test flakiness, linked defects, prior execution evidence), writes a real Playwright spec into your repo, and iterates against a real npx playwright test run until it's genuinely green — not until the model says it's green.

Why this exists

Most "AI writes your tests" tools generate a script and hope. This one is judged by a single rule: testRun.ok is only ever true if a real, execFile-reported Playwright process exited 0, and that run happened after the most recent file write. An earlier passing run that predates the latest edit doesn't count — that's a stale result, not a verification. If the generated script never actually ran, the exit code reflects that honestly, every time.

How it works

flowchart TD
    A[test case UUID] --> B[gather context:<br/>scenario, sibling flakiness,<br/>linked defects, evidence]
    B --> C[draft a Playwright spec]
    C --> D[npx playwright test]
    D --> E{passed?}
    E -- no --> F[read the real failure,<br/>revise the spec]
    F --> D
    E -- yes --> G[report: verified: true<br/>+ the written file path]

The context-gathering and drafting methodology itself lives in Appliqation's own appq:automate MCP prompt — this repo is deliberately thin: it just gives that workflow two tool surfaces (read-only Appliqation context tools, and real filesystem + an allowlisted shell) and lets it do the work.

• No appq write tool, no git operation. This agent writes local files only — nothing is synced back to Appliqation and nothing is committed. appliqation-pr-raise handles turning the result into a real PR.
• A hardcoded, non-negotiable shell allowlist. npm init/install -D, npx playwright install/--version/test, node --version, git status/diff — nothing else can run, checked before execution, spawned via execFile with an explicit argv array (never a shell string).

Quick start

npm install -g @appliqation/scriptgen

Create a .env file (in whatever directory you'll run it from) with:

APPQ_API_KEY=your-appliqation-api-key
ANTHROPIC_API_KEY=your-anthropic-key   # or OPENAI_API_KEY — pick one

appliqation-scriptgen generate \
  --test-case-uuid <uuid> \
  --repo-path /path/to/your/checkout

Add --environment <name> if the target repo needs a fresh Playwright config bootstrapped (its baseURL context), --role <name> to authenticate as a specific role (otherwise inferred per-TC automatically), --autotest-run-id <id> to ground the draft in a prior appliqation-autotest run's real execution evidence, and --json/--ci for a structured summary + CI-friendly exit code.

CLI reference

appliqation-scriptgen generate [options]

Required:

| Option | Description | |---|---| | --test-case-uuid <uuid> | Test case UUID to generate a script for. |

Optional:

| Option | Description | |---|---| | --environment <name> | Environment name — its URL (from get_project_settings) is offered as context for the target app's baseURL, only if the target repo needs a Playwright config bootstrapped from scratch. | | --role <name> | Authenticate as this role in the generated script (setupAuth). Omit for per-TC inference from the TC's own tag/name (same mechanism appliqation-autotest uses), or for ungated projects where neither applies. | | --autotest-run-id <id> | A prior appliqation-autotest run for this TC. Passed straight through as appq:automate's autotest_run_id arg — pulls that run's execution evidence and switches to autonomous mode (no confirmation steps, no live-browsing phase). | | --repo-path <path> | Target repo root every file/command tool call is scoped to. Defaults to the current directory. | | --file-path <path> | Local spec file to extend or create. Omit to let the AI discover/propose one. | | --max-turns <n> | Override BUDGET_MAX_TURNS for this run. | | --json | Print a single structured JSON summary on stdout instead of a human-readable report. | | --ci | Shorthand for --json; exit code already reflects the real, execFile-verified outcome either way. |

Configuration

Copy .env.example to .env. Requires APPQ_API_KEY and one of ANTHROPIC_API_KEY/OPENAI_API_KEY.

Running this safely

This is the one agent in the family with a real shell (npm/npx/git, gated by commandGate.ts's hardcoded allowlist) and filesystem surface, driven by the model's own decisions about what to bootstrap and write. run_command's child processes get a scoped, explicit env allowlist, not this process's full environment — but that allowlist deliberately includes @appliqation/automation-sdk's own vars (APPLIQATION_API_KEY, APPQ_AUTH_STATE_DIR, per-project SUT credentials), since the generated spec genuinely needs them to authenticate and report. That's the correct trade-off — the spec has to actually run — but it does mean a real project credential is reachable from whatever npx playwright test executes.

Run this inside a container with an egress allowlist, not directly on a machine with broad network access. This process (and what it spawns) only ever legitimately needs to reach:

• your LLM provider (api.anthropic.com or api.openai.com)
• your configured APPQ_ORIGIN (appq.appliqation.io by default)
• registry.npmjs.org — only while npm install -D actually runs
• Playwright's browser-download host — only while npx playwright install actually runs
• the project's own site under test — wherever the generated spec/login.ts points

Anything else this process (or a spawned npm/npx/git command) tries to reach is unexpected and worth investigating, not routing around.

Development

git clone https://github.com/appliqation/scriptgen.git
cd scriptgen
npm install
cp .env.example .env   # fill in APPQ_API_KEY and one LLM provider key
npm run dev -- generate --test-case-uuid <uuid> --repo-path <path>
npm run typecheck
npm test

See CLAUDE.md for a map of this repo if you're working in it with an AI coding assistant.

License

MIT — see LICENSE.