Dramaturge

Autonomous QA testing for web applications. Point Dramaturge at your app and it will explore, test, and report issues—no test scripts required.

Quick Start

# Install
npm install dramaturge

# Always run this after first install to sanity-check prerequisites.
# If Playwright's Chromium browser is missing, doctor will prompt you and can install it automatically.
npx dramaturge doctor

# If you're in CI/Docker (non-interactive) or prompts are disabled, install Chromium manually:
# npx playwright install chromium

# Generate config
npx dramaturge auto-config

# Set API key (choose one provider)
export ANTHROPIC_API_KEY="your-key-here"
# See LLM Providers section for other options

# Run
npx dramaturge --config dramaturge.config.json

What It Does

Dramaturge uses LLM-driven browser agents to autonomously test web applications:

• Explores — Navigates links, fills forms, tests workflows
• Finds bugs — Console errors, broken pages, network failures, validation issues
• Checks accessibility — Runs axe-core on every page (WCAG compliance)
• Tests APIs — Validates contracts, auth boundaries, error responses
• Security testing — OWASP scenarios, injection attacks (opt-in)
• Visual regression — Pixel-diff comparison (opt-in)
• Provides evidence — Screenshots, reproduction steps, network traces

No test scripts. No brittle selectors. Works with any web framework.

Configuration

Minimal example:

{
  "targetUrl": "https://your-app.example.com",
  "appDescription": "Your application's purpose",
  "auth": {
    "type": "interactive",
    "loginUrl": "/login",
    "successIndicator": "selector:[data-testid='dashboard']"
  },
  "models": {
    "planner": "anthropic/claude-sonnet-4-6",
    "worker": "anthropic/claude-haiku-4-5"
  }
}

Auth strategies: interactive (manual login), form (automated), oauth-redirect (multi-step), stored-state (reuse session), none (public). See Authentication Guide for details.

Capture auth state:

npx dramaturge auth capture --profile user

For full options, see dramaturge.config.example.json or Configuration Reference.

LLM Providers

Dramaturge supports multiple providers via model-string prefixes (e.g., anthropic/claude-sonnet-4-6). Omitting the prefix defaults to Anthropic.

| Prefix | Provider | Environment Variables | |--------|----------|----------------------| | anthropic/… | Anthropic | ANTHROPIC_API_KEY | | openai/… | OpenAI | OPENAI_API_KEY, OPENAI_BASE_URL (optional) | | google/… | Google Generative AI | GOOGLE_GENERATIVE_AI_API_KEY | | azure/… | Azure AI Foundry | AZURE_AI_ENDPOINT, AZURE_AI_API_KEY | | openrouter/… | OpenRouter | OPENROUTER_API_KEY | | github/… | GitHub Models | GITHUB_TOKEN | | ollama/… | Ollama | OLLAMA_BASE_URL, optional: OLLAMA_API_KEY | | custom/… | OpenAI-compatible | OPENAI_COMPATIBLE_BASE_URL, optional: OPENAI_COMPATIBLE_API_KEY |

Agent modes: "dom" (DOM inspection, portable) or "cua" (computer-use, typically works best with a vision-capable model).

Note: AWS Bedrock, Cohere, and Mistral native APIs require OpenAI-compatible proxies via custom/….

Confirming Fixes

After fixing a bug, replay saved actions to verify:

# Confirm one finding from latest report
npx dramaturge confirm --finding BUG-0042

# Confirm all major+ findings from specific report
npx dramaturge confirm --severity major+ --from-report ./dramaturge-reports/2026-05-20T18-46-40

# Confirm all findings
npx dramaturge confirm --all

Exit codes: 0 = all fixed, 1 = issues remain, 2 = cannot confirm, 3 = needs review.

Building Regression Tests

Promote findings to durable Playwright specs:

# Show promotable findings
npx dramaturge regress list

# Preview generated spec
npx dramaturge regress promote BUG-0042 --dry-run

# Write spec to ./tests/dramaturge
npx dramaturge regress promote BUG-0042

Quality scores consider URL context, actions, evidence, screenshots, and confidence. Only findings with replay actions and clear expected/actual differences are promotable.

CI/CD Integration

Add to .github/workflows/qa.yml:

- uses: aram10/[email protected]
  with:
    config: dramaturge.config.json
    anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    fail-on-severity: major

See GitHub Action Reference for all options.

Documentation

• Configuration Reference — Full config options
• Authentication Guide — All auth strategies with examples
• GitHub Action Reference — CI/CD integration details
• Troubleshooting — Common issues and solutions
• Development Guide — Contributing to Dramaturge

Configuration Reference

Core Settings

{
  "targetUrl": "https://your-app.example.com",
  "appDescription": "What your app does and its main features"
}

Models

{
  "models": {
    "planner": "anthropic/claude-sonnet-4-6",
    "worker": "anthropic/claude-haiku-4-5",
    "browserOps": "anthropic/claude-sonnet-4-6",
    "agentMode": "dom",
    "agentModes": {
      "navigation": "dom",
      "form": "dom",
      "crud": "dom",
      "adversarial": "dom"
    }
  }
}

• planner — Task planning (use smarter model)
• worker — Execution (use faster/cheaper model)
• browserOps — Browser agent runtime (defaults to planner)
• agentMode — "dom" (faster, cheaper) or "cua" (sees viewport)
• agentModes — Per-worker overrides

Budget & Exploration

{
  "budget": {
    "globalTimeLimitSeconds": 900,
    "maxStepsPerTask": 40,
    "maxStateNodes": 50
  }
}

Output

{
  "output": {
    "dir": "./dramaturge-reports",
    "format": "markdown",
    "screenshots": true
  }
}

Formats: "markdown", "json", or "both"

Optional Features

{
  "apiTesting": {
    "enabled": true,
    "maxEndpointsPerNode": 4,
    "unauthenticatedProbes": true
  }
}

{
  "adversarial": {
    "enabled": true,
    "safeMode": true
  }
}

{
  "visualRegression": {
    "enabled": true,
    "baselineDir": "./.dramaturge/visual-baselines",
    "diffPixelRatioThreshold": 0.01
  }
}

{
  "memory": {
    "enabled": true,
    "dir": "./.dramaturge",
    "warmStart": true
  }
}

See dramaturge.config.example.json for complete schema.

Authentication Guide

Log in manually once. Dramaturge captures and reuses the session.

{
  "auth": {
    "type": "interactive",
    "loginUrl": "/login",
    "successIndicator": "selector:[data-testid='user-menu']",
    "stateFile": "./.dramaturge-state/user.json",
    "manualTimeoutSeconds": 120
  }
}

Provide credentials and selectors for automated login.

{
  "auth": {
    "type": "form",
    "loginUrl": "/login",
    "fields": [
      { "selector": "input[name='email']", "value": "${TEST_USER_EMAIL}" },
      { "selector": "input[name='password']", "value": "${TEST_USER_PASSWORD}", "secret": true }
    ],
    "submit": { "selector": "button[type='submit']" },
    "successIndicator": "selector:[data-testid='user-menu']"
  }
}

Script multi-step IdP flows.

{
  "auth": {
    "type": "oauth-redirect",
    "loginUrl": "/login",
    "steps": [
      { "type": "click", "selector": "button[data-provider='google']" },
      { "type": "fill", "selector": "input[type='email']", "value": "${TEST_USER_EMAIL}" },
      { "type": "click", "selector": "input[type='submit']" },
      { "type": "fill", "selector": "input[type='password']", "value": "${TEST_USER_PASSWORD}", "secret": true },
      { "type": "click", "selector": "input[type='submit']" }
    ],
    "successIndicator": "selector:[data-testid='user-menu']"
  }
}

Capture state once with dramaturge auth capture, then reuse:

npx dramaturge auth capture --url https://your-app.example.com/login --profile user
# or from config: npx dramaturge auth capture --config dramaturge.config.json --profile user

{
  "auth": {
    "type": "stored-state",
    "stateFile": "./.dramaturge-state/user.json",
    "successIndicator": "selector:[data-testid='user-menu']"
  }
}

Test public-facing pages without authentication.

{
  "auth": { "type": "none" }
}

GitHub Action Reference

Inputs

| Input | Description | Default | |-------|-------------|---------| | config | Path to config file | dramaturge.config.json | | target-url | Override target URL | — | | anthropic-api-key | Anthropic API key | — | | openai-api-key | OpenAI API key | — | | google-api-key | Google Generative AI API key | — | | fail-on-severity | Fail if findings ≥ severity | — | | post-comment | Post PR comment | true | | upload-report | Upload as artifact | true |

Outputs

| Output | Description | |--------|-------------| | report-path | Path to report directory | | finding-count | Number of findings | | max-severity | Highest severity found |

Example

- uses: aram10/[email protected]
  with:
    config: dramaturge.config.json
    anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
    fail-on-severity: major
    post-comment: true

Troubleshooting

Playwright browser not installed

This usually means Playwright's browser binaries aren't installed on the machine yet. Install Playwright browsers:

npx playwright install chromium

In some Linux containers you may also need OS dependencies:

npx playwright install --with-deps chromium

Authentication failures

Check that your successIndicator selector matches an element on the authenticated page.

No findings but issues exist

Increase exploration time:

{
  "budget": {
    "globalTimeLimitSeconds": 1800
  }
}

High LLM costs

Use cheaper models:

{
  "models": {
    "planner": "anthropic/claude-haiku-4-5",
    "worker": "anthropic/claude-haiku-4-5"
  }
}

Development

corepack enable
pnpm install
pnpm test
pnpm build

See CONTRIBUTING.md for contribution guidelines.

License

Apache License 2.0 — see LICENSE.

Links

• Repository
• Issues
• Changelog

Built with TypeScript, Node.js, Playwright, and Stagehand