web-automation-kit

A general-purpose, file-based workflow recorder for web apps (TypeScript + Playwright). You drive the browser manually; the kit records what happens: API requests/responses, navigations, form submits, and optional screenshots. This is intended to be shared infra: keep domain specifics (URLs, schedules, downstream parsing, naming conventions) in consumer repos.

What This Repo Contains

The core is src/recorder/ (browser + recording) and src/runs/ (run index). Local helpers live in scripts/ and justfile.

The kit writes data relative to the current working directory (your domain repo): data/.auth-state.json, data/runs/, and runs-index.json. Most consumer repos should gitignore data/ (and often runs-index.json as well).

Install

npm install
npx playwright install chromium
npm run build

How It’s Consumed (Domain Repos)

The intended model is “run the kit from the domain repo root”. That keeps data and config in the domain repo, while the recorder stays generic.

If you want a wrapper, a simple domain justfile can call the kit directly:

# in your domain repo justfile
auth:
    npx web-automation-kit auth

record:
    npx web-automation-kit record

status:
    npx web-automation-kit status

Configuration can live in .env or config.json. Environment variables win over config.json.

Minimal .env:

SOURCE_LOGIN_URL=https://example.com/login

Optional URL hints (helps auth detection):

AUTH_LOGIN_URL_HINTS=/login,/signin,/auth
AUTH_SUCCESS_URL_HINTS=/app,/dashboard

Minimal config.json (schema: config/config.schema.json):

{ "schema_version": "1", "source_login_url": "https://example.com/login" }

Validate config quickly:

just validate-config

Template Repo

For a ready-to-fork consumer skeleton, see ~/dev/web-automation-kit-template. It ships a minimal CLI wrapper and example config using this package.

Library Usage (Optional)

You can import the recorder from another project and provide config overrides programmatically:

import { recordWorkflow, setConfig } from '@albeorla/web-automation-kit';

setConfig({
  rootDir: process.cwd(),
  overrides: {
    SOURCE: { LOGIN_URL: 'https://example.com/login' },
  },
});

await recordWorkflow({ screenshots: true });

Consumer Helpers

If a consumer repo wants to avoid direct dotenv or playwright dependencies, it can import them from this package instead:

import '@albeorla/web-automation-kit/env';
import { chromium, type Page } from '@albeorla/web-automation-kit/playwright';

This keeps consumer package.json lean and centralizes dependency updates in the kit.

Run

Authenticate once (saves data/.auth-state.json):

npx web-automation-kit auth

Record a workflow (stop with Ctrl+C):

npx web-automation-kit record
# or: npx web-automation-kit record --screenshots

View recent runs:

npx web-automation-kit status

Output Format

Each run creates data/runs/<run-id>/ with step-numbered files like 00_metadata.json, *_page_navigation.json, *_graphql_request.json, *_graphql_response.json, *_rest_request.json, *_rest_response.json, *_form_submit.json, and optional *_screenshot.png. The exact sequence depends on the workflow you record.

Note: form submit capture currently includes field values. If that is too sensitive for a given domain, the consumer repo should patch src/recorder/capture.ts or wrap/disable that hook.

Scheduling (Per Domain)

This repo intentionally does not ship domain jobs. Follow the pattern from ~/dev/launchd-kit: domain repos define their own job scripts and plists, and use launchd-kit to install/reload/status. A template plist for the future replay feature lives at scripts/com.user.web-automation-kit.replay.plist.

How It’s Extended (Collaborators)

This repo should stay “boringly generic”. Collaborators typically extend it in one of two ways.

Domain extension lives in the consumer repo: parse captured JSON, derive domain schema, schedule captures, and add any alerting/monitoring. Treat the recorder output as raw evidence and build the domain meaning elsewhere.

Recorder extension lives here: add new capture capabilities that are broadly useful (redaction, extra event types, better request classification, stability improvements). Avoid hardcoding domains (URLs, selectors, request names) and prefer config flags when behavior needs to vary.

Development

Build: npm run build
Typecheck: npm run typecheck
Clean: npm run clean