@andyrmitchell/dom-step-runner

Declarative, in-browser UI automation using serialisable JSON steps.

DOMStepRunner lets you walk through a UI to reliably achieve a goal — by describing the journey as simple, declarative steps.

It runs entirely in the browser. Perfect for Chrome extensions, dynamic control flows, scraping flows, and LLM-driven UI control.

What It Is

DOMStepRunner executes a sequence of JSON-defined DOM steps:

• Wait for element
• Click element
• Optionally skip based on state
• Fail cleanly if something never appears

You define what should happen. The runner handles the timing, polling, resilience, and state transitions.

Think of it as:

A lightweight, declarative UI walkthrough engine that runs directly in the browser.

Why It’s Different

✅ Runs Inside the Browser

No headless browser. No Puppeteer. No background server. Works directly in:

• Chrome extensions
• Content scripts
• Injected scripts
• In-page automation

✅ Declarative & JSON-Based

Steps are pure serialisable objects.

This means:

• You can fetch updated steps from a server if the DOM changes
• You can let an LLM generate or repair steps dynamically
• You can store flows in a database
• You can diff and version them

Useful for brittle UIs like Gmail that change frequently.

✅ Designed for Dynamic UIs

It:

• Waits for elements instead of sleeping blindly
• Skips intelligently if the UI is already in the desired state
• Handles transient UI states
• Fails with precise diagnostics

Core Use Cases

1️⃣ State Preparation for Scraping

Example: YouTube transcript extraction.

To scrape a transcript you must:

1. Click “More”
2. Click “Show transcript”
3. Wait for transcript panel
4. Then extract text

DOMStepRunner handles the UI manipulation phase cleanly before your scraping logic runs.

2️⃣ UI Control (Chrome Extension Style)

Example: Gmail automation.

You might want to:

• Click compose
• Wait for modal
• Click formatting options
• Wait for dropdown
• Insert content

Instead of imperative brittle code, define the flow declaratively.

Especially useful in Chrome Extensions where you can't update the code quickly if the DOM changes, but you can download fresh JSON to get the latest steps.

3️⃣ LLM-Driven Automation

The steps are a codified language for working with a page, so an LLM could define the steps and fix them when the DOM changes.

An AI can:

1. Read the page
2. Set a goal
3. Identify necessary UI steps
4. Output a DOMStep[]
5. Run it
6. Detect failures
7. Repair and retry

How It Works

Step 1 — Install

npm install @andyrmitchell/dom-step-runner

Step 2 — Define Your Steps

Each step describes:

• What element to target
• What action to perform
• Optional timeout
• Optional delay
• Optional skip condition

Example:

const steps = [
  {
    name: "Open transcript menu",
    target: "button[aria-label='More actions']",
    action: "click"
  },
  {
    name: "Click show transcript",
    target: "ytd-menu-service-item-renderer",
    action: "click"
  },
  {
    name: "Wait for transcript panel",
    target: "#transcript",
    action: "wait_for"
  }
];

Step 3 — Run the Flow

import { DOMStepRunner } from "@andyrmitchell/dom-step-runner";

const runner = new DOMStepRunner(true, 5000);

const result = await runner.run(steps);

if (!result.success) {
  console.error("Failed at:", result.failedStepTarget);
}

Step 4 — React to Result

You get structured failure data:

{
  success: false,
  message: "Timeout waiting for '#transcript'",
  failedStepIndex: 2,
  failedStepTarget: "#transcript"
}

Use the feedback for:

• Retry logic
• LLM repair loops
• Telemetry
• Debugging UI breakages

API Reference

This section is intended for developers and LLM agents generating flows.

Types

DOMStep

export interface DOMStep {
  name: string;

  /** CSS selector of the DOM element to interact with */
  target: string;

  /** Action to perform */
  action: "click" | "wait_for";

  /**
   * Skip condition.
   * 'next_available' skips this step if the NEXT step's target
   * already exists in the DOM.
   */
  skipOn?: { condition: "next_available" };

  /** Override default timeout for this step (ms) */
  timeoutMs?: number;

  /** Delay before action execution (ms) */
  delayActionMs?: number;
}

DOMStepRunResult

export interface DOMStepRunResult {
  success: boolean;
  message: string;
  failedStepIndex?: number;
  failedStepTarget?: string;
}

Class

new DOMStepRunner(verbose?, defaultTimeoutMs?)

constructor(verbose: boolean = true, defaultTimeoutMs: number = 5000)

• verbose — Logs detailed step progression to console
• defaultTimeoutMs — Default wait time for element polling

run(steps: DOMStep[]): Promise<DOMStepRunResult>

Executes steps sequentially.

Behavior:

• Checks skip conditions
• Waits for target element
• Applies optional delay
• Re-checks skip
• Executes action
• Fails gracefully on timeout

Returns structured result object.

Examples

Example 1 — Idempotent Modal Dismissal

Dismiss cookie banner only if present.

const steps = [
  {
    name: "Dismiss cookie banner",
    target: "#accept-cookies",
    action: "click",
    skipOn: { condition: "next_available" } // If `#app-root` already exists, the click is skipped.
  },
  {
    name: "Wait for main app",
    target: "#app-root",
    action: "wait_for"
  }
];

Example 2 — Gmail Compose Automation

const steps = [
  {
    name: "Click Compose",
    target: "div[gh='cm']",
    action: "click"
  },
  {
    name: "Wait for compose modal",
    target: "div[role='dialog']",
    action: "wait_for"
  }
];

Example 3 — LLM-Generated Flow

An LLM might output:

[
  {
    "name": "Open menu",
    "target": ".menu-button",
    "action": "click"
  },
  {
    "name": "Wait for dropdown",
    "target": ".dropdown",
    "action": "wait_for"
  }
]

Your system simply:

await runner.run(llmGeneratedSteps);

If it fails:

• Feed failure back to the LLM
• Regenerate corrected steps
• Retry

When To Use This

Use DOMStepRunner when:

• You need reliable in-browser automation
• You’re building Chrome extensions around changing DOMs (code is slow to update, JSON can be instant)
• The DOM changes frequently
• You want server-driven automation definitions
• You’re building AI agents that manipulate UIs
• You want deterministic, debuggable flows

Do not use it if:

• You need full browser automation across pages (use Puppeteer)
• You need cross-origin control

License

MIT