@approxima/playwright

Playwright with a babysitter. When a test fails, an LLM agent steps in to diagnose — and, if the failure is just a flake, can get the test to pass. When it's a real break, it writes a diagnosis into the test report and (if possible) suggests the fix.

The agent returns one of three verdicts per failure:

• flake — the test code is correct; the failure was transient. Test is marked passed with an intervention annotation.
• fix_required — the test's intent is valid but the code is stale (e.g. a selector changed). Test stays failed; a suggested fix is attached.
• broken — the application is genuinely wrong. Test stays failed; a diagnosis is attached.

Install

yarn add -D @approxima/playwright @playwright/test

@approxima/playwright is a drop-in replacement for @playwright/test — re-exports everything unchanged when disabled.

Configure

Approxima talks to a hosted agent-service that runs the LLM and orchestrates diagnosis. By default, it will use our managed instance, but you can also self-host your own agent-service. Simply point the package at your deployment in playwright.config.ts:

import { defineConfig } from "@playwright/test";
import type { ApproximaOptions } from "@approxima/playwright";

export default defineConfig<ApproximaOptions>({
  testDir: "./tests",
  reporter: [["list"], ["@approxima/playwright/reporter"]],
  use: {
    approxima: {
      enabled: true,
      // required - either put the key we generated for you, or use
      //            the same key as in agent-service if self-hosting.
      apiKey: '...',
      // optional - override agent-service URL for local development or if self-hosting.
      agentServiceURL: "https://...",
      // optional - defaults shown
      budget: {
        agentMs: 180_000,         // total wall-clock budget per dispatched agent
        maxToolCalls: 20,         // max tool calls the agent can make per test
        clampActionFraction: 0.5, // fraction of remaining test time any one action may use
      },
    },
  },
});

In your tests, import from @approxima/playwright instead of @playwright/test:

import { test, expect } from "@approxima/playwright";

test("login", async ({ page }) => {
  await page.goto("/login");
  await page.getByRole("button", { name: "Sign in" }).click();
  await expect(page).toHaveURL("/dashboard");
});

That's it. Failures are now babysat.

How it works

1. Approxima wraps Playwright's Page, Locator, FrameLocator, and expect with Proxies. Every action records itself, and failing actions have their timeouts clamped so rescue budget remains for the agent.
2. When the test body throws, approxima catches before Playwright marks the test failed.
3. The package opens a WebSocket to the configured agent-service and sends the test source, error, and surrounding context. The service drives the LLM loop and dispatches tool calls (screenshot, read DOM, read console/network logs, try_click, try_fill, etc.) back to the package, which executes them locally against the live browser.
4. The agent returns a verdict. If flake, approxima returns normally from the catch — Playwright sees a passed test. Otherwise it re-throws and attaches annotations.

Reporter output

With the approxima reporter installed, each test prints inline:

  ok  login [approxima:flake]
       -> transient cookie banner intercepted the click

And at the end of the run:

--- approxima summary ---
42 tests: 40 passed, 2 failed
approxima interventions:
  3 flake(s) salvaged (marked passed with agent help)
  1 fix(es) required (test code update suggested)
  1 real failure(s) diagnosed

Two reports are written to approxima-report/ by default: a machine-readable report.json for tooling and a human-readable report.md (linked from the console summary). Override the location:

reporter: [
  ["list"],
  ["@approxima/playwright/reporter", { outputFile: "reports/approxima.json" }],
],

Both files are written next to each other — the markdown filename is derived from outputFile by swapping its extension to .md (e.g. reports/approxima.md above). Relative paths resolve against the current working directory; absolute paths are used as-is.

Disabling

Don't set enabled: true in config, or omit the approxima key entirely. The package becomes a pass-through with zero overhead — the proxy get traps short-circuit when no ctx is present.

Limitations

• Failures in beforeEach / afterEach / beforeAll / afterAll hooks are not intercepted — only failures inside test bodies.
• Pages created outside the page fixture (e.g. chromium.launch().newContext().newPage()) are not wrapped. Use the page fixture.