JUCE WebView Agent Bridge

A debug-only JUCE module that lets an external agent (or script) drive a live juce::WebBrowserComponent over a loopback TCP socket — a "mini-CDP" giving you a browser-like toolkit on the real embedded WebView:

• eval arbitrary JavaScript and read the result;
• stream console.*, uncaught errors, and network — fetch/XHR, WebSocket / EventSource (SSE) frames, navigator.sendBeacon, plus PerformanceObserver resource timing;
• DOM / click / fill (React-safe synthetic input);
• screenshot the host window — or a cropped UI region — via the OS compositor (captures WebGL/canvas, which in-WebView snapshot APIs can miss) on macOS and Windows 11; Linux capture is a TODO.

No Chrome DevTools Protocol is required. WKWebView (macOS) exposes no CDP at all for an embedded view; WebView2 (Windows) can expose CDP, but only behind a remote-debugging flag you don't want in a shipping build. This bridge gives you one path that works the same on both without depending on that flag — eval/console/network/DOM ride on evaluateJavascript, which every JUCE backend already provides. macOS is the daily-use platform; Windows/WebView2 has also been verified against the home project's real Debug standalone. See Status below.

⚠️ Security. The bridge evaluates arbitrary JS. Access is gated only by the loopback (127.0.0.1) bind plus a per-session token — and the token sits in plaintext in the discovery file, so loopback is the real trust boundary (there is no TLS; anything that can reach localhost + read your home dir is trusted). It is gated to JUCE_DEBUG by default (WEB_AGENT_BRIDGE_ENABLED). Never ship it enabled in a release build.

Why

This started in Better Message Mycelia, a JUCE app with a dense React + PixiJS/WebGL plugin UI. A browser copy could not reproduce native state or timing, WKWebView had no CDP, and in-page screenshots missed GPU content. The bridge made the running plugin observable and controllable, which enabled:

• agent debugging against the real UI and C++ engine;
• live e2e tests for controls, presets and native state;
• performance probes using event streams and rAF frame-gap measurements.

Why not just Playwright?

Playwright cannot attach to WKWebView and needs an explicitly enabled CDP endpoint for WebView2. This bridge instead uses JUCE's existing evaluateJavascript and native-function surfaces. It can also capture compositor-rendered WebGL and assert on real C++ state through backend().

It is not a Playwright replacement: input is synthetic, network is observe-only, and there is one page. You can still use the @playwright/test runner as a harness; see examples/playwright-test.

Status — what's actually been exercised

Honest test coverage, so you know what you're getting:

| Area | macOS (WKWebView) | Windows (WebView2) | Linux | |---|---|---|---| | eval / console / network / DOM / e2e | ✅ used daily against a real plugin | ✅ verified against a real WebView2 Debug standalone | ⚠️ untested against a real app | | Native screenshot (shot) | ✅ (macOS 14+, ScreenCaptureKit) | ✅ (Windows 11, Windows.Graphics.Capture + D3D11) | ❌ TODO | | C++ + JS test suites | ✅ CI | ✅ CI | ✅ CI |

The Windows real-app verification covers discovery/auth, Unicode eval, console and fetch capture, DOM locators, React-safe fill/click, accessibility snapshots, large chunked reads, sink replay, and native full-window/region screenshots. WebKit layer inspection remains macOS-only. Linux real-app reports and fixes are very welcome.

Install

Requirements: JUCE 8, C++17, a juce::WebBrowserComponent-based UI; the clients need Node ≥ 22 (zero runtime dependencies).

It's a standard JUCE module. Get the repo — as a submodule:

git submodule add https://github.com/mateusz28011/juce-webview-agent-bridge.git modules/juce-webview-agent-bridge

or with CMake FetchContent (after JUCE has been added, so juce_add_module is available):

include(FetchContent)
FetchContent_Declare(juce_webview_agent_bridge
    GIT_REPOSITORY https://github.com/mateusz28011/juce-webview-agent-bridge.git
    GIT_TAG        v0.5.1)
FetchContent_MakeAvailable(juce_webview_agent_bridge)

The module itself lives in the juce_webview_agent_bridge/ subdirectory (JUCE requires the module directory to be named exactly like the module ID). Register and link it:

# Submodule/manual checkout:
juce_add_module(path/to/juce-webview-agent-bridge/juce_webview_agent_bridge)

# FetchContent_MakeAvailable registers the module through the repo's root CMakeLists.
target_link_libraries(MyPlugin PRIVATE juce_webview_agent_bridge)

Install the zero-runtime-dependency Node client separately in the project that owns your e2e tests. The npm package is independent of how the C++ module was fetched and includes strict TypeScript declarations:

npm install --save-dev juce-webview-agent-bridge
npx juce-webview-agent-bridge ping

Integrate (3 steps)

#include <juce_webview_agent_bridge/juce_webview_agent_bridge.h>

// 1. Fold capture into the WebView Options (before creating the WebView):
auto bridge = std::make_shared<web_agent::WebAgentBridge>();
options = web_agent::withCapture (std::move (options), bridge);   // no-op if disabled

// 2. After the WebView exists, wire eval + bounds and start listening:
web_agent::connect (*bridge, *webView, *this /* component for screen bounds */);
bridge->start();                                                  // 127.0.0.1:8930

// 3. On teardown, before the WebView is destroyed:
bridge->stop();

Wrap the calls in #if WEB_AGENT_BRIDGE_ENABLED if you keep the bridge member unconditionally (a std::shared_ptr to a forward-declared type is fine).

Agent skill

skills/juce-webview-agent-bridge/ is a skill that teaches a coding agent (e.g. Claude Code) to drive the bridge — commands, the e2e client, diagnostic techniques, and the gotchas below in agent-digestible form:

npx skills add mateusz28011/juce-webview-agent-bridge

CLI client

With the npm package installed, use npx juce-webview-agent-bridge. A checkout can run the committed tools/web-agent.mjs build directly without installing dependencies:

npx juce-webview-agent-bridge ping
npx juce-webview-agent-bridge eval "document.title"
npx juce-webview-agent-bridge logs                     # live console + network stream
npx juce-webview-agent-bridge shot /tmp/ui.png         # native compositor shot of the host window
npx juce-webview-agent-bridge shot /tmp/panel.png "#panel"  # crop to an element

Other commands: hello, dom, click, fill, capture, backlog, layerdebug, and layertree. Port and token are normally auto-discovered.

E2E (Playwright-style)

The package root exports a Playwright-shaped client with client-side auto-waiting:

import { connect, expect } from 'juce-webview-agent-bridge';

const page = await connect({ activate: 'My App' });
await page.getByTestId('patch-name').fill('warm pad', { enter: true });
await page.locator('text=Save').click();
await expect(page.locator('.patch-row')).toHaveCount(1);
await expect.poll(() => page.backend('getPatchCount')).toBe(1);
page.close();

The client includes CSS/text/role locators, click/fill/type/drag actions, auto-retrying assertions, accessibility snapshots, native screenshots, live console/network/error events, large-value reads, render-performance probes and JUCE native calls. Full API and operational guidance: docs/e2e.md.

Discovery & auth

On start() the host binds 127.0.0.1:8930 (scanning upward on collision) and publishes {port, token} to ~/.web_agent_bridge.json (plus a per-instance file under ~/.web_agent_bridge.d/ so several hosts coexist). Clients read it automatically; a random per-session token gates every connection. The wire is newline-delimited JSON over TCP (hello / ping / auth / eval / bounds / shot / layerdebug / layertree / sink_replay + the unsolicited sink event stream) — the full op table, sink-frame format, and discovery details are in docs/protocol.md.

Known limits

• Native screenshots require macOS 14+ or Windows 11 and a compositor-capturable, non-minimized window. macOS permission/signing setup is in docs/screen-recording.md. Linux capture is a TODO.
• Synthetic input dispatched from JS has isTrusted === false, so APIs gated behind a user gesture (file pickers, some clipboard/fullscreen) are out of reach.
• eval errors are only reported on WKWebView; on WebView2 a failed eval looks like a null result — rely on the console/error stream there.

Testing

Both suites live in tests/ and run independently of any host app, so they keep working after this module is extracted into its own repo.

# JS client — zero dependencies, Node >= 22
npm test                     # or: node --test tests/*.test.mjs

# Maintainer build — strict TypeScript source -> committed .mjs + .d.mts
npm ci --ignore-scripts
npm run build
npm run test:types

# C++ bridge — fetches JUCE + Catch2 on first configure
cmake -S tests -B build/test
cmake --build build/test
ctest --test-dir build/test --output-on-failure

# reuse a local JUCE checkout instead of downloading it:
cmake -S tests -B build/test -DWAB_JUCE_DIR=/path/to/JUCE

The JS tests drive the real clients against a mock bridge; the C++ suite drives the real loopback server. Neither requires a host app.

Contributing

See CONTRIBUTING.md — ground rules (app-agnostic, zero client deps, debug-only), how to run both test suites, and the release flow. Linux capture support and Linux real-app verification are the most wanted contributions.

License

MIT — see LICENSE.