@opendatalabs/remote-surface

Stream and control a remote browser from any container element — desktop or mobile, CDP or WebRTC, in a few lines.

remote-surface is a host-neutral substrate for building remote-browser products: an admin console that hands off to a live session, a QA harness that streams a headed browser to a reviewer, an agent product that lets a human take the wheel. It owns the parts every such product needs and gets wrong the same way twice — geometry, input translation, viewport negotiation, mobile text input, clipboard policy — so you can build the actual product on top instead of re-deriving them.

Features

• Container-fit viewer surface — fits a remote browser stream into any container element, tracks resizes, and maps pointer coordinates between client and stream space, letterboxing included.
• Viewport-match controller — resizes the remote browser to match the container instead of scaling a fixed-size stream, with debounce and transition classification so keyboard-inset and rotation churn don't cause resize thrashing.
• CDP backend — a first-class server-side backend that relays Page.screencastFrame, dispatches pointer/keyboard/text/clipboard input via Input.*, and applies viewport changes via Emulation.setDeviceMetricsOverride.
• n.eko (WebRTC) backend — contracts and safe client descriptors for stealth-sensitive flows that need a same-origin WebRTC session instead of raw CDP.
• Mobile IME input — a Guacamole-derived MobileTextInputController that translates beforeinput/compositionstart/compositionupdate/ compositionend events into keysym events or text-commit batches, so typing on a remote page works with real mobile IMEs.
• Form overlay (advanced alternative) — invisible native inputs positioned over remote form fields when an application needs local autocomplete and caret behavior beyond the default hidden-textarea IME bridge.
• Clipboard policy — direction and capability negotiation (local-to-remote, remote-to-local, bidirectional) per browser and session backend.
• Diagnostics — redacted, replayable event helpers and bounded in-memory buffers for input, viewport, and lifecycle events.
• Surface leases — host-neutral acquire/renew/release primitives for managing a capped pool of backend browser capacity.

Dependency-light (one runtime dependency, transformation-matrix), ESM-only, Node >=24.

Install

pnpm add @opendatalabs/remote-surface
# or
npm install @opendatalabs/remote-surface
# or
yarn add @opendatalabs/remote-surface

Quick start

Run the real server + vanilla viewer example:

pnpm example

Scan the printed QR code from a phone on the same network. The command binds to 0.0.0.0, prints the LAN URL as a fallback, launches a real headed Chromium, and streams https://example.com into the viewer. The first run may download Chromium once (about 90 seconds); the command says so before it starts. Choose another target with pnpm example -- https://your-app.test or REMOTE_SURFACE_START_URL=https://your-app.test pnpm example.

Embed in your app

Two files, wired together over one WebSocket: a server that owns a real headed Chromium and speaks CDP, and a client that fills a container with the stream. This is the full both-sides shape — copy both, connect them, and you have a live remote browser in a <div>.

Server (server.ts) — launches Chromium via Patchright, wraps the CDP session as a CdpCommandTransport, and relays frames/input over ws:

import { WebSocketServer } from "ws";
import { chromium } from "patchright";
import {
  createCdpServerBackend,
  type CdpCommandTransport,
} from "@opendatalabs/remote-surface/backends/cdp";

const browser = await chromium.launch({ headless: false });
const page = await browser.newPage();
await page.goto("https://example.com");
const cdpSession = await page.context().newCDPSession(page);

const transport: CdpCommandTransport = {
  send: (method, params) => cdpSession.send(method as never, params as never),
  on: (eventName, handler) => {
    cdpSession.on(eventName as never, handler);
    return { unsubscribe: () => cdpSession.off(eventName as never, handler) };
  },
};

const backend = createCdpServerBackend({ targetId: "session-123", transport });
// start() begins the screencast and returns a *lifecycle* — every
// subsequent input/setViewport/onEvent call goes through it, not `backend`.
const lifecycle = await backend.start({ type: "viewport", width: 1280, height: 720 });

const wss = new WebSocketServer({ port: 8787 });
wss.on("connection", (socket) => {
  lifecycle.onEvent((event) => socket.send(JSON.stringify(event)));
  socket.on("message", (raw) => {
    const msg = JSON.parse(String(raw));
    if (msg.type === "pointer") {
      void lifecycle.input({ type: "pointer", action: "pointerdown", x: msg.x, y: msg.y });
    }
  });
});

Client (client.ts) — the default host integration is a session. Adapt your WebSocket into its tiny transport port; the session owns fit, rendering, pointer/touch input, viewport matching, the hidden-textarea IME bridge, and clipboard flows:

import {
  createRemoteSurfaceSession,
  type RemoteSurfaceTransport,
} from "@opendatalabs/remote-surface/client";

const container = document.querySelector<HTMLElement>("#remote-surface")!;
const canvas = document.querySelector<HTMLCanvasElement>("#remote-surface-canvas")!;
const ws = new WebSocket("ws://localhost:8787");
const handlers = new Set<(message: Record<string, unknown>) => void>();

const transport: RemoteSurfaceTransport = {
  send: (message) => ws.send(JSON.stringify(message)),
  subscribe: (handler) => (handlers.add(handler), () => handlers.delete(handler)),
};

ws.addEventListener("message", (event) => {
  const message = JSON.parse(String(event.data)) as Record<string, unknown>;
  for (const handler of handlers) handler(message);
});

const session = createRemoteSurfaceSession({
  container, canvas, transport, initialViewport: { width: 1280, height: 720 },
});
window.addEventListener("beforeunload", () => {
  session.dispose();
  ws.close();
});

Run the one-command version with pnpm example, or run the lower-level server half yourself with pnpm examples:server (see examples/server) and the client half via examples/vanilla-viewer — both are typechecked in CI against the real published package, and both speak the exact message shape shown above.

remote-surface does not open the CDP connection, launch the browser, or create HTTP routes for you — it assumes a host that owns the browser process, transport, and authorization, and gives that host typed, tested primitives for everything downstream of "I have a CDP session." For every other task — viewport matching, mobile IME, form-overlay alternatives, clipboard policy, the n.eko backend — see docs/COOKBOOK.md.

Session and backend options

• touchMode: "native" | "wheel" on createRemoteSurfaceSession chooses raw touch (the default, for browser-native scrolling) or drag-to-wheel (for a backend that only accepts wheel input).
• detectTextInputFocus: true on createCdpServerBackend emits keyboard_focus events so the session raises its hidden-textarea keyboard only for actual remote text fields; use it for mobile IME support on CDP.
• focusBlurGraceMs sets the CDP focus detector's blur delay (50ms by default); raise it when focus moves between remote fields briefly dismiss the keyboard.
• renderNativeSelectPopupsInPage: true makes ordinary Chromium <select> pickers page-painted and screencast-visible; use it only when its observable CSS injection and rendering change are acceptable.
• keyboardOcclusion is a viewport message that keeps the remote viewport stable while reporting the soft-keyboard-covered bottom inset; have the host reveal the focused field instead of resizing the page (the vanilla example shows the flow).
• lifecycle.readRemoteSelection() reads selected remote text for a host to return over its transport; the session's copyRemoteSelection() uses that request/response flow before writing the local clipboard.
• clickCount on pointer down/up preserves browser single-, double-, and triple-click semantics; include it when forwarding a native MouseEvent.

The form-overlay primitives remain an advanced alternative for products that need native local controls over detected remote fields. They are not the default mobile-input path: start with the session's hidden-textarea IME bridge and add an overlay only for that stronger local-control trade-off.

Playground

playground/ is a local, hands-on acceptance harness: it launches a real headed Chromium and streams it to your browser over CDP so you can judge the UX directly, not a deployable demo (no serverless target — it needs a real browser process).

pnpm install
pnpm playground:dev

Open the printed URL. First run downloads a Chromium build (roughly 1-2 minutes cold, seconds after that). To test from a phone on the same network:

REMOTE_SURFACE_PLAYGROUND_HOST=0.0.0.0 pnpm playground:dev

See playground/TESTING.md for the full walkthrough — viewport presets, the form-overlay toggle, and the telemetry panels used to judge input accuracy.

Concepts

• Backends (/backends/cdp, /backends/neko) implement a narrow RemoteSurfaceBackendAdapter contract — start/stop, dispatch input, apply viewport, emit frames — so the client and protocol layers don't care which transport is underneath. CDP is a direct server-side integration; n.eko is a same-origin WebRTC session for stealth-sensitive use.
• Client (/client) provides the browser-side primitives — the container-fit surface, the viewport-match controller, geometry math, clipboard policy, and the mobile IME/form-overlay input path — independent of which backend is behind them.
• Protocol (/protocol) defines the JSON-safe event, input, viewport, and clipboard payload shapes that cross the host-owned wire, plus safe backend descriptor helpers that keep raw CDP/n.eko authority server-side.
• Server, leases, diagnostics (/server, /leases, /diagnostics) are optional host-neutral building blocks — a session/token store, a capacity lease manager, and redacted diagnostics buffers — for hosts that don't want to write that plumbing themselves.

The package deliberately stops short of routing, authorization, persistence, and browser process management. Those stay host-owned.

Development

pnpm install
pnpm verify          # typecheck + lint + tests + package validation + dist drift
pnpm test            # unit tests only
pnpm playground:dev  # local acceptance harness (see playground/TESTING.md)
pnpm playground:test # headless acceptance + mobile specs

License

Code is licensed under Apache-2.0. Documentation is licensed under CC-BY-4.0.

Links

• Issues
• Security policy