@petepetrash/munari

HTML, 3D, and Shaders, Unified.

Munari lets React content appear in a Three.js scene while keeping its original DOM and state. It captures the HTML, keeps its texture current as the user works with it, routes input, and coordinates the handoff between page and scene. The project is named for the Italian designer and artist Bruno Munari.

Requirements

• Peer dependencies: react and react-dom 19 or later, three ~0.186.1, and @react-three/fiber ^9.8.1.
• Optional peers: @types/three ~0.186.0 for TypeScript, and @zumer/snapdom ^3.3.3 for the snapDOM capture engine.
• Rendering: SurfaceCanvas renders with Three's WebGPURenderer. Where a browser has no WebGPU adapter, it falls back to WebGL 2. Neither accepts GLSL, so ShaderMaterial and onBeforeCompile do not work. Neither do the drei helpers built on them, such as Text, MeshDistortMaterial and MeshTransmissionMaterial. In development, SurfaceCanvas logs which object uses one. Write custom materials as TSL node materials. See Materials on a SurfaceCanvas.
• Capture: HTML-in-canvas runs in Chrome with the CanvasDrawElement feature on, either through --enable-features=CanvasDrawElement or on an origin that carries the HTML-in-canvas origin trial token. Other browsers need snapDOM. In Safari, which has no Element.moveBefore, a handoff works but restarts a running CSS animation. The capture engines table compares the two.

Without a capture engine, handoff content stays native and usable, but it never enters the scene. useSurfaceSupport() reports capability after hydration; supportsSurfaces() is the imperative check for an event handler.

Capture engines

Munari ships two ways to turn your HTML into pixels. Both feed the same textures, pointer routing and handoff; they differ in what they need from the browser and what they can draw.

| | HTML-in-canvas (default) | snapDOM | |---|---|---| | Needs | Chrome with --enable-features=CanvasDrawElement | any browser with a document | | Draws | the live element | a rasterized copy of the subtree | | What the texture follows | everything, for free | what the user does to the content; a resize, a font or image landing, a transition's ends. Content that changes on its own needs live | | Caret and selection | real, painted by the browser | not painted | | Everything else on the page | pixel-exact | close: laid-out lengths snap to whole CSS pixels, and vertical geometry can sit up to two device pixels off | | Input to texture, Knobs panel at dpr 2 | 12 ms | 37 ms for a change that lands on a quiet Surface; up to 150 ms more on one that is already re-capturing | | Pointer route "auto" | native: the browser hit-tests the real element | relay | | Page-to-scene handoff | yes | yes, in every browser; a running CSS animation restarts where Element.moveBefore is missing (Safari) | | Sub-pixel CSS lengths | exact | a fractional border-width, padding or offset rounds to whole pixels; type and SVG strokes keep their fractions |

HTML-in-canvas is the default engine. It needs no code, but the browser must provide it (see Requirements). To add snapDOM, install it and enable it once, before you render:

npm install @zumer/snapdom

import { enableSnapdomCapture } from '@petepetrash/munari/snapdom'

enableSnapdomCapture()

That call keeps HTML-in-canvas when the browser has it and falls back to snapDOM when it does not. Pass { always: true } to force snapDOM everywhere, which is what a cross-engine test run wants. Call it before the first Surface mounts. A Surface that is already mounted keeps the engine it was built with.

Write for HTML-in-canvas and the same code runs on snapDOM. The one difference you have to declare: a Surface whose content moves on its own (an animation, a clock, a ticker, a video) needs live for snapDOM to keep re-capturing it, and it then re-captures at most about four times a second. Without live, snapDOM shows the content as the user last left it: typing, hover, focus, a click that re-renders the component, a drag, a scroll, and the layout, fonts and images are all followed; a change nobody asked for is not. That includes a change made from outside the Surface (new props, data that arrives after mount, a control elsewhere on the page), so a component that loads its content needs live too. HTML-in-canvas ignores the flag because it follows everything at no cost. useElementCapture takes the same live option, and there it applies to both engines, because rebuilding the copy of a native element is the cost, not the capture.

<Surface inScene={inScene} live><Clock /></Surface>

useSurfaceStatus().engine names the engine a Surface is using, or is null when no engine can run. Authoring lists the content rules each engine adds.

Install

npm install @petepetrash/munari @react-three/fiber
npm install [email protected] --save-exact

Pin three. The peer range is ~0.186.1, but npm rewrites three@~0.186.1 to ^0.186.1 in package.json, which accepts 0.187. Three changes its WebGPU API between minor versions. Keep ~0.186.1 there by hand if you prefer a range.

In a TypeScript project, also install @types/three:

npm install --save-dev @types/three

Import the stylesheet once, as the example below does. A TypeScript project that imports style.css needs a CSS module declaration, such as the one in vite/client types. The package leaves React, Three.js and R3F external as peers.

A minimal app bundles to about 2 MB minified (about 550 to 640 kB gzip), mostly three/webgpu, and Vite warns about the chunk size. Load the route that mounts SurfaceCanvas with React.lazy or a dynamic import() to keep it off the first page.

Your first Surface

Put your existing HTML component inside Surface. Its inScene boolean requests where it should appear. The component stays mounted once, so its local state and uncontrolled inputs survive the round trip.

import { useId, useState } from 'react'
import { Surface, SurfaceCanvas, useSurfaceSupport } from '@petepetrash/munari'
import '@petepetrash/munari/style.css'

function Counter() {
  const [count, setCount] = useState(0)
  return (
    <button type="button" style={{ width: 240, height: 100 }} onClick={() => setCount(count + 1)}>
      Count {count}
    </button>
  )
}

export function Example() {
  const canvasId = useId()
  const supported = useSurfaceSupport()
  const [inScene, setInScene] = useState(false)
  return (
    <main style={{ position: 'relative', minHeight: 400, padding: 32 }}>
      {supported && <SurfaceCanvas
        id={canvasId}
        pointerMode="surfaces"
        frameloop="demand"
        style={{ position: 'absolute', inset: 0 }}
      />}
      <Surface canvasId={canvasId} inScene={inScene}><Counter /></Surface>
      <button type="button" disabled={!supported} onClick={() => setInScene(value => !value)}>
        {inScene ? 'Return to page' : 'Show in scene'}
      </button>
    </main>
  )
}

SurfaceCanvas owns the R3F renderer, camera, and scene. It fills its parent, so give the parent a size; an auto-height parent leaves a 150 px canvas. Surface supplies a flat mesh matching the HTML's page position. Switching alone preserves its appearance; flight, deformation, lighting, and shader effects come from your scene code. The running starter uses this pattern.

If "Show in scene" stays disabled

No capture engine can run in this browser. That happens in Chrome without the CanvasDrawElement feature when snapDOM is not enabled, and in any other browser. useSurfaceSupport() is false, so the example renders no canvas and the content stays on the page. useSurfaceStatus().reason says so, and a Surface asked to enter the scene logs the same sentence once in development. Fix it in one of three ways:

• Start Chrome with --enable-features=CanvasDrawElement.
• Serve the page with the HTML-in-canvas origin trial token.
• Call enableSnapdomCapture(), which also covers other browsers.

canvasId selects the SurfaceCanvas with a matching id. Omit canvasId when using the unnamed default canvas. Several Surfaces can share the same canvas while their HTML stays in different parts of the page.

Add custom scene content

The snippets from here on assume a sized SurfaceCanvas is mounted on the page, as in the first example. Use the explicit composition form when the effect needs its own meshes or logic:

<Surface.Root inScene={selected} canvasId="controls">
  <Surface.HTML><ControlBoard /></Surface.HTML>
  <Surface.Scene>
    <Surface.Mesh alpha="source" pointerRoute="auto">
      <ControlsHardware />
    </Surface.Mesh>
  </Surface.Scene>
</Surface.Root>

<SurfaceCanvas id="controls"><ControlsLights /></SurfaceCanvas>

ControlBoard, ControlsHardware, and ControlsLights are application components in this example. The first draws HTML; the second supplies the physical controls and their motion; the third supplies ordinary R3F lights. Munari's Surface.Mesh draws the captured HTML and provides its texture, named anchors, and input mapping. See the complete Controls caller.

Surface.Scene retains custom children through preparation, reversal, and return. It contributes to the shared canvas rather than creating another renderer. Declare it before it is requested. Shared cameras and lights belong under SurfaceCanvas. Use placement="manual" and normal R3F geometry/transforms for scene-owned placement. group, mesh, and planeGeometry are R3F's Three.js elements.

For several pieces of HTML that must transfer together, give each Surface.HTML a distinct part name and select that part on its mesh. sampledParts records additional sources sampled by a material. A manual pointer proxy cannot claim those draw receipts. Knobs and Logo show coordinated parts and anchors.

Materials on a SurfaceCanvas

SurfaceCanvas converts each fragment to the canvas's sRGB encoding and blends it there, as WebGLRenderer did. The conversion expects premultiplied color:

• A stock material that draws its own translucency, such as a fading meshBasicMaterial, sets premultipliedAlpha: true. With Three's default of false, its alpha applies twice and it draws too dark. Surface.Mesh sets the flag on the material in its material slot for you.
• A custom outputNode returns its color through premultipliedOutput, for premultiplied linear color, or encodedOutput, for premultiplied color already in sRGB. Both come from @petepetrash/munari.
• A material that samples a Surface capture returns it through premultipliedOutput. The capture is already premultiplied, so a colorNode or a stock map multiplies it by alpha again, and translucent pixels draw darker than the page. premultipliedAlpha: true does not fix that.
• A material that samples the canvas, such as Three's transmission or a viewport-texture node, draws too bright. Render the scene behind it into a render target and sample that instead, as registry/glass/ does.

Opaque materials need neither. Custom materials are TSL node materials, because the renderer does not accept GLSL. Decision #72 has the measurements.

A custom material on Surface.Mesh

The slot does not apply the capture to the material, so the material samples it itself. Build it in a component that calls useSurfaceNodes(), return it with <primitive attach="material" />, and pass the component as material:

import { useEffect, useMemo } from 'react'
import { MeshBasicNodeMaterial } from 'three/webgpu'
import { uv, vec3, vec4 } from 'three/tsl'
import { Surface, premultipliedOutput, useSurfaceNodes } from '@petepetrash/munari'

function TintMaterial() {
  const surface = useSurfaceNodes()
  const material = useMemo(() => {
    const m = new MeshBasicNodeMaterial({ transparent: true })
    const sample = surface.map.sample(uv()).mul(surface.radiusMask())
    m.outputNode = premultipliedOutput(vec4(sample.rgb.mul(vec3(0.5, 1, 1)), sample.a))
    return m
  }, [surface])
  useEffect(() => () => material.dispose(), [material])
  return <primitive object={material} attach="material" />
}

<Surface.Root inScene={inScene}>
  <Surface.HTML><Card /></Surface.HTML>
  <Surface.Scene>
    <Surface.Mesh material={<TintMaterial />} />
  </Surface.Scene>
</Surface.Root>

surface.map is the capture, surface.radiusMask() applies the Surface's rounded corners, and the tint halves red in linear light. useSurfaceNodes() and useSurfaceTexture() work only inside the component passed as material.

material also takes a material instance you own and dispose, such as material={tint}. An instance cannot call useSurfaceNodes(), so one that shows the capture builds its texture node from useSurfaceTextureOf(handle), which is null until the first upload.

For a lit slab, pass material={<Surface.LitMaterial roughness={0.4} />} and add lights under SurfaceCanvas. A JSX tag such as <meshBasicNodeMaterial /> throws "is not part of the THREE namespace" until you register it with R3F's extend(), so build the material as above instead.

Renderer settings

gl passes options to the WebGPURenderer that SurfaceCanvas creates. Two of them decide most of a canvas's GPU memory:

<SurfaceCanvas gl={{ antialias: false, depth: false }}>

• antialias defaults to true. It smooths triangle edges by keeping 4 color and 4 depth samples per pixel. On WebGPU, those buffers take 32 bytes per canvas pixel, about 160 MB for a full-window canvas at 1440×900 and device pixel ratio 2. Turn it off when the canvas draws no visible triangle edges, such as a full-screen shader quad, or HTML on unrotated planes square to the camera.
• depth defaults to true. Turn it off when nothing on the canvas needs depth testing, such as a single mesh or transparent layers drawn in order. With antialias on, it saves half of those 32 bytes.

Three 0.186.1 ignores antialias on its WebGL 2 fallback, so a browser without WebGPU draws aliased edges either way (three.js#34947).

HTML that belongs in a scene

<SurfaceCanvas camera={{ position: [0, 0, 5] }}>
  <group position={[0, 0.5, 0]}>
    <SceneSurface size={[240, 80]}>
      <div style={{ width: 240, height: 80, background: 'white' }}>Scene label</div>
    </SceneSurface>
  </group>
</SurfaceCanvas>

Import SceneSurface from the package. size is the HTML's capture size in CSS pixels. Its convenience mesh is one scene unit high and preserves that aspect ratio; the surrounding group controls its position and scale. For custom materials, geometry, or multiple parts, use SceneSurface.Root, .HTML, and .Mesh. This form has no native page presentation to return to; supply an appropriate fallback when the scene is essential to the experience.

Capture an element that stays native

const capture = useElementCapture()
return (
  <>
    <article ref={capture.ref}>Native selectable content</article>
    <SurfaceCanvas><GlassEffect capture={capture} /></SurfaceCanvas>
  </>
)

Import useElementCapture from the package. GlassEffect is application scene code: inside it, useCaptureFrame(capture).get() reads the latest painted texture, dimensions, and anchors, or null while unavailable. Paints wake the renderer without causing a React render per frame. The texture is borrowed; consumers do not dispose it.

The native element remains in place. Attach the ref to an element, document.body, or document.documentElement; exclude unsupported content and the capture's own preview when capturing a document. The copy is rebuilt when the user acts on the element, when its layout, stylesheets, fonts or images change, and when a transition ends. A change nobody made on it (data arriving, a store update, a control elsewhere on the page) is not followed: call capture.refresh() from whatever makes it, or pass useElementCapture({ live: true }) to follow every mutation and every running animation. CaptureContent instead supplies separately authored React children or a detached element to a useCaptureHandle() identity. CaptureContent requires explicit dimensions for either input. Selection and Veil are complete element-capture callers.

CaptureContent children are HTML, so CaptureContent with children belongs in the page tree. Inside SurfaceCanvas it throws. There, pass a built element with element instead. For children, create the handle in a page component, render CaptureContent there, and pass the handle to the scene component:

import { useRef } from 'react'
import { useFrame } from '@react-three/fiber'
import type { Mesh, MeshBasicMaterial } from 'three'
import { CaptureContent, SurfaceCanvas, useCaptureFrame, useCaptureHandle, type CaptureHandle } from '@petepetrash/munari'

function CapturedPlane({ capture }: { capture: CaptureHandle }) {
  const frame = useCaptureFrame(capture)
  const mesh = useRef<Mesh>(null)
  const material = useRef<MeshBasicMaterial>(null)
  useFrame(() => {
    const current = frame.get()
    if (!current || !mesh.current || !material.current) return
    if (material.current.map !== current.texture) {
      material.current.map = current.texture
      material.current.needsUpdate = true
    }
    mesh.current.scale.x = current.width / current.height
    mesh.current.visible = true
  })
  return (
    <mesh ref={mesh} visible={false}>
      <planeGeometry />
      <meshBasicMaterial ref={material} />
    </mesh>
  )
}

export function Page() {
  const capture = useCaptureHandle()
  return (
    <>
      <CaptureContent capture={capture} size={[240, 80]}><p>Authored in React</p></CaptureContent>
      <div style={{ position: 'relative', height: 400 }}>
        <SurfaceCanvas><CapturedPlane capture={capture} /></SurfaceCanvas>
      </div>
    </>
  )
}

The plane is one unit high, and width / height gives its aspect. The texture is premultiplied, so this stock material draws opaque content correctly and translucent pixels too dark. See Materials on a SurfaceCanvas.

Sharpness by default

The canvas uses the display's native pixel ratio, and capture density follows the rendered size on each axis. Stationary, flat HTML meshes align their rendered pixels to the display grid. Their animation state and DOM layout remain unchanged. Moving or warped content keeps its continuous pose. Explicit dpr or resolution limits remain available when you choose a rendering budget; large textures still obey the 4096-pixel edge limit.

Observe and animate a Surface

| Read | Meaning | |---|---| | useSurfaceStatus().requestedInScene | The author's boolean intent, including fallback | | presentation | Accepted page or scene hold; null if none exists | | sceneReady | Required sources and preparation draws are ready | | isTransitioning | The handoff is progressing or awaiting preparation | | supported, reason | Capability/content support and an unavailable reason | | engine | 'html-in-canvas' or 'snapdom'; null when no engine can run | | useFreezeSurface() | Non-live content is frozen for the scene; stop your own clocks | | useSurfaceProgress().get() | Raw 0..1 motion, identical to driver input | | useSurfaceProgress().eased() | The explicitly eased version of that motion |

When no engine can run, supported is false and reason says what to enable: "No capture engine is available. Enable Chrome's CanvasDrawElement flag or an HTML-in-canvas origin trial, or call enableSnapdomCapture() from @petepetrash/munari/snapdom." That reason takes precedence over a content reason such as inline handlers, which appears only when the engine works.

useSurfaceStatus() without a handle reads the nearest enclosing Surface, so it throws in the component that renders the Surface. That component creates the handle and passes it down:

import { Surface, useSurfaceHandle, useSurfaceStatus, useSurfaceTextureOf } from '@petepetrash/munari'

function Card({ inScene }: { inScene: boolean }) {
  const surface = useSurfaceHandle('card')
  const status = useSurfaceStatus(surface)
  const texture = useSurfaceTextureOf(surface)
  return (
    <>
      <Surface surface={surface} inScene={inScene}><CardContent /></Surface>
      <p>{status.supported ? `${status.presentation}, texture ${texture ? 'ready' : 'pending'}` : status.reason}</p>
    </>
  )
}

useSurfaceHandle() creates a new identity. It does not return the enclosing Surface's, so call it once in the owner and pass the handle to Surface. Children of the Surface can call the same hooks with no handle.

onPresentationChange uses page | scene | null; onMotionComplete and a driver's string target use page | scene. useSurfaceDriver(step, handle?) supplies a raw progress value. useSurfaceMotion(step, handle?) supplies position, a numeric 0/1 target, and scenePresented. The protocol still decides when the page releases. Without a handle, these hooks read the nearest Surface identity.

useSurfaceBeforeRender belongs inside Surface.Mesh. It runs after pose writers and world-matrix updates, before each render pass, with the actual camera and render target. It may run several times per animation frame. Advance physics in the frame step; update shadows or other companions here. canvasMayDraw permits the upcoming draw and is separate from an accepted presentation receipt. The postcard demonstrates this boundary.

Changing page layouts and renderer availability

Use usePageTarget() when the same content must return to different React layout parents. Render ref={target.ref} on the current slot and pass target to Surface.HTML; the content itself stays at one stable React position. Flight uses one target and Root per card. Ordinary handoffs in a fixed slot need no target. Keyed target lists can be prepended, reordered and removed without moving React's own list anchors. Server-rendered targeted HTML hydrates the original node.

One unnamed canvas is the default. Multiple hosts use unique IDs and an explicit canvasId association. Reusable client components can use React useId; independent SSR roots need distinct matching identifierPrefix values during server render and hydration, or document-unique authored IDs. A scene-side Surface belongs to its enclosing canvas and rejects a conflicting association.

A missing/lost renderer keeps handoff HTML usable. Preparation waits stop claiming renderer work when only missing input can unblock them. Development warns once after ten seconds of an unresolved host or preparation episode, without changing state or invoking onError. Declared but unrequested scenes remain quiet. Unsupported media, custom elements, and external form associations remain native with a reason until their retained-content behavior is supported.

Native pointer routing is opt-in on Surface.Mesh with pointerRoute="auto". It requires a known planar geometry and an interactive source. Multiple scene poses sharing one source all use relay. pointerEvents="none" disables scene input while page-owned preparation remains interactive.

Surface.Mesh has no onClick, because the gate swallows the browser click after any press on a Surface. Use onPointerUp. onDoubleClick and onContextMenu fire under the relay route.

When enhanced rendering is unavailable

A Surface keeps its HTML native and preserves author intent. No scene-only completion callback can finish a gesture that never entered the scene, so branch inside actions with supportsSurfaces() and commit their ordinary HTML outcome. Scene-only visuals need their own fallback. Keep the content-root sizing, paint, mask, and hover rules in authoring.

Next.js and server rendering

The @petepetrash/munari and @petepetrash/munari/advanced entries start with 'use client'. A Next.js server component can import Surface, SurfaceCanvas and the hooks. Surface and SurfaceCanvas render on the client only. A component that calls a hook still needs its own 'use client', and Surface needs an inScene boolean prop.

useSurfaceSupport() is false on the server and on the first client render, then reports the browser. Gate the canvas on it, as the first example does, and the server and client markup match.

@petepetrash/munari/snapdom has no directive. Call enableSnapdomCapture() at the top level of a client module that loads before the first Surface mounts. The call is safe during server render:

'use client'
import { enableSnapdomCapture } from '@petepetrash/munari/snapdom'
import { Example } from './Example'

enableSnapdomCapture()

export function ExampleWithSnapdom() {
  return <Example />
}

DOM shape and testing

A Surface can put two copies of its content in the DOM. One is live. The other is inert and aria-hidden:

• While the Surface is in the scene, an inert copy stays at the page position to hold layout, and the live content sits inside the canvas.
• In page mode, a hidden data-munari-source-host element in a parked canvas also holds an inert copy.

document.querySelector('[data-testid="count"]') can return the inert copy, and querySelectorAll('input') can return two elements. Role-based queries, such as Testing Library's getByRole, skip aria-hidden content and find the live one. For a CSS selector, exclude the copy:

const live = [...document.querySelectorAll('[data-testid="count"]')]
  .filter(element => !element.closest('[inert]'))

Known console output

• THREE.Clock: This module has been deprecated. Please use THREE.Timer instead. comes from @react-three/fiber 9.8.1, which constructs a THREE.Clock. Munari does not. It is harmless.
• Where the browser has no WebGPU adapter, Three logs that WebGPU is not available and that it runs under the WebGL 2 backend, with a few related warnings. These are Three's own messages for the fallback.

Run the lab locally

The repo uses Node 24 and npm 11. The local launcher starts Vite, opens an isolated Chrome with CanvasDrawElement enabled, and stops the server when you close that Chrome window:

npm ci
npm run lab

Set CHROME_PATH if Chrome is installed somewhere unusual. npm run dev still starts only Vite for a browser that already has the flag enabled.

Go further

• Demo source map links the implementations across Flight, Controls, Knobs, Selection, Gallery and the other scenes.
• Authoring rules describe the browser capture constraints.
• Agent workflow maps a change to its owner and check.
• /advanced exports frame sources, manual presentation receipts, readSurfaceFrameState, and capture inspection. These are for custom renderers.
• The tracked Munari skill teaches this API to agents.

Repo shape

| path | what it is | | --- | --- | | packages/core | the renderer-agnostic core; no dependencies, bundled into the package | | packages/react | the @petepetrash/munari package: React/three components over core | | registry/ | source you copy into your project (nothing published) | | apps/lab | the demo and development app | | instruments/ | browser probes and CI gates | | tests/conformance/ | the test suites that define core's behavior |

Dependencies point one way: apps depend on packages/react, which depends on packages/core. tests/boundary.test.ts checks the actual imports.

See AGENTS.md for the working rules, docs/decisions.md for the numbered design decisions, docs/platform.md for what the platform is measured to do, docs/authoring.md for how to write markup a Surface can draw, and docs/focus.md for the focus and spatial-navigation contract.

Development

npm ci
npm run lab              # Vite + a compatible local Chrome
npm run dev              # Vite only
npm run typecheck
npm test
npm run lint
npm run gate:idle-zero   # browser gate: idle Surfaces cost 0 paints/s

package.json lists available gate commands; the CI workflow selects the gates run on each push. The instrument guide gives each check's scope and limits. Run GPU gates in series. A capability skip is not a passing behavior check; use STRICT_CAPABILITY=1 where HTML-in-canvas must be present.

npm run build stages the package with core bundled in and peers left external. The staged package includes the canonical root README, license, changelog, llms.txt, and Munari skill. To use a checkout's API in another project, run the build and install the staged packages/react/dist directory in that project. To publish, inspect the staged package, then publish from its directory:

npm run build
npm pack --dry-run packages/react/dist
npm publish packages/react/dist