lecodes-renderer

Formerly canvas-ui. The createCanvasUI(...) factory and Canvas2DPainter keep their names; the npm package is lecodes-renderer.

A clean, portable reimplementation of the UI renderer (the _creatorUI end of the worker authoring layer), built around a canvas backend instead of the DOM. It runs headless under Bun, reuses the existing creator-ui (Yoga) WASM for layout, and can emit the interface as a semantic JSON description as well as paint it to a canvas.

See ARCHITECTURE.md for the design and the C++-port plan.

Usage

import { createCanvasUI, Canvas2DPainter, BrowserHost } from "canvas-ui"

// Headless / tests (deterministic): default HeadlessHost + creator-ui WASM.
const ui = await createCanvasUI()
ui.resize(390, 844)
ui.openScreen(screenNode) // a worker UI node tree

console.log(ui.toText())          // → JSON description of the UI (AI feedback / assertions)

// Browser: paint to a canvas.
const ui2 = await createCanvasUI({ host: new BrowserHost() })
ui2.resize(width, height)
ui2.openScreen(screenNode)
ui2.paint(new Canvas2DPainter(canvas.getContext("2d")!))

ui is a drop-in _creatorUI (openScreen / insertNode / updateStyle / updateNode / …) plus the new rendering surface: resize, renderTree, serialize/toText, paint(painter), and pointerDown/Move/Up.

Headless sessions & scenarios (Node)

The lecodes-renderer/headless entry runs a compiled project (the CLI's iife bundle) in a node:vm sandbox. Beyond the one-shot renderToJson / renderToPng, createSession keeps the app alive and drives it like a finger would — snapshots land between actions:

import { createSession, runScenario, parseScenarioSteps } from "lecodes-renderer/headless"

const s = await createSession(js, { width: 390, height: 844 })
await s.tap("login-button")     // by `name` (→ exact text → substring), real pointer pipeline
await s.type("email", "[email protected]")  // focus + onChange, like the simulated keyboard
await s.scroll(300, "feed")     // px along the scrollable's axis
await s.wait(500)               // real time: app timers fire; animation + game clocks advance
await s.frames(30)              // 30 fixed 1/60 steps, no sleeping: setLoop + 2D physics + 3D core tick
await s.drag("hero", { x: 300, y: 200 }, { ms: 300 })  // press, interpolated moves, release
await s.hold("charge", 800)     // long press
await s.key("ArrowRight", 500)  // hold a physical key (Input.key) for 500 ms of game time; keyDown/keyUp too
s.camera("iso")                 // 3D inspection camera for the scene3d report: iso|top|front|side|back
s.camera({ at: [5, 5, 5], lookAt: [0, 0, 0], fov: 40 }); s.camera("off")   // …or fixed; app camera untouched
s.json(); s.text()              // semantic frame (screen + widget overlays [+ scene2d/3d])
await s.png()                   // pixels (needs @napi-rs/canvas; raster: "auto"|true|false)
await s.png({ clip: "card" })   // one component / region (a node name or { x, y, width, height })
s.logs()                        // the project's console lines ("log: score 3")
s.screenName(); s.value("email")
s.dispose()

Game time is deterministic: setLoop handlers (dt in seconds), the 2D engine's early → physics → animation → late phases and the 3D core advance in fixed 1/60 steps, only when a verb moves the clock — never on a wall clock. 3D physics is the real thing: creator-physics' wasm (JoltPhysics, the same simulation the native hosts run) steps bodies, character controllers, triggers, vehicles and ragdolls, with the SDK's updateFixed between the substeps; the JSON snapshot reports the simulated positions next to the configured shapes (physics per node, physics at scene level for the world). Input.key reads the keys the session holds. Pointers the UI leaves unclaimed reach the 2D scene's touch system (_creator2d.setOnPointer) for the whole gesture, so sprite taps and ev.track() drags work headless.

The scenario engine executes a JSON step list against a session — the format behind lecodes render --script <file> and lecodes test:

{ "steps": [
  { "expect": { "screen": "home", "text": "Taps: 0" } },
  { "tap": "go", "expect": { "screen": "second" }, "screenshot": "02.png" },
  { "type": "email", "text": "[email protected]" },
  { "expect": { "value": { "target": "email", "equals": "[email protected]" } } }
] }

One action per step (tap / type+text / fill (bulk-fill empty inputs) / scroll[+by] / wait / frames / drag+to[+ms] / hold+ms / key+ms / keyDown / keyUp / view[+fov] / camera (3D inspection camera: { "view": "iso" }, { "camera": { "at", "lookAt", "fov"? } }, "off")), then expect (screen, node/notNode, text/notText, value, log — semantic, never pixels), then outputs (screenshot [+clip] / json). The first failing step stops the run and dumps the failing frame (failure.json + failure.png) so a red step explains itself. A scenario's envelope can carry fixtures (URL pattern → { status?, json?, text? } canned fetch responses — or a path to a JSON file of them), which makes backend-gated screens reachable headless. lecodes test additionally derives flow tests from a design's meta.json activator edges (auto-loading the project's tests/fixtures.json) — the app keeps the design's names on interactive elements and names screen roots with design screen ids, and the designed navigation becomes a regression suite.

Test

bun test

Scope

Implemented: screen, box, row, column, text, image, button, input/textarea, scrollable, widget, vlist, pager (tabs + per-tab stacks — only the visible page mounts; commands reconcile instantly), video (placeholder), plus pointer input, router (screen history), animations (animateTo/animateFrom + press/focus/responsive layers), gradients, borders, transforms, setSourceRect, tint, background image, registerFont, overflow:hidden, and a simulated keyboard (the UI is re-laid-out above the band, not scaled). Still deferred: toast, scrollbar visuals, true vlist windowing, radial/conic gradients, real video playback. See COMPONENTS.md for the per-component status and ARCHITECTURE.md for the design + C++-port mapping.