playreel

Video as code, for the live web. Give your LLM the vocabulary of human website gestures — scroll, swipe, hover, type, zoom. Brainstorm with your agent; render deterministic 60 fps film of real pages; surgically edit the storyboard for easy updates.

A storyboard file in your repo, rendered against your actual product — cinematic post-production, every aspect ratio, re-renderable from CI on every release. If a browser renders it, playreel can film it.

Why

Every screen-recorded product video is stale the week after recording: the product ships a redesign, someone re-records by hand, or the video quietly rots. Teams that ship weekly effectively can't keep marketing video current.

And re-recording by hand is the good case. Every real-time recording approach — screen capture, Playwright/Cypress video, CDP screencast — produces variable frame rates and dropped frames, structurally: capture competes with rendering for the same real-time budget.

How

playreel treats the video as a build artifact:

• The storyboard is a plain JS file in your repo. Versions are commits, variants are parameters, different videos are different files. No schema language, no GUI, no cloud.
• Capture runs on a virtual clock. The page's Date/performance.now/rAF/timers are frozen and advanced exactly 1/60 s per screenshot; CSS animations are seeked per frame via the Web Animations API. Every frame is perfect regardless of machine speed — the technique Remotion uses to render React compositions, pointed at your actual live site.
• Interactions are authored physics, not recordings: momentum swipes that feed the page's own inertia, iOS-decay-curve scrolls, hover tours, real :hover states with a visible injected cursor.
• Post-production is ffmpeg in the same storyboard: crossfade chains, haloed titles keyed to timing markers, Ken Burns photo bursts, 16:9 / 9:16 / 4:5 presets from one storyboard.

What that looks like — a minimal but complete storyboard. This is the entire build; there is no other config. Input: any live URL. Output: an MP4.

import { record, assemble, decayScroll } from 'playreel';

const scenes = {
  hero: async (page, t) => {
    await page.goto('https://your-site.example', { waitUntil: 'load' });
    await t.start();                    // freeze the clock; capture begins
    t.mark('hero');                     // timing marker for overlays
    await t.cap(t.F(3.0));              // capture 3 s = 180 frames
  },
  features: async (page, t) => {
    await page.goto('https://your-site.example', { waitUntil: 'load' });
    await t.start();
    const ys = decayScroll(1200, { flicks: 1 });   // momentum flick, iOS decay curve
    await t.cap(ys.length, { act: (i) => ({ scrollY: ys[i] }) });
  },
};

// capture: one 60 fps frame dir per scene → out-desktop/hero/f000000.jpg …, plus timings.json (the marks)
await record({ scenes, format: 'desktop' });

// edit: white fade-in, hero, 0.4 s crossfade, features → promo-desktop.mp4 (3.0 s + features − 0.4 s overlap)
await assemble({
  name: 'promo', format: 'desktop', dir: 'out',
  timeline: [
    { scene: 'hero', filter: 'fade=t=in:st=0:d=0.5:color=white' },
    { scene: 'features', fade: 0.4 },
  ],
});

Overlays keyed to the marks, photo bursts, and multi-format runs are the same file growing — see the travel storyboard for the full-size version.

What makes the determinism hold on real-world pages (all handled by the engine):

• CSS animations/transitions run on the compositor clock, outside any JS clock fake — they are paused and seeked per frame, each released just before its natural end so transitionend/animationend still fire.
• Synthetic input runs in-page, in the same JS task as the animation sync, so triggered transitions can't leak real-time progress.
• setPointerCapture on synthetic pointer IDs is no-opped (libraries would throw).
• Real :hover needs real input: drive page.mouse between ticks and the hover transition syncs like any other animation.
• Lazy-loaded images get a pre-capture scroll pass (preloadByScroll).
• The native text caret blinks on the browser's UI clock — during typed scenes it is hidden and replaced by a caret blinked on the virtual clock.
• Page timers armed before capture starts fire at wall-clock-dependent frames (pre-capture time free-runs at real speed). Trigger scene state changes after t.start() — then they land on deterministic frames.
• Clicking a real link mid-capture navigates and kills the scene. To demo a click on a link, swallow it first: document.addEventListener('click', e => e.preventDefault(), true) — :active and click effects still render.
• <video>/<audio> elements play on the browser's real-time media clock — the engine pauses them and seeks currentTime per tick, so page videos advance in exact lockstep with the virtual clock (embedded players inside iframes remain real-time; record notes them). Sites with streaming media may also never fire load — navigate with waitUntil: 'domcontentloaded' plus a settle wait.
• SVG SMIL animation (<animate>, <animateTransform>, <animateMotion>, <set>) runs on the SVG document timeline, which getAnimations() does not report — so it needs its own pass: animated SVG roots are paused and seeked per frame like everything else. This matters for the pages SMIL is used on (data-viz, dashboards, animated logos, explainer graphics), and page JS that reads svg.getCurrentTime() to drive DOM gets the same instant the graphic is showing. SVG referenced as an image — <img src="…svg">, <object>, a CSS background-image — is a document you cannot script, so it stays on the real clock; inline the SVG to film it.
• Floating chat widgets (WhatsApp buttons, greeting bubbles) are site chrome, not content — hide them before capture (match a[href*="wa.me"] and the bubble's text, set display:none).
• Never wait on the page's own clock during capture. page.waitForFunction polls with rAF or timers — both frozen after t.start() — so it hangs until it times out. Use until(page, fn): it polls from Node, where real time still runs, and captures no frames while it waits.
• Live MediaStreams (getUserMedia, canvas.captureStream) have no seekable timeline, so they are exempt from media sync and keep running on the real clock. Paint a still frame into the source and every render comes out identical anyway.
• Every frame gets its own clock, and all of them are ticked together. An app running inside an <iframe> — the device-stage pattern, where the phone shell is drawn by the host page — advances in lockstep with the host instead of sitting frozen. That is what makes a two-device shot (one action, two screens reacting) a single capture rather than a composite.

Recording an app, not a marketing page, adds three moves:

• Point at things through the stage, not by hand. deviceStage().at() maps an element inside a scaled iframe to page coordinates and throws when the result falls outside the phone. A bad coordinate otherwise renders perfectly happily, with the cursor stranded in the background — a film that ships.
• Fake the device, keep the code path. Shim navigator.mediaDevices.getUserMedia in an init script to return a canvas.captureStream() painted with a still image. The app's own preview → drawImage → toBlob pipeline runs for real; nothing films the room, and the frames are identical every render.
• Gate the backend, don't fake the UI. A slow, paid, or destructive endpoint belongs behind page.route. Park the fulfilment on a promise the storyboard resolves and the loading state lasts an exact number of frames — real client code, scripted latency. Best practice: capture the real response once and replay that, so nothing on screen is invented.
• Drag-and-drop needs a ghost. Synthetic drags render no drag image, so append an absolutely-positioned element that follows mousemove, then dispatch dragover/drop with a DataTransfer carrying a real File.

Quickstart

Requires Node ≥ 20. Two dependencies: playwright and ffmpeg-static — no system installs. (ffmpeg-static bundles a GPL ffmpeg binary, invoked as a subprocess, not linked. The examples' title overlays use macOS font paths; adjust on other platforms.)

npm install playreel && npx playwright install chromium   # once

Copy the storyboard above into promo.mjs, replace https://your-site.example with your own site's URL, and run:

node promo.mjs        # → promo-desktop.mp4, 60 fps

Your first render is your actual product, not a demo page. Different videos = different storyboard files. Variants (format, language, campaign) = parameters. Outputs are gitignored build artifacts named <name>-<format>.mp4.

Workflow: from URL to finished cut

The fastest path is a coding agent driving playreel; the machine does research → storyboard → render → validate, the human judges taste. The loop that converges in hours:

1. Research. Before any storyboard: the agent reads the site (and press, docs, subpages) and codifies it in research.md (template) — including a facts table with source and checked-date per fact. Overlay/caption copy may only use rows from that table; stale rows get re-verified before a re-render. Facts drift — that's why this tool exists.
2. Storyboard. The agent proposes a numbered scene list — acts, durations, overlay copy, cards — for approval before building. Photo and people picks are resolved on a labeled contact sheet; for identifiable people, confirm consent explicitly. Decisions and their reasons go into a decision-log block in the storyboard header.
3. Generate. The agent writes the storyboard against the API and renders. Two card techniques beat burning text into images with ffmpeg: let the browser typeset — title/section/end cards as local HTML using the target site's own fonts and color tokens render brand-exact and animate with CSS under the virtual clock; and drive card content from the site's data — if the site has a data file (portfolio list, feature matrix), render the card from it and assert the on-card numbers against it at build time, so a re-render updates the video's facts. A third technique: variants are env parameters — language or campaign versions share the storyboard, with per-variant cards/captions and side-by-side output dirs (out-feed, out-feed-en), so every variant re-renders independently. And where a site's own localization lags, rewrite the missing strings in-page before capture (match only the source-language strings — the rewrite deactivates itself once the site catches up). Tuning is empirical: extract frames, fix what they show (overlay collisions, scroll landing points, a dark logo turning into a black box on a light tile) — two iterations per scene is normal. Deficit feedback on a draft converges; adjectives up front don't.
4. Validate (machine). Before showing a cut, check extracted frames against the deterministic rules: frame 0 must work as a share thumbnail (messengers thumbnail from the first frame and ignore poster metadata — never open on black); the last frame must work as a held poster (feeds freeze on it — put the CTA there, fadeOut: { d: 0 }); animated text must reach full opacity ≥ 1.5 s before its cut; audio length must cover video length. A built-in validate() pass is on the roadmap.
5. Review (human). Taste decisions — music, pacing — are made between 2–3 rendered candidates, not described in advance. Freeze approved cuts explicitly (archive/vN/, see Project structure) before iterating on the rest.
6. Extract learnings. After a production, the artifacts themselves carry what was learned: the decision log, the research file, the diff. Generic techniques worth sharing — a new recipe, a quirk on some framework, a validation rule — feed them back: open an issue or PR. playreel improves by accumulated production experience, and there's no telemetry; issues and PRs are the only channel.

Project structure

A video project is a directory your future self can re-render:

my-promo/
├── research.md          # evolving fact base — start here, before the storyboard
├── storyboard.mjs       # the program; header holds the decision log (picks + why)
├── assets/              # tracked sources (photos, fonts, cards); baked text/caption
│                        #   images regenerated by a checked-in script, never by hand
├── archive/v1/          # every delivered cut, frozen: renders + MANIFEST.md
│                        #   (date, storyboard git ref, music pick, why superseded)
├── out-<format>/        # frames — gitignored, but track timeline.json + timings.json
└── my-promo-<format>.mp4  # always the LATEST cut (gitignored)

Versions live in git (sources) and archive/ (outputs), never in filename suffixes. Binaries can stay untracked; the manifests are the tracked trace.

API

From index.js:

| Export | What it does | |---|---| | record({ scenes, format, formats, outRoot, fps, pointerTarget, css, only, launch, context }) | Runs scenes in headless Chromium under the virtual clock → one 60 fps JPEG frame dir per scene + timings.json markers. Scenes get (page, t): t.cap(n, {act}) captures frames, t.run advances time silently, t.mark records a labeled timestamp, t.F(s) converts seconds → frames. act(i) returns a synthetic-input payload: {pdown|pmove|pup|hover: [x,y], scrollY, wheel, click, center}. Scene defs can add localStorage seeding and a per-scene pointerTarget. launch/context pass through to chromium.launch / browser.newContext (device fakes, permissions, locale, timezone). | | assemble({ timeline, dir, name, format, output, fps, crf, fadeOut }) | One ffmpeg filter graph: scene frame dirs and { burst: [imgs] } Ken Burns bursts, crossfade chain, final fade (fadeOut: { d: 0 } to end on a held frame). Writes timeline.json (scene boundaries — useful for scoring). | | FORMATS / OUT_FORMATS | Viewport and encode presets: desktop 16:9 1920×1080, mobile 9:16 1080×1920 (real mobile viewport at 2× DPR), feed 4:5 1080×1350. | | swipeActs, silentDrag, decayScroll, hoverTour, preloadByScroll, easeInOut | Motion vocabulary: hand-flick with release velocity (feeds e.g. OrbitControls damping), silent pre-positioning, momentum scrolls, hover tours with dwells, lazy-load pre-scrolling. | | typeActs(text, { cps, jitter, seed }) | Human-cadence typing as a per-frame act plan for t.cap: variable inter-key delays, longer pauses after punctuation, seeded so re-renders are identical. Writes via the native setter (React/Vue controlled inputs see it) and fires key/input events; the native caret — which blinks on the browser's UI clock — is hidden and replaced by a caret on the virtual clock. Acts: {focus: sel}, {insert: text}. | | deviceStage(page, { origin, phones, allowFraming, fly }) | Films an installed app as an installed app: the phone shell is drawn by a shipped template, the real app runs in an iframe inside it, and the stage is served from the app's own origin so coordinates are shared. Returns at(sel) / atText(sel, text) — page coordinates of an element inside a phone, throwing if it maps off the screen — plus reveal, frame(i) and fly(). Two phones reacting to one tap is then a single capture. allowFraming strips X-Frame-Options/frame-ancestors for the capture only. | | tap(page, t, [x,y]), movePointer(page, [x,y]) | Glide-and-press, the gesture every product demo is made of, keeping the engine's pointer position true so the next glide starts where the last ended. | | cursor(page, opts), glide(page, t, [x,y], opts) | Engine-owned visible cursor that follows all pointer input and pulses on every press (the pulse is a CSS animation — deterministic under the virtual clock); eased glides of the real mouse, so genuine :hover/:active fire along the way. Given a Page, the cursor is injected into every frame and each copy hides itself when the pointer leaves its document — a cursor living only in the host freezes the moment it crosses into an iframe, because that is where mousemove is delivered. refreshCursor(target, opts) re-injects it after a document swap — a navigation takes the injected cursor with it, and a tap with no cursor films as nothing happening. | | pointerAt(page, [x,y]), pointerOf(page) | The engine's memory of where it last put the pointer, so glide starts where the last gesture ended. Set automatically by tap/movePointer; call pointerAt(page, null) when the pointer's position is no longer known (a new page or stage). Reading it back from the DOM does not work once the cursor lives inside a frame — hence the bookkeeping. | | zoomTo({ z, cx, cy, at, inD, hold, outD, format }) | Screen-Studio-style punch-in: eased zoom into a region, hold, eased release — applied as a scene filter, keyed to scene-relative seconds (pair with readTimings marks). Keep z ≤ ~1.4 at base resolution, or record the scene at a higher deviceScaleFactor. | | cardScene(page, t, file, seconds), loadPage(page, url, opts) | HTML-card scene in one call (local file → settle → capture); settled live-page load with the lazy-load pre-pass. | | until(page, fn, { timeout, every, what }) | Wait for an in-page condition during capture. page.waitForFunction polls on the page clock, which virtual time freezes; this polls from Node instead, so it resolves without capturing a frame. Use it after every click that triggers async work. | | drawTitle, fadeAlpha, readTimings | Overlay helpers: centered haloed drawtext chains, fade-in/out alpha expressions, timing-marker access. | | validate({ dir, fps, scenes, expect, video, audio }) | Post-render checks, because a render can finish green while the film is wrong: frames contiguous from f000000 (a gap silently truncates ffmpeg's input), none zero-byte or featureless (one flat colour is a page that never painted — expect.<scene>.allowFlat for a deliberate card), marks present and inside their scene, per-scene min/max seconds. With video: duration and decoded frame count against the timeline copy, plus audio continuity on a scored cut. Pass the storyboard's Object.keys(scenes) as scenes to catch a scene that never recorded at all. Throws naming every problem; soft: true returns the report instead. |

The storyboard is JS, so anything not covered by the API is a page.evaluate away.

playreel/music — score the cut

import { generate, mix, assertContinuous } from 'playreel/music';

AI score with zero new install weight: generate is an HTTP client to an ACE-Step server (backend: 'local', default http://127.0.0.1:8001, or 'hf-space' for the official Space's free GPU pool — needs npm install @gradio/client, respects HF_TOKEN). The ~13 GB model is a runtime service, never a dependency; mixing is the already-bundled ffmpeg.

| Export | What it does | |---|---| | generate({ prompt, duration, bpm, keyScale, candidates, backend, outDir }) | N candidate cues saved as wavs. Asks the model for 1.28× the film's length — it returns materially less content than requested, and a cue that falls short must be spliced, which costs the opening the brief was built on. Pick by ear, mix by number. | | mix({ video, cue, timeline, mode, bpm, tail, lufs, output }) | End-aligns the cue's final hit tail seconds before the last frame so the chord resolves on the close and rings through it (measure the tail per film); splices a short cue against itself on the bar grid, keeping the decay; two-pass loudnorm to lufs (default −14); muxes with the video stream copied, never re-encoded. mode: 'enter-late' starts the cue untouched at at seconds, for a film whose cold open should play dry. A cue that cannot cover the film fails loudly — padding a shortfall with silence is how a cut once shipped with a mute final third. | | assertContinuous(file, { window, floor }) | The gate mix runs on its own output: no window-second stretch may average below floor dB (defaults 4 s, −50). Cheap enough to run on anything about to be published. |

Examples

Four examples. The archetype is the first storyboard decision: what role does the site play in the video? It's a spectrum —

• examples/travel/ — site as subject: the site itself is the story, and the video demonstrates it (most product demos live here). Shown as a full reference storyboard: a 32 s promo of a live travel site with a WebGL globe — hand-swipe with damping glide, tooltip grid-scan for viewport-independent country targeting, per-scene localStorage seeding, three formats from one storyboard. Its picks.json is a trimmed sample whose paths point at the author's local photo files — read it as a reference, and point the picks at your own assets.
• examples/contacts-quick-capture/ — app as subject: the same archetype pointed at an application rather than a marketing page, where the story is the workflow, not the layout. Shown as a finished production: ▶ contacts-quick-capture-feed-scored.mp4, a 50 s demo of a local tool that turns a business card into a Google contact. Three input methods in one film — a paste, a file dragged in with a ghost following the pointer, and a webcam photo through a shimmed getUserMedia — then a punch-in on the fields the model labelled, and both save paths. The parse response is a real captured backend reply replayed through page.route, so the loading state is frame-exact and nothing on screen is invented.
• examples/lr-ventures/ — site as identity: the subject is the organization that owns the site, and the site is its identity system — brand tokens, copy, people, and structured data (firms, agencies, consultancies, personal brands). Shown as a finished production: ▶ lr-ventures-promo-feed-scored.mp4, a 28 s film for the angel-investment firm lr-ventures.de. Every card is typeset by the browser in the site's own fonts and colors; the portfolio wall renders from the site's data file, so a re-render updates the numbers; live captures appear only where the site is the proof.
• examples/travel-atlas/ — data as subject: the subject is an artefact the site generates rather than a page it serves, and the camera move is the demonstration. Shown as a finished production: ▶ atlas-descent-feed-scored.mp4, an 89 s descent through the Visual Atlas — 2,814 photographs placed by SigLIP embeddings and UMAP, where nothing was ever tagged. One continuous flight: the whole map, then seven regions, each descending until the photographs are large enough to look at. The route comes from a hierarchical taxonomy built from the same k-NN graph the layout uses, so the names on screen are the model's own. Labels are pinned to the map in image coordinates and scale with the zoom, growing past the frame as the camera goes inside them; the viewer itself is driven as a camera through viewport.zoomTo/panTo.
• examples/family-calendar/ — problem as subject: the film opens on the pain rather than the product, and the app appears as the answer to something the viewer has already felt. Shown as a finished production: ▶ family-calendar-story-feed-scored.mp4, a 74 s film for a household PWA. It opens on two problem cards: a family group chat and a packed work day with an empty evening, each a still blurred just enough that its shape reads without inviting reading, under one centred card asking a second-person question — the problem ("Running the family on group chat — and kids still miss things?"), then the wish ("Wishing kids had their own reminders — and adults their own calendar?"), which is exactly what the walkthrough then grants. The app is filmed live inside drawn phone shells, one colour per person; every screen is the real product against a throwaway database, and the film closes on that same work day — sharp this time — with the pickup now in it. The activity created on camera, the reminder and both calendar shots are all given one time constant, so the hour has been read more than once before it appears in his calendar.
• examples/nandri/ — site as source: the story lies beyond the site — the site supplies photos, facts and a few live moments for a narrative about the cause behind it (brand and cause films). Shown as a finished production: ▶ nandri-promo-feed-scored.mp4, a 60 s charity promo for nandrikinderhilfe.de (published with the charity's permission). Four-act narrative — captioned photo bursts, a slow eased scroll, a real :hover on the donate button, AI score end-aligned to the last frame. Recut on a board note to run slower for older viewers: the holds roughly doubled, but as much came from type size, contrast and shorter caption lines, and from dropping momentum scrolling — which is fastest exactly when the eye is still catching up.

For the finished-production examples, the sources stay with their owners — this repo carries the outcome; travel carries the code. Decide your archetype before writing the storyboard: it changes what you capture, what you compose, and what the video must never claim.

Roadmap

• npx playreel CLI wrapper
• record resume mode (skip scenes whose frames already exist, invalidating on scene-source change)
• GPU rendering for WebGL-heavy pages (SwiftShader is the current fallback)
• Burned-in captions

License

Apache-2.0 — includes an explicit patent grant; the playreel name is not licensed for use on derived works.