@videoflow/renderer-browser

Render VideoFlow videos to MP4 entirely in the browser — no server, no upload, no backend. Built on WebCodecs, MediaBunny, and a per-layer rasterization pipeline that pushes encoding to a Web Worker so the page stays responsive.

Live demo: videoflow.dev/playground · Renderers docs: videoflow.dev/renderers

Why use this package?

• Real MP4 files generated client-side — H.264 video + AAC (or Opus) audio, muxed by MediaBunny.
• Zero server cost. The user's device does the work; you keep their footage on-device.
• Worker-based encoding. Frame rasterization stays on the main thread (SVG <foreignObject> decode requires DOM), but encoding/muxing run in a dedicated Worker — UI stays smooth.
• Tier-based per-layer rasterization. Static / simple-transform layers paint with a direct drawImage fast path; complex layers go through a cached SVG rasterizer.
• WebGL effect compositor. Layers with effects are piped through a ping-pong shader pipeline.
• Audio sub-mixes for groups. A $.group(...) whose children produce audio is rendered as its own buffer first, then placed on the parent timeline — group-level volume / pan / pitch / mute / fade transitions apply to the sub-mix as a whole.

This is the export renderer. For interactive playback / scrubbing in the same browser, pair it with @videoflow/renderer-dom — they share the same transition + effect registries, so live preview and exported MP4 always agree.

Installation

npm install @videoflow/core @videoflow/renderer-browser

Quick Start

import VideoFlow from '@videoflow/core';
import VideoRenderer from '@videoflow/renderer-browser';

// 1. Define a video
const $ = new VideoFlow({ width: 1920, height: 1080, fps: 30 });
const title = $.addText({ text: 'Hello!', fontSize: 6, color: '#fff' });
title.fadeIn('1s');
$.wait('3s');
title.fadeOut('1s');

// 2. Compile to JSON, render to an MP4 Blob
const json = await $.compile();
const blob = await VideoRenderer.render(json);

// 3. Download
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = 'hello.mp4';
a.click();
URL.revokeObjectURL(a.href);

You can also bypass compile() and let VideoFlow auto-detect the environment:

const blob = await $.renderVideo();   // → Blob in the browser

API

VideoRenderer.render(videoJSON, options?)

Static one-shot — creates a renderer, exports an MP4, and tears everything down.

const blob = await VideoRenderer.render(videoJSON, {
  signal: controller.signal,            // AbortSignal — cancel mid-encode
  onProgress: (p) => console.log((p * 100).toFixed(1) + '%'),
  worker: true,                         // default; pass false to encode on the main thread
});

Options

| Option | Type | Description | | --- | --- | --- | | signal | AbortSignal | Cancel the encode mid-flight | | onProgress | (p: number) => void | Called with 0..1 during encode | | worker | boolean | Encode in a dedicated Worker (default true). Set false for environments without Worker support |

Returns: Blob (video/mp4).

VideoRenderer.renderFrame(videoJSON, frame)

const canvas = await VideoRenderer.renderFrame(videoJSON, 30);  // OffscreenCanvas

VideoRenderer.renderAudio(videoJSON)

const audioBuffer = await VideoRenderer.renderAudio(videoJSON); // AudioBuffer | null

Returns null when the project has no audio layers.

Instance API

For long-lived previews (re-rendering many frames, sharing one rasterizer cache, …) use the constructor + captureFrame() / exportVideo():

import BrowserRenderer from '@videoflow/renderer-browser';

const renderer = new BrowserRenderer(videoJSON);
try {
  for (let f = 0; f < total; f++) {
    const offscreen = await renderer.captureFrame(f);
    // …consume the OffscreenCanvas…
  }
} finally {
  renderer.destroy();
}

How it works

1. Layer mounting. Each layer becomes a DOM element in an off-screen [data-renderer] container. CSS handles transforms, filters, blend modes, fonts, and mix-blend-mode blending.
2. Per-frame property pass. Every layer's interpolated properties at the current frame are written as inline CSS / custom properties.
3. Tier-based rasterization (LayerRasterizer):
   • Tier 1 — simple transform + no filters/borders/shadows → straight drawImage from the layer's source bitmap onto the destination canvas.
   • Tier 3 — anything else (rotation, 3D, filters, text, shapes, effects-bearing layers) → rasterized through an SVG <foreignObject>, cached per layer until the resolved props change.
4. Effect pipeline. Layers with effects are piped through a WebGLEffectCompositor (ping-pong FBOs) before composite.
5. Composite onto the final canvas. Layers paint in sorted track order with their blendMode applied via globalCompositeOperation. Groups composite their children onto a private project-sized surface first, then drop that surface onto the parent.
6. Audio mix. An OfflineAudioContext mixes every audio-bearing layer (recursing through groups). volume/pan keyframes drive AudioParam automation; pitch is decoupled from speed via an offline granular pitch shifter; mute short-circuits the source.
7. Encode + mux. Frames and audio are fed into a Worker-resident MediaBunny pipeline (WebCodecs VideoEncoder / AudioEncoder), which produces the final MP4 buffer.

Performance notes

Video decoding

Video layers decode through WebCodecs via an internal VideoFrameSource that adapts to the access pattern instead of seeking per frame. A currentTime seek re-decodes from the preceding keyframe, so the old path cost O(frames × GOP) — 207.7 ms/frame on a 1080p clip with a stock 250-frame GOP, versus 15.6 ms/frame decoding the same frames sequentially. Forward playback (every export) decodes each packet at most once; reverse playback (speed < 0) decodes forward windows and serves them backwards.

Nothing to configure. Sources whose codec has no WebCodecs decoder on the current platform fall back to the previous <video> element path, including its smooth-playback mode.

renderer.enableElementCapture(scale?, mode?)

Switches the renderer to Chromium's "HTML in canvas" (drawElementImage) as its single rasterization primitive. There is no mixing: with the API available <foreignObject> leaves the pipeline entirely; without it, everything runs through <foreignObject> exactly as before. It is a polyfill relationship, not a per-layer choice.

What that removes, per frame:

| <foreignObject> has to | element capture | | --- | --- | | re-fetch @font-face rules and base64 them into the SVG | fonts already loaded | | toDataURL() every nested <canvas> — a PNG encode per frame | bitmaps already there | | inline the whole renderer stylesheet | CSS already applies | | clone the subtree, XMLSerializer, percent-encode, img.decode() | one draw call |

The font embedder is not merely bypassed — it is never invoked, so no font is fetched or encoded for rasterization at all.

Tier 1 is untouched. An image or video layer under a plain translate/scale with no filters, borders, shadows or radii is still blitted straight onto the target and never rasterized by either primitive. Element capture only replaces tier 3.

Measured end-to-end on real server exports (wall clock, encoding included):

| scene | <foreignObject> | element capture | | | --- | --- | --- | --- | | 09-effects | 970.0 ms/frame | 480.3 ms/frame | 2.0× | | 10-groups | 218.3 ms/frame | 140.4 ms/frame | 1.55× |

Per-frame composite time at 1080p:

| scene | rasterizer | element capture | | --- | --- | --- | | basic text | 49.8 ms | 10.8 ms | | image background | 334.1 ms | 11.8 ms | | transitions (10 layers) | 76.7 ms | 32.1 ms | | groups | 272.7 ms | 0.3 ms |

Output is pixel-identical, WebGL effects included. Resolves false — and changes nothing — when the API is unavailable, which is the normal case in a browser tab: it needs --enable-blink-features=CanvasDrawElement at launch. @videoflow/renderer-server passes that flag and enables this automatically (elementCapture: false forces it off, true forces 'force' mode, and elementCaptureScale sets the supersample factor).

One caveat, and it is the reason for both parameters: painting the live DOM bypasses the rasterizer's scale/position latch, so Chrome's glyph grid-snapping comes back — a quarter of a device pixel across, a WHOLE one down. A tween slower than that quantum freezes for several frames and then jumps. Measured on scale: 1 → 1.03 over 4s, mean frame-to-frame jerk of the sub-pixel side ink edges:

| path | text layer | html component | | --- | --- | --- | | latched rasterizer | 0.013 px (5/119 frozen) | 0.017 px (5/119) | | element capture, 1x | 0.197 px (72/119) | 0.199 px (71/119) | | element capture, 2x | 0.050 px (23/119) | 0.051 px (24/119) |

No CSS property changes this — will-change, contain: paint, opacity: .999, filter, text-rendering: geometricPrecision and <svg><text> all measure the same 0.22 px. The quantum is defined in DEVICE pixels, so the only lever is making a device pixel smaller:

• mode is the cure, and the only mitigation on by default. 'auto' scans the project and declines element capture when some DOM layer's animated scale / position moves slower than 1 / scale px per frame — a "life push" is ~0.2 px/frame — so those projects keep the latch. 'force' takes the speed regardless. Only projects with sub-grid DOM motion pay; everything else still gets the table above.
• scale (default 1 — off; max 4) supersamples the capture and downsamples to the frame, dividing the quantum by the factor: 1x → 0.208 px jerk, 2x → 0.050, 3x → 0.026, 4x → 0.004, for scale² capture pixels. It is off because the cost is quadratic and at 2x it more than consumed the speedup (113.8 ms/frame vs 95.9 for the rasterizer, against 50.2 at 1x), while only halving the vertical snap. Raise it when very slow type motion is the point and you would rather pay pixels than fall back.

Note that enabling it moves the renderer container on-screen, because drawElementImage silently yields a blank frame for an element parked off-screen. That is free in a headless export; in a live page the project will be painted over the document for the duration.

External layer types

Every renderer instance owns its own layer-type registry, seeded with the seven built-ins (text, captions, image, video, audio, shape, group). Register your own type on an instance — no fork, no patch, no global state:

import BrowserRenderer, {
  RuntimeVisualLayer,
  type LayerTypeDescriptor,
} from '@videoflow/renderer-browser';

class RuntimeCustomLayer extends RuntimeVisualLayer {
  async generateElement(): Promise<HTMLElement | null> {
    if (this.$element) return this.$element;
    const el = document.createElement('div');
    el.setAttribute('data-element', 'custom');
    el.setAttribute('data-id', this.json.id);
    this.$element = el;
    return el;
  }
}

const renderer = new BrowserRenderer(videoJSON);
renderer.registerLayerType('custom', {
  runtime: RuntimeCustomLayer,
  propertiesDefinition: CustomLayer.propertiesDefinition,
});
const blob = await renderer.exportVideo();

Any layer in the VideoJSON with type: 'custom' now renders through RuntimeCustomLayer — including layers nested at any depth inside a group, since groups build their descendants through the same registry.

Registration lifecycle

| Renderer | Register after | Register before | | --- | --- | --- | | BrowserRenderer | construction | the first renderFrame() / captureFrame() / renderAudio() / exportVideo() | | DomRenderer | construction | the first loadVideo() | | ServerRenderer | construction | the first renderVideo() / renderFrame() / renderAudio() |

BrowserRenderer defers creating its runtime layers until the first of those calls, which is exactly what keeps the registration window open. Registering afterwards throws rather than silently rebuilding live layers (which would discard their loaded media and mounted DOM).

• Duplicate registration replaces the previous descriptor — including for built-in types, so you can override text on one renderer instance.
• Registries belong to instances. Two renderers on the same page can map the same type name to different implementations without interfering.
• Unknown types throw a descriptive error naming the type and the renderer, rather than silently rendering nothing.

Descriptor

type LayerTypeDescriptor = {
  runtime: RuntimeLayerConstructor;                    // new (json, fps, width, height, renderer)
  propertiesDefinition: Record<string, PropertyDefinition>;
};

Both halves are registered together, so overriding a type replaces its runtime and its property definitions. Property lookups go through renderer.getPropertyDefinition(layerType, prop?).

Registry API

| Method | Description | | --- | --- | | renderer.registerLayerType(type, descriptor) | Register or replace a type on this renderer | | renderer.getLayerType(type) | The registered descriptor, or undefined | | renderer.listLayerTypes() | All registered type names, built-ins included | | renderer.createRuntimeLayer(layerJSON) | Instantiate the registered runtime class for a layer |

Exported for building external types: RuntimeBaseLayer, RuntimeVisualLayer, RuntimeTextualLayer, RuntimeMediaLayer, RuntimeLayerConstructor, LayerTypeDescriptor, LayerTypeRegistry, createBuiltinLayerTypeRegistry.

Rasterization hooks

RuntimeBaseLayer exposes two overridable hooks that LayerRasterizer calls, so a custom layer can control caching and the DOM it rasterizes without touching the rasterizer:

class RuntimeCustomLayer extends RuntimeVisualLayer {
  // Fold your own content revision into the raster cache key when the layer's
  // pixels aren't fully determined by its VideoJSON properties.
  getRasterCacheKey(props: Record<string, any>): string {
    return `${this.contentRevision}:${super.getRasterCacheKey(props)}`;
  }

  // Materialize specialised DOM for the `<foreignObject>` raster pass.
  // Default: clones `$element`, inlining every `<canvas>` as an `<img>`.
  async createRasterClone(): Promise<HTMLElement | null> {
    return super.createRasterClone();
  }
}

Transitions

Built-in transition presets (the same library used by @videoflow/renderer-dom) auto-register on import. See the core README → Transitions for the full categorised table and the signed-p contract.

Custom transitions

BrowserRenderer.registerTransition() writes to a registry shared with DomRenderer — register once, works in both export and live preview.

import BrowserRenderer from '@videoflow/renderer-browser';

BrowserRenderer.registerTransition('spinIn', (p, properties, params, ctx) => {
  const t = 1 - Math.abs(p);                     // 0 at edges, 1 at rest
  properties.rotation = (properties.rotation ?? 0) + (1 - t) * (params.angle ?? 360);
  properties.opacity  = (properties.opacity  ?? 1) * t;
  return properties;
}, {
  defaultEasing: 'easeOut',
  layerCategory: 'visual',                       // 'all' | 'visual' | 'audio' | 'textual'
});

The function receives:

• p — signed progress in [-1, +1], already eased per the layer's easing. -1 is the start of transitionIn, 0 is rest, +1 is the end of transitionOut.
• properties — the layer's resolved properties at this frame. Mutate in place or return a new object.
• params — values from the layer's transitionIn.params / transitionOut.params.
• ctx — { seed, frame, fps, projectWidth, projectHeight } for deterministic per-layer randomness and aspect-aware geometry.

Set injectsEffects: true if your preset pushes synthetic effects onto properties.__effects — the renderer keeps the effect overlay mounted across the layer's lifetime so the WebGL pipeline always engages.

GLSL effects

Built-in effects (chromaticAberration, pixelate, vignette, rgbSplit, invert, bloom, colorCorrection, frostedGlass, lightSweep, gaussianBlur, motionBlur, noiseDissolve, …) are auto-registered on import. Reference them by name from a layer's effects property; animate any param via a dot-path key.

Custom effects

import BrowserRenderer from '@videoflow/renderer-browser';

BrowserRenderer.registerEffect(
  'glitchShift',
  `
vec4 effect(sampler2D tex, vec2 uv, vec2 resolution) {
  vec2 shifted = uv + vec2(u_amount * sin(uv.y * 40.0), 0.0);
  return texture2D(tex, shifted);
}`,
  {
    amount: { type: 'float', default: 0.02, min: 0, max: 0.1, animatable: true },
  },
);

The GLSL snippet defines a single vec4 effect(sampler2D tex, vec2 uv, vec2 resolution). Each declared param becomes a u_<name> uniform. The compositor wraps the snippet with the precision/uniform/varying boilerplate at registration time.

Param types: 'float', 'int', 'bool', 'vec2', 'vec3', 'vec4', 'color' (CSS colour string → vec4).

End-to-end example: an export button

<button id="exportBtn">Export Video</button>
<progress id="prog" max="1" value="0"></progress>

<script type="module">
  import VideoFlow from '@videoflow/core';
  import VideoRenderer from '@videoflow/renderer-browser';

  document.getElementById('exportBtn').addEventListener('click', async () => {
    const $ = new VideoFlow({ width: 1280, height: 720, fps: 30 });

    $.addImage(
      {
        fit: 'cover',
        effects: [
          { effect: 'vignette', params: { strength: 0.6 } },
          { effect: 'chromaticAberration', params: { amount: 0.003 } },
        ],
      },
      { source: './photo.jpg', sourceDuration: '5s' },
    );

    const title = $.addText(
      { text: 'Made with VideoFlow', fontSize: 5, fontWeight: 800 },
      {
        startTime: '0.5s',
        sourceDuration: '4s',
        transitionIn:  { transition: 'slideUp', duration: '600ms' },
        transitionOut: { transition: 'fade',    duration: '500ms' },
      },
    );
    $.wait('5s');

    const json = await $.compile();
    const blob = await VideoRenderer.render(json, {
      onProgress: (p) => { document.getElementById('prog').value = p; },
    });

    const a = document.createElement('a');
    a.href = URL.createObjectURL(blob);
    a.download = 'export.mp4';
    a.click();
    URL.revokeObjectURL(a.href);
  });
</script>

Notes & requirements

• WebCodecs. Required for video encoding. Available in Chrome / Edge / recent Firefox / Safari 17+. The library probes for AAC support and falls back to Opus when AAC isn't available (notably on Linux Chrome). If neither is available the audio track is dropped and a warning is emitted — the video still encodes.
• Cross-origin sources. Video / image / audio sources must be CORS-readable for decode() / decodeAudioData() to succeed. Same-origin or blob-URL sources always work.
• Bundlers. The encoder Worker is bundled inline as a Blob URL, so no special bundler config is needed (esbuild / Vite / webpack all just work).

Related packages

• @videoflow/core — Define and compose videos programmatically
• @videoflow/renderer-dom — Live preview / scrubbable playback in the browser
• @videoflow/renderer-server — Render to MP4 on Node.js

Resources

• Renderers docs
• Live playground
• React Video Editor

License

Apache-2.0