ui-reel-pipeline

Agentic pipeline that turns a one-line product brief into a polished animated UI showcase reel (app demos, feature walkthroughs) — ready for Instagram Reels, TikTok, or LinkedIn.

brief → [Script Agent · LLM] → storyboard.json
      → [TTS · edge-tts]     → scene voiceover clips (scene durations auto-fit)
      → [Renderer · Remotion] → video.mp4 (animated UI inside a phone mockup)
      → [Assembly · FFmpeg]   → final_reel.mp4

Installation

Requirements

| | | |---|---| | Node.js | ≥ 20.12 (node -v) | | Disk | ~1 GB — ~660 MB for the global install, plus ~220 MB (Chrome) and ~130 MB (build cache) written into your working directory on first render | | OS | Linux, macOS, Windows | | API key | not required — voiceover uses free edge-tts voices. Only --brief (LLM storyboard generation) needs one. | | Network | needed on first render (Chrome download) and for edge/Gemini TTS |

ffmpeg/ffprobe ship as static binaries inside the package, so there is nothing to install system-wide.

Install

npm install -g ui-reel-pipeline

Verify it resolved:

which ui-reel-pipeline          # or `where` on Windows
npm list -g --depth=0 ui-reel-pipeline

First run

Run from any directory — the package is fully self-contained and never writes to its own install location:

mkdir ~/my-reel && cd ~/my-reel
ui-reel-pipeline                    # renders the bundled sample storyboard

First run takes a few minutes: Remotion downloads Chrome, bundles the composition, then renders ~900 frames. Subsequent runs skip the download.

You get:

my-reel/
├── out/
│   ├── final_reel.mp4          ← the finished reel (video + voiceover)
│   ├── video.mp4               ← silent render
│   ├── audio/*.mp3             ← per-scene narration
│   └── storyboard.final.json   ← storyboard with durations fitted to the narration
├── .remotion/                  ← Chrome, downloaded once (safe to delete)
└── .cache/                     ← webpack cache (safe to delete)

API keys (optional)

Only needed for --brief, which generates a storyboard with an LLM. Create a .env in the directory you run from — real environment variables take precedence:

# .env — any one of these
GEMINI_API_KEY=...            # or OPENROUTER_API_KEY / ANTHROPIC_API_KEY
GEMINI_MODEL=gemini-flash-latest

ui-reel-pipeline --brief "Show how the Kaya app onboarding works"

Set GEMINI_API_KEY and the voiceover upgrades to Gemini TTS automatically; without it you get free edge-tts voices. See TTS providers for local options.

Register the MCP server (optional)

To let an AI agent build and render storyboards through tool calls, point your MCP client at the installed server:

// .mcp.json — replace <global> with the output of `npm root -g`
{
  "mcpServers": {
    "ui-reel": {
      "command": "node",
      "args": [
        "--import", "<global>/ui-reel-pipeline/node_modules/tsx/dist/loader.mjs",
        "<global>/ui-reel-pipeline/pipeline/mcp-server.ts"
      ]
    }
  }
}

Renders started this way run from your working directory, so output lands in ./out/ exactly as with the CLI.

Install from source (development)

git clone https://github.com/md-shadhin-mia/ui-reel-pipeline.git
cd ui-reel-pipeline
npm install
cp .env.example .env            # then fill in your LLM keys/models
npm run reel -- --storyboard storyboards/sample.json

.env.example documents every supported variable. .env is gitignored and loaded by both the CLI and the MCP server.

Troubleshooting

| Symptom | Cause / fix | |---|---| | Timed out after 25000 ms while trying to connect to the browser | Remotion's downloaded Chrome won't start on your machine. Point it at one you have: REMOTION_BROWSER_EXECUTABLE=/usr/bin/google-chrome | | Chrome download fails with an invalid-filename error | You're on an NTFS/exFAT mount, which rejects the ? in a Chrome filename. Use REMOTION_BROWSER_EXECUTABLE, or run from a native filesystem. | | ENOSPC: no space left on device during install | The install plus Chrome and build cache needs ~1 GB. npm cache clean --force usually reclaims plenty. | | command not found: ui-reel-pipeline | Your npm global bin directory isn't on PATH. It is $(npm prefix -g)/bin. |

Usage

The examples below use the repo-dev form. With a global install, substitute ui-reel-pipeline for npm run reel -- — the flags are identical.

Render the hardcoded sample storyboard (no API key needed):

npm run reel -- --storyboard storyboards/sample.json

Generate a storyboard from a brief with an LLM (Agent 1):

export GEMINI_API_KEY=...        # or OPENROUTER_API_KEY / ANTHROPIC_API_KEY
npm run reel -- --brief "Show how the Kaya app onboarding works"

Flags:

| Flag | Meaning | |---|---| | --brief "..." | Generate the storyboard with the script agent | | --storyboard path.json | Use an existing storyboard instead | | --provider openrouter\|gemini\|claude | LLM provider (auto-detected from which API key is set) | | --model <id> | Override the provider's default model | | --no-tts | Skip voiceover (silent render) | | --out <dir> | Output directory (default out/) |

Live-edit the composition in the browser: npm run studio.

Iterating on a reel

• Every run saves its resolved storyboard to out/<dir>/storyboard.final.json — edit that and re-render with --storyboard instead of regenerating with the LLM.
• Use npm run studio to scrub through scenes live while polishing; full renders take a few minutes.
• Same storyboard works for both platforms: flip meta.aspect between "9:16" (Reels/TikTok) and "16:9" (LinkedIn/YouTube) and re-render.

Providers

| Provider | Env key | Default model | Cost | |---|---|---|---| | OpenRouter | OPENROUTER_API_KEY | meta-llama/llama-3.3-70b-instruct:free | free tier (model list rotates — see openrouter.ai/models, override with OPENROUTER_MODEL) | | Gemini | GEMINI_API_KEY | gemini-flash-latest (alias — survives Google's model retirements) | generous free tier | | Claude | ANTHROPIC_API_KEY | claude-sonnet-5 | paid |

Auto-detect order when --provider is not given: OpenRouter → Gemini → Claude (or force with LLM_PROVIDER).

Templates, themes & animation presets (AI tool calls)

The pipeline ships a predefined library that AI agents apply via MCP tool calls. Register the server (see Register the MCP server) and these tools appear under the name ui-reel:

| Tool | What it does | |---|---| | list_templates | 10 storyboard blueprints: app-onboarding, feature-showcase, saas-metrics, ai-assistant, product-launch, before-after, testimonial, pricing, how-to, integration | | list_themes | 6 brand color themes: midnight, ocean, sunset, rose, forest, mono | | list_animation_presets | 16 entrance animations: fade-up, snappy, pop, slide-in-left, slide-in-right, slide-down, zoom-in, blur-in, bounce, flip-up, flip-in-right, rise, swing, scale-up, elastic, glitch | | create_storyboard | Build a storyboard from a template (productName, tagline, theme, features, stats, …) and save it | | customize_storyboard | Patch an existing storyboard: theme, presets (global or per scene), transitions, narration, callouts | | render_reel / render_status | Kick off a background render and poll it |

Example agent flow: create_storyboard {template: "ai-assistant", productName: "PilotDesk", theme: "ocean"} → customize_storyboard {scenes: [{id: "conversation", animationPreset: "bounce"}]} → render_reel.

The same presets are available to the LLM script agent (--brief) via scene.animationPreset, and the template builders can be used directly from code (src/library/templates.ts). Animation implementations live in src/components/animate.ts.

Storyboard schema

See src/schema.ts (zod, single source of truth) and storyboards/sample.json for a working example. Key knobs:

• meta.aspect: "9:16" (1080×1920 Reels/TikTok) or "16:9" (1920×1080 LinkedIn/YouTube)
• meta.frame: device framing for all scenes — "phone" mockup (app demos), "browser" window (web/SaaS), "full" slide card (educational/technical explainers); override per scene with scene.frame
• meta.animationStyle: overall feel — spring · snappy · cinematic · playful · elegant · techy · bold; sets the default entrance preset for scenes without an explicit animationPreset (descriptions in ANIMATION_STYLE_GUIDE, src/schema.ts; mapping in src/components/animate.ts)
• meta.brand: primary/accent colors + logo text baked into every frame
• scene.screen.layout: 15 layouts — app-style: hero · form · list · stats · chat; explainer (technical/educational): compare · diagram · code · terminal · timeline; showcase: chart · quote · pricing · notification · grid. The code layout takes its code text in screen.body. See storyboards/kitchen-sink.json for a working example of every new layout.
• scene.transition: fade · slide-left/right/up/down · wipe · flip · clock-wipe · iris · zoom-fade · none (transition into the scene)
• scene.callout: floating badge outside the phone (e.g. "Under 60 seconds")

Scene durations are treated as minimums — the pipeline extends them so the narration always fits.

TTS providers

Select with --tts <provider> or TTS_PROVIDER in .env:

| Provider | Where it runs | Setup | Notes | |---|---|---|---| | gemini (default when GEMINI_API_KEY is set) | Gemini API | none — reuses GEMINI_API_KEY | expressive voices via GEMINI_TTS_VOICE (Kore, Puck, Charon, Aoede, …); free-tier quota, preview models | | edge (default otherwise) | free Microsoft web API | none — bundled (msedge-tts) | best quality/zero setup; voice via meta.voice (e.g. en-US-AriaNeural) | | kokoro | local, fast on CPU | python3.12 -m venv .venv-tts && .venv-tts/bin/pip install kokoro soundfile (~330MB model auto-downloads on first use) | no internet needed at render time; speaker via KOKORO_VOICE (af_heart, am_adam, bf_emma, …) | | vibevoice | local, GPU recommended | in .venv-tts: pip install torch (CUDA build) + pip install git+https://github.com/microsoft/VibeVoice.git | most expressive; clones the voice from VIBEVOICE_SPEAKER_WAV; 1.5B model — CPU is impractically slow |

Local providers live in a separate .venv-tts on Python ≤3.13 (their deps don't build on 3.14 yet); the pipeline finds it automatically. They are the only providers that need Python — everything else (edge, gemini, rendering, assembly) is pure Node/npm.

Optional extras

• Music: drop assets/music.mp3 in the project root; it's looped under the voiceover at low volume.

Rendering notes

The renderer roots itself in your working directory and writes only there (see First run for the layout) — nothing is written into the global install, so the package works from a read-only or shared location.

Remotion downloads its own Chrome on first render. If that download or launch fails, point the pipeline at a browser you already have, via the environment or .env:

REMOTION_BROWSER_EXECUTABLE=/usr/bin/google-chrome ui-reel-pipeline

The path is existence-checked: if it's wrong, you get a warning and Remotion's own Chrome is used instead.

Extending

• New screen layout → add a component under src/components/screens/, register it in screens/index.tsx, extend LAYOUT_NAMES in src/schema.ts, and mention it in the script-agent prompt. (New transitions: TRANSITION_NAMES in schema + a case in presentationFor() in src/Reel.tsx.)
• Figma input / real screenshots → add an image-based layout that renders <Img> inside PhoneFrame.
• Whisper captions, brand intro/outro cards → extra steps in pipeline/run.ts.