Remotion MCP Server

MCP (Model Context Protocol) server that exposes Remotion's video composition and rendering APIs as tools for Claude and other MCP clients.

Features

• Manifest Loading: Load and validate CaptureManifest JSON files for video rendering
• Video Rendering: Render demo videos from captured browser sessions to MP4/WebM
• Render Progress: Track render job progress with status polling
• Preview Server: Start Remotion Studio for real-time composition preview
• Transitions: Built-in support for fade, slide, wipe, flip, and clockWipe transitions

Installation

npm install remotion-mcp

Or run directly with npx:

npx remotion-mcp

Configuration

No API key required - Remotion runs locally.

System Requirements

• Node.js 18+
• FFmpeg (for video encoding)
• Chrome/Chromium (for rendering)

Available Tools

| Tool | Description | |------|-------------| | remotion_load_manifest | Load and validate a CaptureManifest JSON file | | remotion_render_video | Render a demo video from a CaptureManifest | | remotion_get_render_status | Get render job progress and status | | remotion_preview | Start Remotion Studio preview server | | remotion_list_effects | List available transitions and effects |

Usage with Claude Code

Option 1: Direct Configuration (.mcp.json)

Add to your .mcp.json:

{
  "mcpServers": {
    "remotion": {
      "command": "npx",
      "args": ["-y", "remotion-mcp"]
    }
  }
}

Option 2: mcp-proxy Integration

For projects using mcp-proxy for unified tool access, add to your .mcp-proxy.json:

{
  "childServers": {
    "remotion-mcp": {
      "command": "node",
      "args": ["./packages/remotion-mcp/dist/index.js"],
      "transport": "stdio",
      "env": {
        "NODE_ENV": "production"
      },
      "idleTimeout": 300000
    }
  }
}

Then verify tools are discoverable:

search_tools({ query: "remotion" })

Should return all 5 Remotion tools:

• remotion_load_manifest
• remotion_render_video
• remotion_get_render_status
• remotion_preview
• remotion_list_effects

Example Workflow

1. List available effects

remotion_list_effects()

Returns available transitions (fade, slide, wipe, etc.) and timing functions.

2. Load a capture manifest

remotion_load_manifest(
  manifest_path="./capture-manifest.json"
)

Returns parsed manifest with metadata about screenshots, actions, and cursor paths.

3. Start rendering

remotion_render_video(
  manifest_path="./capture-manifest.json",
  output_path="./output/demo.mp4",
  codec="h264",
  fps=30
)

Returns: { "render_id": "render-1234567890" }

4. Check render progress

remotion_get_render_status(
  render_id="render-1234567890"
)

Returns status, progress percentage, and estimated time remaining.

5. Preview before rendering (optional)

remotion_preview(
  manifest_path="./capture-manifest.json",
  port=3000
)

Opens Remotion Studio at http://localhost:3000 for real-time preview.

Tool Parameters

remotion_load_manifest

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | manifest_path | string | Yes | Path to the capture-manifest.json file |

remotion_render_video

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | manifest_path | string | Yes | Path to the capture-manifest.json file | | output_path | string | Yes | Output path for the rendered video | | codec | string | No | Video codec: h264, h265, vp8, vp9 (default: h264) | | fps | number | No | Frames per second (default: 30) | | concurrency | number | No | Parallel render processes (default: half CPU cores) |

remotion_get_render_status

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | render_id | string | Yes | The render job ID from remotion_render_video |

remotion_preview

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | manifest_path | string | Yes | Path to the capture-manifest.json file | | port | number | No | Port for preview server (default: 3000) |

remotion_list_effects

No parameters required.

CaptureManifest Format

The CaptureManifest is a JSON file produced by browser capture tools (Playwright) that describes a recorded session. This is the bridge between the capture phase and Remotion composition.

{
  "metadata": {
    "url": "https://example.com",
    "viewport": { "width": 1920, "height": 1080 },
    "totalDurationMs": 15000,
    "capturedAt": "2026-01-28T10:30:00.000Z"
  },
  "screenshots": [
    {
      "id": "s1",
      "path": "screenshots/step-001.png",
      "timestamp": 0,
      "dimensions": { "width": 1920, "height": 1080 },
      "label": "initial"
    },
    {
      "id": "s2",
      "path": "screenshots/step-002.png",
      "timestamp": 1000,
      "dimensions": { "width": 1920, "height": 1080 }
    }
  ],
  "actions": [
    {
      "id": "a1",
      "type": "click",
      "timestamp": 500,
      "duration": 300,
      "target": {
        "selector": "#login-btn",
        "boundingBox": { "x": 100, "y": 200, "width": 80, "height": 40 }
      },
      "screenshotBefore": "s1",
      "screenshotAfter": "s2"
    },
    {
      "id": "a2",
      "type": "type",
      "timestamp": 1500,
      "duration": 800,
      "target": {
        "selector": "#email-input",
        "boundingBox": { "x": 100, "y": 300, "width": 200, "height": 40 }
      },
      "data": { "text": "[email protected]" },
      "screenshotBefore": "s2",
      "screenshotAfter": "s3"
    }
  ],
  "cursorPaths": [
    {
      "actionId": "a1",
      "points": [
        { "x": 960, "y": 540 },
        { "x": 500, "y": 300 },
        { "x": 140, "y": 220 }
      ],
      "durationMs": 500
    }
  ],
  "annotations": [
    {
      "text": "Click the login button",
      "timestamp": 400,
      "duration": 2000,
      "position": { "x": 200, "y": 180 },
      "style": "callout"
    }
  ]
}

Action Types

| Type | Required Fields | Description | |------|-----------------|-------------| | click | target | Click on an element | | hover | target | Hover over an element | | type | target, data.text | Type text into an input | | scroll | data.scrollDelta | Scroll the page | | navigate | data.url | Navigate to a URL | | wait | - | Wait for a duration |

Annotation Styles

| Style | Description | |-------|-------------| | callout | Callout box with pointer | | caption | Caption text overlay | | highlight | Highlight/emphasis effect |

TypeScript Types

This package exports TypeScript interfaces for all manifest components:

import type {
  CaptureManifest,
  CaptureMetadata,
  Screenshot,
  Action,
  ActionType,
  ActionTarget,
  ActionData,
  CursorPath,
  Annotation,
  AnnotationStyle,
  Vector,
  BoundingBox,
  Viewport,
} from 'remotion-mcp';

Zod Schema Validation

For runtime validation, this package exports Zod schemas:

import {
  CaptureManifestSchema,
  parseManifest,
  safeParseManifest,
  formatValidationError,
} from 'remotion-mcp';

// Parse and validate (throws on error)
const manifest = parseManifest(jsonData);

// Safe parse (returns success/error result)
const result = safeParseManifest(jsonData);
if (result.success) {
  console.log(result.data.metadata.url);
} else {
  console.error(formatValidationError(result.error));
}

Available schemas:

• CaptureManifestSchema - Complete manifest validation
• CaptureMetadataSchema - Metadata object validation
• ScreenshotSchema - Screenshot entry validation
• ActionSchema - Action entry validation with type-specific rules
• CursorPathSchema - Cursor path validation
• AnnotationSchema - Annotation entry validation
• VectorSchema, BoundingBoxSchema, ViewportSchema - Geometry types

Supported Transitions

| Transition | Description | |------------|-------------| | fade | Opacity crossfade between scenes | | slide | Directional slide (left, right, top, bottom) | | wipe | Directional wipe between scenes | | flip | 3D flip transition | | clockWipe | Radial clock wipe |

Timing Functions

| Timing | Description | |--------|-------------| | linearTiming | Constant speed transition | | springTiming | Spring physics for natural motion |

Render Architecture

The rendering process uses a two-phase approach:

Phase 1: Bundling

• The Remotion project is bundled using @remotion/bundler
• The bundle is cached for subsequent renders
• Bundling typically takes 5-15 seconds on first run

Phase 2: Rendering

• Frames are rendered in parallel using headless Chrome
• Progress is tracked via onProgress callback
• Default concurrency is half of CPU cores
• Output is encoded using FFmpeg

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│ CaptureManifest │ ──▶ │  Remotion       │ ──▶ │   FFmpeg        │
│   (JSON)        │     │  Composition    │     │   Encoding      │
└─────────────────┘     └─────────────────┘     └─────────────────┘
        │                       │                       │
        │  Screenshots         │  Frame Images         │  Final Video
        │  Actions             │  (JPEG)               │  (MP4/WebM)
        │  Cursor Paths        │                       │
        ▼                       ▼                       ▼

Render Job Lifecycle

| Status | Description | |--------|-------------| | pending | Job created, waiting to start | | bundling | Webpack bundling Remotion composition | | rendering | Rendering frames with Chrome | | completed | Video successfully rendered | | failed | Render failed with error |

Programmatic Usage

You can use the RemotionClient directly in Node.js:

import { RemotionClient, getRemotionClient } from 'remotion-mcp';

// Using singleton client (recommended for bundle caching)
const client = getRemotionClient({ verbose: true });

// Load and validate manifest
const manifest = await client.loadManifest('./capture-manifest.json');

// Start render (returns immediately)
const renderId = await client.startRender(
  manifest,
  './output/demo.mp4',
  { codec: 'h264', fps: 30, concurrency: 4 }
);

// Poll for completion
const status = await client.pollRenderUntilComplete(renderId, {
  timeoutMs: 600000,     // 10 minutes
  pollIntervalMs: 2000,  // Check every 2 seconds
});

if (status.status === 'completed') {
  console.log('Video rendered to:', status.output_path);
} else {
  console.error('Render failed:', status.error_message);
}

Manual Status Polling

// Check status without waiting
const status = client.getRenderStatus(renderId);
console.log(`Progress: ${Math.round(status.progress * 100)}%`);

// Cancel a running render
client.cancelRender(renderId);

Render Options

interface RenderOptions {
  codec?: 'h264' | 'h265' | 'vp8' | 'vp9';  // Video codec
  fps?: number;                               // Frames per second (1-120)
  concurrency?: number;                       // Parallel render threads
  quality?: number;                           // JPEG quality (0-100)
  width?: number;                             // Override video width
  height?: number;                            // Override video height
}

Error Handling

All tools return structured error responses:

{
  "success": false,
  "error": "Manifest file not found: ./missing.json",
  "error_code": "FILE_NOT_FOUND",
  "details": {
    "manifestPath": "/path/to/missing.json"
  }
}

Common error codes:

• NOT_FOUND: Resource not found (manifest, render job)
• FILE_NOT_FOUND: Manifest file does not exist
• INVALID_JSON: Manifest file contains invalid JSON
• INVALID_MANIFEST: Manifest fails schema validation
• INVALID_OPTIONS: Invalid render options provided
• LOAD_ERROR: Error loading manifest file
• CANCELLED: Render was cancelled by user

Dependencies

This package uses:

• remotion 4.0.409 - Core video composition framework
• @remotion/renderer 4.0.409 - Video rendering engine
• @remotion/bundler 4.0.409 - Webpack bundling for compositions
• @remotion/transitions 4.0.409 - Transition effects library
• @modelcontextprotocol/sdk ^1.12.0 - MCP protocol implementation

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Watch mode
npm run build:watch

# Preview composition (requires manifest)
npm run preview

Testing

Test Structure

tests/
  client.test.ts          # Unit tests for RemotionClient
  integration/
    server-startup.test.ts   # Server startup verification
    tool-discovery.test.ts   # Tool listing and schema validation
    pipeline.test.ts         # Full workflow testing
    error-handling.test.ts   # Error scenario coverage
  e2e/
    full-pipeline.test.ts    # Browser capture + render (skipped in CI)
  fixtures/
    sample-manifest.json     # Valid test manifest
    sample-script.yaml       # Demo script for capture testing
    screenshots/             # Placeholder PNG files

Running Tests

# Run all tests (integration tests included)
npm test

# Run only unit tests
npm test -- tests/client.test.ts

# Run integration tests
npm test -- tests/integration/

# Run E2E tests (requires browser, skipped in CI)
npm test -- tests/e2e/

CI Configuration

E2E tests are automatically skipped when CI=true is set. Integration tests run in CI but have appropriate timeouts for MCP protocol communication.

Troubleshooting

Server won't start

1. Build not completed: Run npm run build first
2. Missing dependencies: Run npm install
3. Node.js version: Requires Node.js 18+

# Verify Node version
node --version  # Should be 18+

# Rebuild
npm run clean && npm run build

Render fails

1. Disk space: Rendering requires ~2GB temp space
2. Missing screenshots: Verify screenshot paths in manifest exist
3. FFmpeg not installed: Remotion uses FFmpeg for encoding

# Check FFmpeg
ffmpeg -version

# If missing, install:
# Ubuntu/Debian: sudo apt install ffmpeg
# macOS: brew install ffmpeg

Tools not discoverable

1. Server not registered: Check .mcp-proxy.json configuration
2. mcp-proxy not running: Start proxy with claude-workflow proxy start
3. Build outdated: Rebuild with npm run build

# Verify via mcp-proxy
search_tools({ query: "remotion" })

# Should return 5 tools starting with "remotion_"

Invalid manifest errors

1. Schema validation: Use remotion_load_manifest to validate before rendering
2. Missing fields: Ensure all required fields are present
3. Invalid references: Check that screenshot IDs match action references

Common validation errors:

• "At least one screenshot is required" - Empty screenshots array
• "URL must be valid" - metadata.url is not a valid URL
• "capturedAt must be a valid ISO datetime" - Invalid date format

Slow rendering

1. Increase concurrency: Use concurrency option (default is half CPU cores)
2. Lower quality: Use quality: 80 for faster encoding
3. Lower FPS: Use fps: 24 instead of 30

remotion_render_video({
  manifest_path: "./manifest.json",
  output_path: "./output.mp4",
  concurrency: 8,  // More parallel processes
  quality: 80,     // Lower JPEG quality
  fps: 24          // Lower frame rate
})

Cursor Visual Effects

The package provides enhanced cursor rendering components for polished tutorial videos.

CursorTrail

Renders ghost positions behind the cursor to show movement path. Useful for helping viewers follow fast cursor movements.

import { CursorTrail, CursorTrailConfig, TrailPosition } from 'remotion-mcp';

// Configuration options
const config: CursorTrailConfig = {
  trailLength: 20,           // Number of ghost positions (5-50)
  fadeFrames: 15,            // Frames for trail to fully fade
  color: 'rgba(255,255,255,0.8)', // Trail color
  maxOpacity: 0.6,           // Starting opacity for newest ghost
  ghostScale: 1.0,           // Size relative to main cursor
  enableColorGradient: false, // Enable color interpolation
  gradientEndColor: 'rgba(100,100,255,0.3)', // End color if gradient enabled
};

// Position history (typically built from cursor timeline or segments)
const positionHistory: TrailPosition[] = [
  { position: { x: 100, y: 200 }, frame: 0 },
  { position: { x: 150, y: 250 }, frame: 1 },
  // ...
];

// In your composition
<CursorTrail
  positionHistory={positionHistory}
  config={config}
  cursorSize={24}
/>

CustomCursor

Renders different cursor designs for branding and style consistency.

import { CustomCursor, CustomCursorConfig, CursorSkin } from 'remotion-mcp';

// Built-in skins
type CursorSkin = 'default' | 'pointer' | 'macos' | 'windows' | 'custom';

// Configuration options
const config: CustomCursorConfig = {
  skin: 'windows',           // Built-in skin type
  scale: 1.0,                // Scale factor
  color: 'white',            // Fill color (SVG skins only)
  strokeColor: 'black',      // Stroke color (SVG skins only)
  hotspot: { x: 0, y: 0 },   // Custom hotspot offset (optional)
};

// For custom cursor images
const customConfig: CustomCursorConfig = {
  skin: 'custom',
  customSrc: '/cursors/branded-cursor.png', // PNG or SVG
  hotspot: { x: 5, y: 5 },   // Tip position offset
};

// In your composition
<CustomCursor
  position={{ x: 500, y: 300 }}
  config={config}
  size={24}
  animationScale={1} // Scale from click animation
/>

Layer Architecture

These components integrate with the existing cursor system:

| Layer | Z-Index | Component | |-------|---------|-----------| | 2b | 999- | CursorTrail (ghost positions) | | 3 | 1000 | SmoothCursor / TimelineCursor / CustomCursor |

Use CursorTrail with any of the cursor components for enhanced movement visualization.

Stock Sound Effects

The package bundles six CC0/public domain audio files for automatic sound effects in demo videos. These are accessible via Remotion's staticFile() function during rendering.

Available Sounds

| Key | File | Description | Default Volume | |-----|------|-------------|---------------| | click | sfx/click.mp3 | Short click sound | 0.6 | | hover | sfx/hover.mp3 | Subtle hover tone | 0.3 | | whoosh | sfx/whoosh.mp3 | Page transition whoosh | 0.5 (via navigate) | | typing | sfx/typing.mp3 | Keystroke sound | 0.4 | | scroll | sfx/scroll.mp3 | Soft scroll feedback | 0.4 | | success | sfx/success.mp3 | Positive chime | N/A (standalone) |

Action Type Mapping

ACTION_SFX_MAP maps each ActionType to its corresponding sound:

| ActionType | Sound Effect | |-----------|-------------| | click | sfx/click.mp3 | | hover | sfx/hover.mp3 | | scroll | sfx/scroll.mp3 | | type | sfx/typing.mp3 | | navigate | sfx/whoosh.mp3 | | wait | null (no sound) | | capture-video | null (no sound) |

Usage

import {
  STOCK_SFX,
  ACTION_SFX_MAP,
  ACTION_SFX_VOLUMES,
  getSfxForAction,
  getDefaultVolume,
} from 'remotion-mcp';

// Get SFX path for an action type
const clickSound = getSfxForAction('click'); // 'sfx/click.mp3'
const waitSound = getSfxForAction('wait');   // null

// Get default volume for an action type
const volume = getDefaultVolume('click'); // 0.6

// Access all stock SFX paths
console.log(STOCK_SFX.whoosh); // 'sfx/whoosh.mp3'

// Use with Remotion's staticFile() in compositions
import { staticFile } from 'remotion';
const audioSrc = staticFile(STOCK_SFX.click); // Resolves to bundled file

Audio Processing Pipeline

The package includes professional audio processing utilities for voiceover and audio tracks.

Features

• Audio Normalization: EBU R128 compliant loudness normalization
• Noise Reduction: Background hiss/hum removal with configurable strength presets
• Filler Word Removal: AI-powered detection and silence replacement via OpenAI Whisper

Quick Start

import {
  normalizeAudio,
  reduceNoise,
  removeFillers,
  isFfmpegAvailable,
} from 'remotion-mcp';

// Check FFmpeg availability
if (await isFfmpegAvailable()) {
  // Normalize audio to broadcast standard
  const normalized = await normalizeAudio('/input.mp3', '/output', {
    targetLoudness: -24, // LUFS
    truePeak: -2,        // dBTP
  });

  // Reduce background noise
  const denoised = await reduceNoise('/input.mp3', '/output', {
    strength: 'medium', // 'light' | 'medium' | 'heavy'
  });

  // Remove filler words (requires OPENAI_API_KEY)
  const cleaned = await removeFillers('/input.mp3', '/output', {
    fillerWords: ['um', 'uh', 'like', 'you know'],
  });
}

Requirements

• FFmpeg: Must be installed and in PATH
• OpenAI API Key: Required only for filler word removal

For detailed documentation, see docs/audio-processing.md.

License

MIT