@projectyoked/expo-media-engine

A hardware-accelerated video composition and editing engine for Expo. Built on AVFoundation (iOS) and MediaCodec + OpenGL ES 2.0 (Android), it gives you a full editing pipeline — real-time preview, multi-track composition, filters, transitions, audio mixing, and export — without proprietary SDKs or per-minute billing.

Features

• Real-time preview — Native <MediaEnginePreview> view renders your composition at ~30 fps using the exact same pipeline as the export engine, so what you see is what you get
• Multi-track composition — Stack video, audio, image, and text tracks with per-clip transforms, timing, and blending
• 9 filters — Grayscale, sepia, vignette, invert, brightness, contrast, saturation, warm, cool — all with adjustable intensity
• 8 transitions — Crossfade, fade to black, slide (left/right/up/down), zoom in/out — with configurable duration
• Per-clip controls — Opacity, playback speed, source trimming, position, scale, rotation, resize mode (cover/contain/stretch)
• Audio mixing — Multiple audio tracks with per-clip volume, fade in/out, and keyframe-based volume automation
• Text & emoji overlays — Full styling: font size, bold, color, shadow, stroke, background pill, timed visibility
• Image overlays — Positioned, scaled, rotated, and timed image tracks
• Video stitching — Fast concatenation using mp4parser (Android) or passthrough (iOS), with transcoding fallback
• Video compression — Re-encode with H.264 or H.265, target bitrate/resolution, or quality presets
• Waveform generation — Normalized RMS amplitude array from any audio file
• Audio extraction — Pull audio from video to .m4a

Installation

npm install @projectyoked/expo-media-engine
npx expo prebuild

The current pre-release line (for example 1.0.0-alpha-4) is what npm installs by default. To stay on the legacy 0.1.x stable line, pin the version:

npm install @projectyoked/[email protected]

This package requires a native build. Expo Go is not supported — use a development build.

Linking from a monorepo or local path? Point Metro at a single react-native copy. If the package resolves its own tree, MediaEnginePreview can fail to register. Use withMediaEngineMonorepoResolver from this repo’s metro-preset.js — the example app’s metro.config.js shows the pattern.

Requirements

| | Minimum | |---|---| | Expo SDK | 49+ | | expo-modules-core | 1.0.0+ | | iOS | 13.4+ | | Android | API 21+ | | React Native | 0.64+ |

Quick Start

Export a composition

import MediaEngine from '@projectyoked/expo-media-engine';

const outputUri = await MediaEngine.composeCompositeVideo({
  outputUri: `${FileSystem.cacheDirectory}output.mp4`,
  width: 1080,
  height: 1920,
  frameRate: 30,
  quality: 'high', // 'low' | 'medium' | 'high', or set bitrate explicitly

  tracks: [
    {
      type: 'video',
      clips: [
        {
          uri: 'file:///path/to/clip-a.mp4',
          startTime: 0,
          duration: 5,
          filter: 'warm',
          filterIntensity: 0.6,
        },
        {
          uri: 'file:///path/to/clip-b.mp4',
          startTime: 4,      // overlaps clip-a by 1s for the transition
          duration: 5,
          transition: 'crossfade',
          transitionDuration: 1,
        },
      ],
    },
    {
      type: 'audio',
      clips: [
        {
          uri: 'file:///path/to/music.mp3',
          startTime: 0,
          duration: 9,
          volume: 0.8,
          fadeOutDuration: 1.5,
        },
      ],
    },
    {
      type: 'text',
      clips: [
        {
          text: 'Hello World',
          startTime: 0.5,
          duration: 3,
          x: 0.5,
          y: 0.15,
          textStyle: {
            fontSize: 52,
            fontWeight: 'bold',
            color: '#FFFFFF',
            shadowColor: '#000000',
            shadowRadius: 4,
          },
        },
      ],
    },
  ],
});

Real-time preview

The preview view renders the video layer natively. Text, image, and emoji overlays are returned by useCompositionOverlays so your Skia or gesture layer can make them interactive — same coordinate space, no translation required.

import { useRef, useState } from 'react';
import { View, StyleSheet } from 'react-native';
import { MediaEnginePreview, useCompositionOverlays } from '@projectyoked/expo-media-engine';

export function CompositionEditor({ config }) {
  const previewRef = useRef(null);
  const [currentTime, setCurrentTime] = useState(0);
  const [isPlaying, setIsPlaying]     = useState(false);

  // Active text/image clips at the current playback position,
  // with resolved x/y/scale/rotation/opacity — feed directly to Skia
  const overlays = useCompositionOverlays(config, currentTime);

  return (
    <View style={styles.container}>
      {/* Native video layer — filters, transitions, speed all applied here */}
      <MediaEnginePreview
        ref={previewRef}
        config={config}
        isPlaying={isPlaying}
        onTimeUpdate={e => setCurrentTime(e.nativeEvent.currentTime)}
        onLoad={e => console.log('Duration:', e.nativeEvent.duration)}
        style={StyleSheet.absoluteFill}
      />

      {/* Interactive overlay layer — render with Skia, Reanimated, or plain RN */}
      {overlays.map(overlay => (
        <InteractiveOverlay
          key={overlay.id}
          overlay={overlay}
          onMove={(id, x, y) => { /* update config */ }}
        />
      ))}
    </View>
  );
}

Other operations

import MediaEngine from '@projectyoked/expo-media-engine';

// Stitch videos end-to-end
await MediaEngine.stitchVideos(
  ['file:///clip1.mp4', 'file:///clip2.mp4'],
  'file:///output.mp4'
);

// Compress (e.g. before upload)
await MediaEngine.compressVideo({
  inputUri:  'file:///input.mp4',
  outputUri: 'file:///compressed.mp4',
  quality:   'medium',   // or set bitrate / maxWidth / maxHeight explicitly
  codec:     'h265',     // Android only; iOS uses H.264
});

// Waveform for a timeline scrubber
const amplitudes = await MediaEngine.getWaveform('file:///audio.mp3', 200);

// Extract audio from video
await MediaEngine.extractAudio('file:///video.mp4', 'file:///audio.m4a');

API Reference

MediaEngine (default export)

| Function | Description | |---|---| | composeCompositeVideo(config) | Render a multi-track composition to a video file | | stitchVideos(paths, outputUri) | Concatenate videos sequentially | | compressVideo(config) | Re-encode a video at lower bitrate or resolution | | getWaveform(uri, samples?) | Normalized amplitude array (0–1) for audio visualization | | extractAudio(videoUri, outputUri) | Extract audio track to .m4a | | exportComposition(config) | Legacy single-video + overlay export | | isAvailable() | Returns false if the native module is not linked |

iOS: Heavy work is serialized — composeCompositeVideo, exportComposition, stitchVideos, and compressVideo won’t run on top of each other. Exports with text or image overlays use two passes (encode the base video first, then apply Core Animation on a second export). That keeps Simulator stable and matches how we handle trims and transitions on device: timeline duration is honored for trimmed clips, and transition overlaps use a proper multi-track compositor path.

MediaEnginePreview (named export)

A native view component. Renders video tracks with all filters and transitions applied.

| Prop | Type | Description | |---|---|---| | config | CompositionConfig | The composition to render | | isPlaying | boolean | Play / pause | | muted | boolean | Mute audio | | currentTime | number | Seek position in seconds (controlled scrub while paused) | | onLoad | ({ duration }) | Fired once the engine is ready | | onTimeUpdate | ({ currentTime }) | Fires ~30 fps during playback | | onPlaybackEnded | () | Fired when playback reaches the end | | onError | ({ message }) | Fired on fatal errors |

Ref: previewRef.current.seekTo(seconds) for imperative seeking.

useCompositionOverlays(config, currentTime)

Returns ActiveOverlay[] — the text and image clips active at currentTime, with all transforms and keyframe animations resolved. Use this to render an interactive overlay layer in Skia, Reanimated, or plain React Native views.

Each ActiveOverlay includes: id, type, x, y, scale, rotation, opacity, text, textStyle, uri, startTime, duration, and all individual text style fields.

Clip Properties

All clip types share these base properties:

| Property | Type | Description | |---|---|---| | uri | string | File URI (file://...) | | startTime | number | When this clip appears on the timeline (seconds) | | duration | number | How long it plays (seconds) | | x / y | number | Normalized position 0–1 (center of clip) | | scale | number | Size multiplier (1.0 = original) | | rotation | number | Degrees, clockwise | | opacity | number | Transparency 0–1 | | clipStart | number | Trim start within the source file (seconds) | | clipEnd | number | Trim end within the source file (-1 = full) | | speed | number | Playback speed (0.5 = slow-mo, 2.0 = fast-forward) | | resizeMode | string | 'cover' | 'contain' | 'stretch' | | filter | FilterType | One of 9 filter types | | filterIntensity | number | Filter strength 0–1 | | transition | TransitionType | Transition applied at the end of this clip | | transitionDuration | number | Transition window in seconds | | volume | number | Audio volume 0–1 (audio/video clips) | | fadeInDuration | number | Audio fade in (seconds) | | fadeOutDuration | number | Audio fade out (seconds) | | volumeEnvelope | VolumeEnvelope | Keyframe-based volume automation | | animations | ClipAnimations | Keyframe arrays for x, y, scale, rotation, opacity |

Text clips additionally accept text, textStyle (or flat color, fontSize, etc.).

Architecture

JavaScript (src/)
    │
    ├── composeCompositeVideo()   ─────────────────────────────────┐
    ├── MediaEnginePreview             ← native view (ExpoView)    │
    └── useCompositionOverlays         ← JS hook (overlay data)   │
                                                                   │
iOS (AVFoundation)                    Android (MediaCodec + OpenGL ES 2.0)
    │                                     │
    ├── AVMutableComposition             ├── CompositeVideoComposer
    ├── MediaEngineCompositor            │       MediaExtractor → MediaCodec (decode)
    │   (CIFilter per frame)            │       → SurfaceTexture → OpenGL (render)
    ├── AVPlayer (preview)              │       → MediaCodec (encode) → MediaMuxer
    └── AVAssetExportSession            ├── RenderEngine  (preview + export render logic)
        (export)                        ├── TextureRenderer  (GLSL shaders, filters)
                                        ├── AudioMixer  (PCM mixing, volume keyframes)
                                        └── PreviewEngine  (EGL loop, 30 fps playback)

Smart passthrough — both platforms detect when re-encoding can be skipped (single clip, no transforms, no filters) and use a zero-copy path for maximum speed.

Development

npm run validate        # lint + typecheck + tests
npm run lint            # ESLint only
npm run typecheck       # TypeScript (tsc --noEmit)
npm test                # Jest
npm run test:coverage   # coverage report

There is no build step — the package is distributed as source.

Background

Developed at Project Yoked LLC for the Project Yoked fitness app. We open-sourced it under MIT so other teams can use the same native media stack without proprietary SDKs or per-minute video APIs.

Contributing

See CONTRIBUTING.md. All contributions are welcome.

Security

See SECURITY.md for reporting vulnerabilities.

Changelog

See CHANGELOG.md for a full history of changes by release.

License

MIT © Project Yoked LLC