Universal Media Engine (VME)

High-performance, framework-independent browser & native media processing platform.

Universal Media Engine (VME) provides production-grade media compression, transcoding, container muxing/demuxing, adaptive quality planning, frame-accurate timing, and validation. It is built from the ground up to operate reliably across modern browsers, dedicated Web Workers, server environments, and future mobile native runtimes.

Key Features

• Off-Main-Thread Processing: Dedicated Web Worker execution with zero main-thread UI hitching.
• Zero-Boilerplate Public Facade: Simple UniversalMediaEngine.create() API for quick integration.
• Strict Frame & Timing Guarantees:
  • Zero Frame Drops: Never drops frames to relieve queue pressure.
  • Frame Accounting: Every frame lifecycle state is tracked; zero memory leaks.
  • $\le 50\text{ ms}$ A/V Synchronization: Preserves native presentation (PTS) and decode (DTS) timestamps without artificial timeline shifting.
  • Variable Frame Rate (VFR) Support: Bit-exact preservation of variable frame timing.
• Automatic Backend Selection & Fallback: Automatically selects hardware-accelerated WebCodecs Workers, falling back seamlessly to direct main-thread processing when Workers are unavailable.
• Memory Bounded: Governed by dynamic memory budget policies and bounded frame queues.
• Independent Quality Verification: Output artifacts are certified via post-execution ISOBMFF analysis and canary re-decoding.
• Storage Decoupled: Pluggable storage adapters (custom, S3, R2, Cloudinary, etc.) with zero vendor lock-in.

Installation

npm install @universal-media-engine/media-engine
# or
pnpm add @universal-media-engine/media-engine
# or
yarn add @universal-media-engine/media-engine

3-Line Quick Start

import { UniversalMediaEngine } from '@universal-media-engine/media-engine';

const engine = UniversalMediaEngine.create();
const result = await engine.process(file);
const outputBlob = await result.toBlob();

Core Workflows

1. Video Compression with Progress Tracking

import { UniversalMediaEngine } from '@universal-media-engine/media-engine';

const engine = UniversalMediaEngine.create();

const result = await engine.process(videoFile, {
  onProgress: (progress) => {
    console.log(`Stage: ${progress.stage}, Progress: ${progress.percentage}%`);
  },
});

console.log(`Transcoded: ${result.sizeBytes} bytes, duration: ${result.durationSeconds}s`);
const blob = await result.toBlob();

2. Video Analysis & Adaptive Planning

// Analyze input media
const analysis = await engine.analyze(videoFile);
console.log(`Resolution: ${analysis.videoTracks[0]?.width}x${analysis.videoTracks[0]?.height}`);

// Formulate optimal compression plan
const plan = await engine.plan(analysis, {
  qualityPolicy: { qualityTarget: 'balanced' },
});
console.log(`Selected strategy: ${plan.strategy}`);

3. Poster Frame & Thumbnail Generation

const thumbnailPayload = await engine.createThumbnail(videoFile, {
  timeOffsetSeconds: 1.0,
  width: 640,
  height: 360,
  format: 'image/jpeg',
  quality: 0.85,
});

const thumbnailBlob = await thumbnailPayload.toBlob?.() ?? new Blob([await thumbnailPayload.readRange(0, thumbnailPayload.sizeBytes)], { type: 'image/jpeg' });

4. Cancellation with AbortSignal

const abortController = new AbortController();

setTimeout(() => {
  abortController.abort();
}, 2000);

try {
  const result = await engine.process(videoFile, {
    signal: abortController.signal,
  });
} catch (err) {
  if (err.code === 'ERR_OPERATION_CANCELLED') {
    console.log('Transcode cancelled cleanly with zero memory leaks.');
  }
}

5. Custom Storage Uploads

import type { MediaStorageAdapter } from '@universal-media-engine/media-engine';

const customStorage: MediaStorageAdapter = {
  id: 'my-s3-adapter',
  async upload(payload, metadata, signal) {
    const bytes = await payload.readRange(0, payload.sizeBytes);
    // Perform upload to your backend/S3
    return {
      status: 'success',
      bytesTransferred: payload.sizeBytes,
      totalBytes: payload.sizeBytes,
      remoteUri: `https://my-bucket.s3.amazonaws.com/${metadata.filename}`,
    };
  },
};

const result = await engine.process(videoFile);
const uploadResult = await engine.upload(result.artifact, customStorage);
console.log(`Uploaded to: ${uploadResult.remoteUri}`);

Worker Configuration & CSP

Default Bundler Usage (Vite / Webpack / Next.js)

By default, VME resolves the dedicated worker bundle automatically.

Custom Worker URL (Static CDN Hosting)

const engine = UniversalMediaEngine.create({
  workerUrl: '/static/workers/pipeline.worker.js',
});

Custom Worker Factory (Strict CSP / Blob URLs)

const engine = UniversalMediaEngine.create({
  workerFactory: () => new Worker(new URL('./custom-worker.js', import.meta.url), { type: 'module' }),
});

Forcing Direct (Main-Thread) Execution

const engine = UniversalMediaEngine.create({
  backendPreference: 'direct', // Disables Worker execution
});

Error Handling

All engine errors conform to VmeError and include structured classification:

import { VmeError } from '@universal-media-engine/media-engine';

try {
  await engine.process(corruptedFile);
} catch (err) {
  if (err instanceof VmeError) {
    console.error(`Category: ${err.category}`); // 'SOURCE' | 'CAPABILITY' | 'PROCESSING' | 'VALIDATION' | 'CANCELLATION'
    console.error(`Code: ${err.code}`);         // e.g. 'ERR_INVALID_MEDIA_SOURCE'
    console.error(`Retryable: ${err.isRetryable}`);
    console.error(`Has Fallback: ${err.hasFallback}`);
  }
}

Supported Media Matrix

| Media Kind | Format / Codec | Support Level | Execution Pipeline | |---|---|:---:|---| | Video | MP4 (H.264 / AVC + AAC) | Full Support | WebCodecs Worker / Direct + Mp4Muxer | | Video | WebM (VP8/VP9 + Opus) | Supported | WebCodecs Direct / Canvas | | Image | JPEG, PNG, WebP | Full Support | Canvas / ImageBitmap / ImageDecoder | | Audio | AAC / MP4A | Full Support | WebCodecs AudioDecoder & AudioEncoder | | GIF | Animated GIF | Full Support | Demuxing & frame trim pipeline |

Browser Requirements

• Chromium / Chrome / Edge: Version 94+ (Full WebCodecs & Worker support)
• Safari: Version 16.4+ (WebCodecs & OffscreenCanvas support)
• Firefox: Version 130+ (WebCodecs support)

License

MIT