Universal Media Engine (@universal-media-engine/media-engine)

The Universal Media Engine (VME) is a standalone, reusable, high-performance client-side media processing and optimization engine built for modern web applications.

VME provides hardware-accelerated browser-native video transcoding, frame processing, audio multiplexing, A/V synchronization, and output validation directly inside client runtimes using W3C WebCodecs and ISOBMFF box packaging.

Note: VME is a pure, framework-independent media processing engine. It does NOT handle database storage, user authentication, cloud persistence (AWS S3, Cloudflare R2, Supabase, Cloudinary, Firebase), or CDN distribution. Host applications consume VME to process media locally and manage output persistence independently.

Key Features

• Framework-Independent: Pure TypeScript/ESM package with zero React, Vue, Next.js, or framework dependencies.
• Browser-Native WebCodecs: Leverages native hardware-accelerated VideoDecoder, VideoEncoder, AudioDecoder, and AudioEncoder APIs.
• H.264 / AAC MP4 Pipeline: Demuxes, decodes, scales, encodes, and muxes compliant MP4 containers with ftyp, moov, mdat, avcC, esds, stss sync sample boxes, and 64-bit co64 large-file offset support.
• Adaptive Small & Large File Semantics:
  • Small Files (< 10 MB): Default to direct_upload strategy to conserve client CPU and battery.
  • Medium Files (10–30 MB): Apply adaptive light_compression.
  • Large Files (500 MB – 5 GB+): Processed via bounded stream readers with low memory overhead and 64-bit offsets.
• Declarative Quality Presets: Select fast, balanced, maximum, or custom quality target specs.
• Declarative Validation: Evaluates physical produced output Blobs against target resolutions, frame pacing, audio track presence, A/V sync drift, and container box structures.
• Cancellation & Safety: Supports standard W3C AbortSignal with guaranteed resource cleanup (videoFrame.close(), audioData.close()).

Installation

npm install @universal-media-engine/media-engine
# or
pnpm add @universal-media-engine/media-engine

Browser Requirements

VME requires modern browsers with native WebCodecs and Media Source extensions:

• Chromium / Chrome: 94+
• Edge: 94+
• Safari: 16.4+
• Firefox: 130+ (WebCodecs support enabled)

Basic Usage Example

import {
  MediaEngine,
  BrowserSourceAdapter,
  VmeError,
  type OperationOrchestrationResult,
} from '@universal-media-engine/media-engine';

// 1. Instantiate default browser engine
const engine = MediaEngine.createDefaultBrowserEngine();

// 2. Wrap browser File or Blob input
const fileInput = document.querySelector<HTMLInputElement>('#video-file')!.files![0];
const source = BrowserSourceAdapter.fromFile(fileInput);

// 3. Process video
try {
  const result: OperationOrchestrationResult = await engine.process(source, {
    qualityPolicy: { qualityTarget: 'balanced' },
    onProgress: (progress) => {
      console.log(`Current Stage: ${progress.stage}`);
    },
  });

  const outputArtifact = result.processingResult.producedArtifacts[0];
  const outputBlob = outputArtifact.metadata?.blob as Blob;

  console.log(`Successfully processed video: ${outputBlob.size} bytes`);
  console.log(`Validation Status: ${result.validationResult.status}`);

  // Host application handles cloud upload / storage:
  // await myHostStorageService.upload(outputBlob);
} catch (err) {
  if (err instanceof VmeError) {
    console.error(`VME Error [${err.code}]: ${err.message}`);
  }
}

Advanced Usage Examples

1. Selected Segment Trimming

const result = await engine.process(source, {
  metadata: {
    selectedSegment: {
      startTimeSeconds: 5,
      endTimeSeconds: 20, // Process only 5s to 20s window
    },
  },
});

2. Explicit Compression Request (< 10 MB Override)

const smallSource = BrowserSourceAdapter.fromFile(smallFile);

const result = await engine.process(smallSource, {
  qualityPolicy: {
    qualityTarget: 'maximum',
    forceCompression: true, // Overrides default direct_upload for small files
  },
});

3. Cancellation Handling

const controller = new AbortController();

// Cancel operation after 3 seconds
setTimeout(() => controller.abort(), 3000);

try {
  await engine.process(source, { signal: controller.signal });
} catch (err) {
  if (err instanceof VmeError && err.category === 'CANCELLATION') {
    console.log('Media processing operation was safely cancelled.');
  }
}

Host Application Integration Model

VME separates client-side media transformation from host application business logic and cloud persistence:

+-------------------------------------------------------------+
|                     Host Application                        |
|                                                             |
|   1. User selects File/Blob input                           |
|   2. Passes input handle to @universal-media-engine/media-engine|
+------------------------------+------------------------------+
                               |
                               v
+-------------------------------------------------------------+
|           @universal-media-engine/media-engine              |
|                                                             |
|   - Analyze -> Plan -> Process -> Validate                  |
|   - Native WebCodecs H.264/AAC Transcoding                  |
|   - Returns verified MediaArtifact (output Blob)            |
+------------------------------+------------------------------+
                               |
                               v
+-------------------------------------------------------------+
|                     Host Application                        |
|                                                             |
|   3. Receives verified MediaArtifact output Blob            |
|   4. Uploads Blob to Cloud Storage (S3, R2, Supabase, etc.) |
+-------------------------------------------------------------+

License

MIT