@metagptx/video-clip

Headless video editing primitives and unstyled Vue components for browser-based clipping workflows.

@metagptx/video-clip is designed for apps that need to import local videos, split and trim segments, reorder a main sequence, place overlay clips on free tracks, preview the result, and export either project JSON or rendered video.

It is not a full nonlinear editor. The package gives you a small editing engine, Vue state helpers, optional UI building blocks, and exporter adapters that you can compose into your own product UI.

Features

• Import one or more local video files.
• Append multiple videos to the main sequence track.
• Split, trim, delete, reorder, and move segments across tracks.
• Add free overlay tracks for picture-in-picture style clips.
• Automatically create an editable linked audio track below each video track.
• Render linked audio tracks at half the video track height by default.
• Render audio waveform strips for audio timeline clips.
• Drag overlay clips to any timeline position.
• Trim audio clips independently while linked audio positions follow their video clips.
• Delete an audio clip independently while keeping the linked video clip.
• Keep linked audio moving with video when the video clip is moved, reordered, split, trimmed, or deleted.
• Snap dragged clips to the playhead and nearby clip edges.
• Preview overlay clips on top of the main video.
• Drag and resize overlay clips in the preview area while preserving aspect ratio.
• Undo and redo editing operations.
• Zoomable timeline with ruler ticks, scrollable tracks, and draggable playhead.
• Timeline video segments render sampled thumbnail strips that fill each clip block.
• Unstyled Vue UI components with styleConfig hooks for every important part.
• JSON export for persistence or server rendering.
• Browser MP4 export through ffmpeg.wasm.
• WebM fallback export through Canvas and MediaRecorder.
• Server exporter contract for production FFmpeg pipelines.

Installation

npm install @metagptx/video-clip vue sortablejs vuedraggable

If you use browser-side MP4 export, also install ffmpeg dependencies:

npm install @ffmpeg/ffmpeg @ffmpeg/util

Peer dependencies:

| Package | Required when | | --- | --- | | vue | using @metagptx/video-clip/vue or @metagptx/video-clip/ui | | sortablejs | using VideoTimeline | | vuedraggable | using VideoTimeline | | @ffmpeg/ffmpeg | using createFfmpegWasmExporter | | @ffmpeg/util | using createFfmpegWasmExporter |

Package Entrypoints

import { createVideoEditor } from '@metagptx/video-clip/core'
import { useVideoEditor } from '@metagptx/video-clip/vue'
import { VideoPreview, VideoTimeline, VideoTrimmer } from '@metagptx/video-clip/ui'
import { createFfmpegWasmExporter } from '@metagptx/video-clip/export'

| Entrypoint | What it contains | Use it for | | --- | --- | --- | | @metagptx/video-clip/core | Pure TypeScript project model, timeline operations, editor class, geometry helpers | custom state, tests, non-Vue apps | | @metagptx/video-clip/vue | Vue composables and provider | Vue apps that want editor state management | | @metagptx/video-clip/ui | Unstyled Vue components | quickly building an editor surface | | @metagptx/video-clip/export | JSON, ffmpeg.wasm, WebM, and server exporters | exporting project data or video files |

AI Integration Guide

For AI agents, code generators, and detailed API-level integration work, see docs/AI_USAGE_GUIDE.md. It documents the complete public API, parameters, component props/events, type definitions, linked audio behavior, exporter contracts, and best-practice demos.

Quick Start

<script setup lang="ts">
import { computed, ref } from 'vue'
import { useVideoEditor } from '@metagptx/video-clip/vue'
import { VideoPreview, VideoTimeline, VideoTrimmer } from '@metagptx/video-clip/ui'
import { createFfmpegWasmExporter } from '@metagptx/video-clip/export'

const editor = useVideoEditor()
const exporter = createFfmpegWasmExporter()
const exporting = ref(false)

const trimRange = computed({
  get() {
    const segment = editor.selectedSegment.value
    return {
      in: segment?.sourceIn ?? 0,
      out: segment?.sourceOut ?? 0,
    }
  },
  set(value) {
    const segment = editor.selectedSegment.value
    if (!segment) return
    editor.trim(segment.id, {
      sourceIn: value.in,
      sourceOut: value.out,
    })
  },
})

async function loadFiles(event: Event) {
  const files = (event.target as HTMLInputElement).files
  if (files) await editor.loadFiles(files)
}

async function exportMp4() {
  exporting.value = true
  try {
    const job = exporter.export(editor.project.value, {
      format: 'mp4',
      strategy: 'transcode',
    })
    const blob = await job.result
    const url = URL.createObjectURL(blob)
    window.open(url)
  } finally {
    exporting.value = false
  }
}
</script>

<template>
  <input type="file" accept="video/*" multiple @change="loadFiles" />

  <VideoPreview
    :project="editor.project.value"
    :current-time="editor.currentTime.value"
    :playing="editor.isPlaying.value"
    :selected-segment-id="editor.selectedSegmentId.value"
    @timeupdate="editor.seek"
    @ended="editor.isPlaying.value = false"
    @select-segment="editor.selectSegment"
    @update-segment-layout="editor.updateLayout"
  />

  <button type="button" @click="editor.isPlaying.value = !editor.isPlaying.value">
    {{ editor.isPlaying.value ? 'Pause' : 'Play' }}
  </button>
  <button type="button" @click="editor.split()">Split</button>
  <button type="button" :disabled="!editor.selectedSegmentId.value" @click="editor.remove()">Delete</button>
  <button type="button" :disabled="!editor.canUndo.value" @click="editor.undo()">Undo</button>
  <button type="button" :disabled="!editor.canRedo.value" @click="editor.redo()">Redo</button>
  <button type="button" :disabled="exporting" @click="exportMp4">Export MP4</button>

  <VideoTimeline
    :project="editor.project.value"
    :current-time="editor.currentTime.value"
    :selected-segment-id="editor.selectedSegmentId.value"
    @seek="editor.seek"
    @select="editor.selectSegment"
    @split="editor.split"
    @remove="editor.remove"
    @move="editor.move"
    @move-to-time="editor.moveToTime"
    @move-to-track="editor.moveToTrack"
  />

  <VideoTrimmer
    v-if="editor.selectedSegment.value"
    v-model="trimRange"
    :duration="editor.project.value.assets[0]?.duration ?? 0"
  />
</template>

Styling

The UI package is unstyled. Components expose stable structure and accept a styleConfig object. You can provide classes, inline styles, or both.

interface PartStyleConfig {
  class?: string | string[] | Record<string, boolean>
  style?: string | Record<string, string | number> | Array<Record<string, string | number>>
}

Example:

<VideoTimeline
  :project="editor.project.value"
  :current-time="editor.currentTime.value"
  :style-config="{
    root: { class: 'timeline-root' },
    rulerViewport: { class: 'timeline-ruler-viewport' },
    ruler: { class: 'timeline-ruler' },
    tick: { class: 'timeline-tick' },
    majorTick: { class: 'timeline-tick-major' },
    minorTick: { class: 'timeline-tick-minor' },
    tickLabel: { class: 'timeline-tick-label' },
    bodyViewport: { class: 'timeline-body-viewport' },
    body: { class: 'timeline-body' },
    canvas: { class: 'timeline-canvas' },
    playhead: { class: 'timeline-playhead' },
    playheadMarker: { class: 'timeline-playhead-marker' },
    track: { class: 'timeline-track' },
    trackLabel: { class: 'timeline-track-label' },
    dropTargetTrack: { class: 'timeline-track-drop-target' },
    segment: { class: 'timeline-segment' },
    audioSegment: { class: 'timeline-segment-audio' },
    audioWaveform: { class: 'timeline-audio-waveform' },
    audioWaveformBar: { class: 'timeline-audio-waveform-bar' },
    segmentThumbnails: { class: 'timeline-segment-thumbnails' },
    segmentThumbnail: { class: 'timeline-segment-thumbnail' },
    selectedSegment: { class: 'timeline-segment-selected' },
    draggingSegment: { class: 'timeline-segment-dragging' },
    ghostSegment: { class: 'timeline-segment-ghost' },
    chosenSegment: { class: 'timeline-segment-chosen' },
    dropPlaceholder: { class: 'timeline-drop-placeholder' },
    segmentTime: { class: 'timeline-segment-time' },
    segmentName: { class: 'timeline-segment-name' },
    removeButton: { class: 'timeline-remove' }
  }"
/>

For timeline alignment, avoid horizontal margin on tracks. Prefer padding on the outer timeline container, or vertical-only track spacing:

.timeline-track {
  margin: 12px 0;
}

Horizontal margins on tracks shift clips away from the ruler and playhead coordinate system.

Core Model

ClipProject is the full editor state.

interface ClipProject {
  id: string
  version: number
  assets: MediaAsset[]
  tracks: Track[]
  duration: number
  createdAt: number
  updatedAt: number
}

MediaAsset represents an imported source. In the browser, source is usually a File; for custom workflows it can also be a URL string.

interface MediaAsset {
  id: string
  type: 'video' | 'audio'
  name: string
  source: File | string
  duration: number
  width?: number
  height?: number
}

Tracks can be sequential or free-positioned.

interface Track {
  id: string
  linkedTrackId?: string
  type: 'video' | 'audio'
  kind: 'sequence' | 'free'
  name: string
  segments: Segment[]
}

• kind: 'sequence': clips are normalized end to end. This is the main video track.
• kind: 'free': clips keep their own timelineIn. This is useful for overlays.
• Video tracks can have a linked audio track through linkedTrackId. Imported videos create a video segment and a linked audio segment.
• Audio tracks use type: 'audio' and are free-positioned by default, so audio can be trimmed and moved independently.

Segment references a source asset and defines source time, timeline time, and optional preview layout.

interface Segment {
  id: string
  assetId: string
  linkedSegmentId?: string
  sourceIn: number
  sourceOut: number
  timelineIn: number
  timelineOut: number
  layout?: SegmentLayout
}

interface SegmentLayout {
  x: number
  y: number
  width: number
  height: number
}

layout is normalized from 0 to 1 relative to the preview canvas.

Linked video/audio behavior follows the usual editor convention:

• Moving a video clip moves its linked audio by the same timeline delta.
• Reordering a video sequence keeps the linked audio aligned under the new video position.
• Splitting, trimming, or deleting a video clip applies the same operation to its linked audio.
• Trimming an audio clip directly only changes that audio clip.
• Linked audio clips cannot be moved directly; move the video clip to change their timeline position.
• Deleting an audio clip removes only that audio clip and clears the video link. The linked video clip stays in place.
• Cross-track moves are type-safe: video clips can move only between video tracks, and audio clips can move only between audio tracks.

Vue Editor API

useVideoEditor() is the recommended Vue entrypoint.

const editor = useVideoEditor()

State:

| Field | Type | Description | | --- | --- | --- | | project | Ref<ClipProject> | current project | | currentTime | Ref<number> | current timeline time | | selectedSegmentId | Ref<string \| null> | selected segment id | | selectedSegment | ComputedRef<Segment \| null> | selected segment | | isPlaying | Ref<boolean> | preview playback state | | canUndo | Ref<boolean> | whether undo is available | | canRedo | Ref<boolean> | whether redo is available | | playbackResolution | ComputedRef<PlaybackResolution \| null> | active main-track source resolution |

Actions:

| Method | Description | | --- | --- | | loadFile(file) | import one video into the default track | | loadFiles(files, trackId?) | import multiple videos into a target track | | loadFileToTrack(file, trackId?) | import one video into a target track | | addFreeTrack() | add a free overlay track | | split(time?) | split the active segment at the current or provided time | | remove(segmentId?) | remove a segment; defaults to the selected segment | | trim(segmentId, range) | update sourceIn and sourceOut | | move(segmentId, targetIndex) | reorder a segment inside a sequence track | | moveToTime(segmentId, timelineIn) | move a free-track segment to a timeline time | | moveToTrack(segmentId, targetTrackId, options?) | move a segment between main and overlay tracks | | updateLayout(segmentId, layout) | update overlay preview position and size | | undo() / redo() | undo or redo | | seek(time) | set current timeline time | | selectSegment(segmentId) | select a segment |

UI Components

VideoPreview

VideoPreview renders the active main video and visible overlay clips. Overlay clips can be selected, dragged, and resized.

Props:

| Prop | Type | Description | | --- | --- | --- | | project | ClipProject | current project | | currentTime | number | current timeline time | | playing | boolean | playback state | | selectedSegmentId | string \| null | selected segment id | | styleConfig | VideoPreviewStyleConfig | style hooks |

Events:

| Event | Payload | Description | | --- | --- | --- | | timeupdate | time: number | emitted as playback advances | | ended | none | emitted when playback reaches project end | | selectSegment | segmentId: string | overlay selection | | updateSegmentLayout | segmentId, layout | committed overlay drag or resize |

Style keys:

interface VideoPreviewStyleConfig {
  root?: PartStyleConfig
  video?: PartStyleConfig
  empty?: PartStyleConfig
  overlay?: PartStyleConfig
  overlayHover?: PartStyleConfig
  overlaySelected?: PartStyleConfig
  overlayVideo?: PartStyleConfig
  resizeHandle?: PartStyleConfig
  resizeHandleHover?: PartStyleConfig
  resizeHandleNorthWest?: PartStyleConfig
  resizeHandleNorthEast?: PartStyleConfig
  resizeHandleSouthEast?: PartStyleConfig
  resizeHandleSouthWest?: PartStyleConfig
}

VideoTimeline

VideoTimeline renders the ruler, playhead, sequence tracks, free tracks, segment blocks, and drag placeholders.

Props:

| Prop | Type | Description | | --- | --- | --- | | project | ClipProject | current project | | currentTime | number | current timeline time | | selectedSegmentId | string \| null | selected segment id | | segmentDisplayMode | 'details' \| 'slider' | details shows time and name; slider shows a plain block | | pixelsPerSecond | number | zoom level, default 16 | | secondsPerTick | number | fixed minor tick interval override | | pixelsPerTick | number | compatibility zoom input used when pixelsPerSecond is not set | | minTickSpacing | number | minimum pixel spacing between minor ticks, default 10 | | minLabelSpacing | number | minimum pixel spacing between labels, default 96 | | minTrackWidth | number | minimum content width, default 720 | | minBodyHeight | number | minimum scroll body height, default 180 | | trackHeight | number | track height, default 72 | | styleConfig | VideoTimelineStyleConfig | style hooks |

Events:

| Event | Payload | Description | | --- | --- | --- | | seek | time: number | click or drag playhead | | select | segmentId: string | select a segment | | split | time: number | double-click split request | | remove | segmentId: string | remove request | | move | segmentId, targetIndex | sequence reorder | | moveToTime | segmentId, timelineIn | free-track time move | | moveToTrack | segmentId, targetTrackId, options? | cross-track move |

Style keys:

interface VideoTimelineStyleConfig {
  root?: PartStyleConfig
  rulerViewport?: PartStyleConfig
  ruler?: PartStyleConfig
  tick?: PartStyleConfig
  minorTick?: PartStyleConfig
  majorTick?: PartStyleConfig
  tickLabel?: PartStyleConfig
  bodyViewport?: PartStyleConfig
  body?: PartStyleConfig
  canvas?: PartStyleConfig
  playhead?: PartStyleConfig
  playheadMarker?: PartStyleConfig
  track?: PartStyleConfig
  trackLabel?: PartStyleConfig
  dropTargetTrack?: PartStyleConfig
  segment?: PartStyleConfig
  audioSegment?: PartStyleConfig
  audioWaveform?: PartStyleConfig
  audioWaveformBar?: PartStyleConfig
  segmentThumbnails?: PartStyleConfig
  segmentThumbnail?: PartStyleConfig
  draggingSegment?: PartStyleConfig
  ghostSegment?: PartStyleConfig
  chosenSegment?: PartStyleConfig
  selectedSegment?: PartStyleConfig
  dropPlaceholder?: PartStyleConfig
  segmentTime?: PartStyleConfig
  segmentName?: PartStyleConfig
  segmentStart?: PartStyleConfig
  segmentDuration?: PartStyleConfig
  removeButton?: PartStyleConfig
}

Timeline ruler behavior:

• The default zoom is pixelsPerSecond = 16.
• The ruler chooses readable tick steps such as 0.1s, 0.2s, 0.5s, 1s, 2s, 5s, 10s, 30s, and 1min.
• Every tenth minor tick can become a major tick when spacing allows.
• Labels respect minLabelSpacing to avoid overlap.
• Content width is max(minTrackWidth, duration * pixelsPerSecond).
• Horizontal overflow scrolls. Vertical overflow scrolls when there are many tracks.
• Video segment thumbnails are sampled sparsely and repeated as a strip, enough to fill the clip block without extracting every frame.
• Remote video URLs used for timeline thumbnails are fetched once into an in-memory Blob URL cache, so zooming the timeline can reuse the local cached source and only generate missing frame thumbnails.
• Audio tracks render at half of trackHeight by default and show waveform bars decoded from the clip audio when the browser can read the source.
• Free-position dragging snaps segment starts and ends to nearby segment edges, timeline zero/end, and the current playhead.
• Audio clips are not position-draggable in the built-in timeline; moving a video clip keeps linked audio aligned. Invalid targets do not show a drop placeholder and do not emit moveToTrack.

VideoTrimmer

VideoTrimmer edits the selected segment source range.

Props:

| Prop | Type | Description | | --- | --- | --- | | duration | number | source asset duration | | modelValue | { in: number; out: number } | current trim range | | styleConfig | VideoTrimmerStyleConfig | style hooks |

Events:

| Event | Payload | Description | | --- | --- | --- | | update:modelValue | { in, out } | trim range update |

Style keys:

interface VideoTrimmerStyleConfig {
  root?: PartStyleConfig
  label?: PartStyleConfig
  labelText?: PartStyleConfig
  input?: PartStyleConfig
  value?: PartStyleConfig
}

Headless Usage

You can use the core package without Vue.

import {
  appendAsset,
  createEmptyProject,
  createVideoEditor,
  resolvePlaybackTime,
  splitSegment,
} from '@metagptx/video-clip/core'

let project = createEmptyProject()

project = appendAsset(project, {
  id: 'asset-1',
  type: 'video',
  name: 'sample.mp4',
  source: 'sample.mp4',
  duration: 12,
})

const segment = project.tracks[0].segments[0]
project = splitSegment(project, segment.id, 5)

const editor = createVideoEditor(project)
editor.undo()

const playback = resolvePlaybackTime(editor.project, 3)

Core functions are immutable: they receive a project and return a new project. VideoClipEditor wraps the same operations with undo and redo.

Timeline Geometry Helpers

Use these helpers when building a custom timeline so your UI matches the built-in timeline math.

import {
  createTimelineTicks,
  resolveTimelineScale,
  segmentLeftPixels,
  segmentWidthPixels,
} from '@metagptx/video-clip/core'

const scale = resolveTimelineScale({
  duration: project.duration,
  pixelsPerSecond: 16,
  minTrackWidth: 960,
})

const ticks = createTimelineTicks(scale)
const left = segmentLeftPixels(segment, scale)
const width = segmentWidthPixels(segment, scale)

Custom UI Cost

You can ignore all built-in UI components and keep only the engine. The integration work is mostly three mappings:

• Preview: map currentTime to the active asset and set <video>.currentTime.
• Timeline: map seconds to pixels, then call editor actions after drag/drop.
• Trimmer: map input controls to sourceIn and sourceOut.

For preview playback:

import { resolvePlaybackTime } from '@metagptx/video-clip/core'

const resolution = resolvePlaybackTime(project, currentTime)

// resolution = {
//   assetId: string,
//   segmentId: string,
//   sourceTime: number
// }

For timeline actions:

editor.move(segmentId, targetIndex)

editor.moveToTime(segmentId, nextTimelineIn)

editor.moveToTrack(segmentId, targetTrackId, {
  targetIndex, // for sequence tracks
  timelineIn, // for free tracks
})

For trim controls:

editor.trim(segment.id, {
  sourceIn: nextIn,
  sourceOut: nextOut,
})

Export

JSON

Use JSON export for persistence, debugging, or sending work to a backend.

import { createJsonExporter } from '@metagptx/video-clip/export'

const exporter = createJsonExporter()
const job = exporter.export(editor.project.value, { format: 'json' })
const blob = await job.result
const json = await blob.text()

Browser MP4 Through ffmpeg.wasm

import { createFfmpegWasmExporter } from '@metagptx/video-clip/export'

const exporter = createFfmpegWasmExporter()
const job = exporter.export(editor.project.value, {
  format: 'mp4',
  strategy: 'transcode',
})

job.onProgress((progress) => {
  console.log(progress.percent, progress.stage)
})

const blob = await job.result

You can provide your own ffmpeg core files:

const exporter = createFfmpegWasmExporter({
  coreURL: '/ffmpeg/ffmpeg-core.js',
  wasmURL: '/ffmpeg/ffmpeg-core.wasm',
  workerURL: '/ffmpeg/ffmpeg-core.worker.js',
})

Or use a CDN base URL:

const exporter = createFfmpegWasmExporter({
  baseURL: 'https://unpkg.com/@ffmpeg/[email protected]/dist/esm',
})

Export strategies:

• strategy: 'copy': faster, but source codec/container compatibility matters.
• strategy: 'transcode': slower, but more reliable for mixed sources.

Browser MP4 export composes free video tracks over the primary sequence. The server exporter receives the full audio-track timeline in JSON. Browser MP4 audio rendering still uses the current ffmpeg.wasm pipeline limitations described below.

WebM Fallback

The WebM exporter uses Canvas and MediaRecorder. It is useful for preview, demos, and browser-only fallback flows.

import { createMediaRecorderExporter } from '@metagptx/video-clip/export'

const exporter = createMediaRecorderExporter()
const job = exporter.export(editor.project.value, {
  format: 'webm',
  width: 1280,
  height: 720,
  fps: 30,
})

const blob = await job.result

Server Exporter

Use the server exporter when production rendering should happen in your backend FFmpeg pipeline.

import { createServerExporter } from '@metagptx/video-clip/export'

const exporter = createServerExporter({
  endpoint: '/api/video-export',
})

const job = exporter.export(editor.project.value, {
  format: 'mp4',
})

const result = await job.result
console.log(result.url ?? result.jobId)

The backend receives the project JSON and export options. For production, server FFmpeg is usually more predictable than browser ffmpeg.wasm.

Browser Notes

• Local file import relies on browser File objects and generated object URLs.
• Browser MP4 export depends on ffmpeg.wasm and can take time to load on first use.
• Some deployments require cross-origin isolation headers for threaded ffmpeg.wasm builds.
• strategy: 'copy' is sensitive to codec and container compatibility.
• Use strategy: 'transcode' for mixed user uploads.
• WebM fallback depends on MediaRecorder support in the browser.

Playground

This repository includes a Vite playground that covers:

• multi-video import;
• importing into a selected track;
• adding overlay tracks;
• main track split, delete, and reorder;
• linked audio tracks below video tracks;
• independent audio trim;
• linked audio waveform display;
• free-track timeline dragging;
• preview overlay dragging and aspect-ratio resize;
• trimmer updates for sourceIn and sourceOut;
• undo and redo;
• JSON export;
• MP4 ffmpeg.wasm export;
• WebM fallback export.

Run it locally:

npm install
npm run dev

Open:

http://localhost:5173/

Development

npm install
npm run check

Useful commands:

npm run test
npm run typecheck
npm run build:playground
npm run build:lib
npm run test:e2e

Before publishing:

npm run check
npm pack --dry-run

Current Limitations

• The package is focused on video-first editing. Linked audio tracks are editable and show waveform previews when decoding is supported, but advanced audio features such as fades, keyframes, and full browser-side mixing are not complete.
• Browser MP4 export is limited by ffmpeg.wasm startup cost and browser memory.
• Preview supports timeline audio tracks, and server export receives the audio edit data. Browser MP4 export does not yet fully mix arbitrary edited audio tracks.
• Overlay video is composed into exports, but overlay audio is not mixed in the browser MP4 exporter.
• The built-in UI components are editing primitives, not a complete NLE application.