Segment Remixer

A TypeScript CLI tool and library for segment-based media remixing using FFmpeg. Extract, rearrange, and combine video/audio segments with precision and performance.

Features

• Segment-based processing: Extract segments from video URLs using timestamps
• Mixed content support: Process both video clips and static image gaps in a single remix
• Advanced effects: Apply fade-in/out transitions and overlay graphics with timing control
• HLS stream support: Optimized processing of M3U8 streams with decode-based seeking
• Parallel processing: Configurable concurrent segment processing (1-8 parallel jobs)
• Custom video specs: Override output resolution, frame rate, and pixel format
• Custom directory naming: Use custom IDs instead of timestamps for output folders
• Dual logging: Automatic logging to both console and file in output directory
• Input preservation: Original segment JSON saved alongside processing results
• Asset management: Automatic saving of all images (gap images and overlays) to destination folder for debugging and QA
• FFmpeg integration: Safe, performant media processing with advanced FFmpeg techniques
• Standardized output: Force all outputs to 1080p 16:9 30fps format with proper keyframe alignment
• Perfect concatenation: Clean video joins with keyframe-aware stitching (no stuck frames)
• TypeScript-first: Full type safety with strict TypeScript configuration
• CLI interface: Command-line tool with dry-run capabilities and validation
• Structured logging: Observable processing with Pino logger and progress tracking
• Configurable: Environment-based configuration with Zod validation
• Domain-driven architecture: Clean separation of concerns following engineering best practices

Installation

# Clone the repository
git clone <repository-url>
cd segment-remixer

# Install dependencies
npm install

# Build the project
npm run build

Requirements

• Node.js: >= 20.0.0 (LTS)
• FFmpeg: >= 6.0.0 (must be available in PATH)
• FFprobe: >= 6.0.0 (comes with FFmpeg)

Quick Start

# Install globally for CLI usage
npm link

# Show help
segment-remixer --help

# Dry-run planning (shows what would be done)
segment-remixer plan input.mp4 --dry-run

Usage

CLI Commands

# Process segments from a JSON file
npx tsx src/cli/index.ts process example-segments.json

# Or use the built version
node dist/cli/index.js process example-segments.json

# Process with custom output directory
npx tsx src/cli/index.ts process segments.json -o /path/to/output

# Use custom directory name instead of timestamp
npx tsx src/cli/index.ts process segments.json --id "my-remix"

# Enable parallel processing (2-8 concurrent jobs)
npx tsx src/cli/index.ts process segments.json --parallel 4

# Custom video specifications
npx tsx src/cli/index.ts process segments.json --width 1280 --height 720 --fps 24

# Dry run to see what would be processed
npx tsx src/cli/index.ts process segments.json --dry-run

# Validate a segments JSON file
npx tsx src/cli/index.ts validate segments.json

# Increase verbosity for detailed progress
npx tsx src/cli/index.ts process segments.json --verbose

# Suppress output except errors
npx tsx src/cli/index.ts process segments.json --quiet

Segment JSON Format

Create a JSON file with an array of segments. Supports two segment types:

Video Clip Segments

{
  "type": "clip",
  "src": "https://stream.mux.com/example.m3u8",
  "start": 37.565,
  "end": 80.535,
  "fadeIn": 2.0,
  "fadeOut": 3.0,
  "overlay": {
    "src": "https://placehold.co/1920x300/FFFFFF/000000/png?text=Title",
    "start": 0,
    "end": 5,
    "position": "bottom-left"
  }
}

Required fields:

• type: Must be "clip"
• src: Video URL (supports HLS .m3u8 streams and regular video files)
• start: Start time in seconds (must be ≥ 0)
• end: End time in seconds (must be > start)

Optional effects:

• fadeIn: Fade-in duration in seconds
• fadeOut: Fade-out duration in seconds
• overlay: Overlay image/graphic with timing and positioning
  • src: Image URL, path, or data URL (automatically saved to output directory)
    • HTTP(S) URLs: "https://example.com/logo.png" (downloaded and cached)
    • Data URLs: "data:image/png;base64,iVBORw0KGgo..." (decoded and saved)
    • Local paths: "./assets/logo.jpg" (used as-is)
  • start: Overlay start time relative to segment start
  • end: Overlay end time relative to segment start
  • position: (optional) Anchor point for overlay positioning - one of:
    • "top-left", "top-center", "top-right"
    • "center-left", "center", "center-right"
    • "bottom-left", "bottom-center", "bottom-right"
    • Defaults to "top-left" if not specified

Overlay Image Processing: All overlay images are automatically saved to the output directory:

• Saved as segment_XXX_overlay.{ext} for easy identification and debugging
• Used from saved files in FFmpeg operations for improved reliability

Gap Segments (Static Images)

{
  "type": "gap",
  "src": "https://placehold.co/1920x1080/000000/FFFFFF/png?text=Intermission",
  "duration": 5.0,
  "fadeIn": 1.0,
  "fadeOut": 1.0
}

Required fields:

• type: Must be "gap"
• src: Static image URL, path, or data URL
  • HTTP(S) URLs: "https://example.com/image.png" (downloaded and cached)
  • Data URLs: "data:image/png;base64,iVBORw0KGgo..." (decoded and saved)
  • Local paths: "./assets/image.jpg" (used as-is)
• duration: Duration to display image in seconds

Optional effects:

• fadeIn: Fade-in duration in seconds
• fadeOut: Fade-out duration in seconds

Image Processing: All images are automatically saved to the output directory for debugging and QA:

• Data URLs are decoded and saved as segment_XXX_image.png
• HTTP(S) URLs are downloaded and saved as segment_XXX_image.{ext}
• File extensions are intelligently determined from MIME types or URL extensions

Complete Example

[
  {
    "type": "clip",
    "src": "https://stream.mux.com/video1.m3u8",
    "start": 10.5,
    "end": 45.2,
    "fadeOut": 2,
    "overlay": {
      "src": "https://example.com/logo.png",
      "start": 0,
      "end": 5,
      "position": "top-right"
    }
  },
  {
    "type": "gap", 
    "src": "https://placehold.co/1920x1080/000000/FFFFFF/png?text=Coming+Up",
    "duration": 3.0
  },
  {
    "type": "clip",
    "src": "https://stream.mux.com/video2.m3u8",
    "start": 120,
    "end": 180,
    "fadeIn": 1.5
  }
]

CLI Options

Process Command

npx tsx src/cli/index.ts process <segments-file> [options]

Options:

• -o, --output <dir>: Output directory (default: ./output)
• --id <string>: Custom output directory name (instead of timestamp)
• --parallel <number>: Number of parallel segment jobs (1-8, default: 2)
• --width <number>: Video width in pixels (overrides default 1920)
• --height <number>: Video height in pixels (overrides default 1080)
• --fps <number>: Video framerate (overrides default 30)
• --dry-run: Show what would be done without executing
• -q, --quiet: Suppress non-error output
• -v, --verbose: Increase verbosity (can be used multiple times)

Validate Command

npx tsx src/cli/index.ts validate <segments-file>

Validates the JSON structure and reports any errors without processing.

Programmatic API

This package exports a programmatic API for use in other Node.js applications:

npm install @theirstoryinc/segment-remixer

Basic Usage

import { processSegments, validateSegments, dryRunSegments } from '@theirstoryinc/segment-remixer';

// Process segments from file
const result = await processSegments('./segments.json', {
  outputDir: './my-output',
  parallel: 4,
  videoSpecs: { width: 1280, height: 720 }
});

if (result.success) {
  console.log('Final video:', result.finalOutput);
  console.log('Processing time:', result.processingTime / 1000, 'seconds');
} else {
  console.error('Processing failed:', result.error);
}

Process from Segments Array

const segments = [
  {
    type: 'clip',
    src: 'https://example.com/video.mp4',
    start: 0,
    end: 10,
    fadeIn: 1.0
  },
  {
    type: 'gap',
    src: 'https://example.com/image.jpg',
    duration: 2.0
  }
];

const result = await processSegments(segments, {
  outputDir: './output',
  customId: 'my-remix'
});

Validation

// Validate segments before processing
const validation = await validateSegments('./segments.json');
if (!validation.valid) {
  console.error('Validation errors:', validation.errors);
  return;
}

console.log('Valid segments:', validation.segments?.length);

Dry Run

// See what would be processed without executing
await dryRunSegments(segments, {
  outputDir: './test-output',
  videoSpecs: { width: 1920, height: 1080 }
});

API Options

processSegments(input, options)

• input: Array of segments or path to JSON file
• options:
  • outputDir?: string - Output directory (default: './output')
  • customId?: string - Custom directory name instead of timestamp
  • parallel?: number - Parallel jobs 1-8 (default: 2)
  • videoSpecs?: Partial<VideoSpecs> - Custom video specifications
  • logger?: Logger - Custom Pino logger instance
  • verbosity?: number - Log verbosity 0=quiet, 1=normal, 2+=verbose
  • ffmpegPath?: string - Path to ffmpeg binary (default: 'ffmpeg' from PATH)
  • ffprobePath?: string - Path to ffprobe binary (default: 'ffprobe' from PATH)

validateSegments(input, options)

• input: Array of segments or path to JSON file
• options:
  • logger?: Logger - Custom Pino logger instance
  • verbosity?: number - Log verbosity level

dryRunSegments(input, options)

• Same parameters as processSegments but only shows what would be processed

TypeScript Types

import type { 
  Segment, 
  VideoSpecs, 
  ProcessingResult,
  ValidationResult,
  ProcessSegmentsOptions 
} from '@theirstoryinc/segment-remixer';

const videoSpecs: VideoSpecs = {
  width: 1920,
  height: 1080,
  framerate: 30,
  pixelFormat: 'yuv420p'
};

Configuration

Configure via environment variables:

# Set log level (trace, debug, info, warn, error, fatal)
export LOG_LEVEL=debug

# Custom FFmpeg paths (if not in PATH)
export FFMPEG_PATH=/usr/local/bin/ffmpeg
export FFPROBE_PATH=/usr/local/bin/ffprobe

Development

Available Scripts

# Development
npm run dev              # Run CLI with tsx (development mode)
npm run build            # Build TypeScript to JavaScript
npm run typecheck        # Type checking only

# Quality
npm run lint             # Lint with ESLint
npm run lint:fix         # Auto-fix linting issues
npm run format           # Check code formatting
npm run format:write     # Format code with Prettier

# Testing
npm run test             # Run unit tests
npm run test:watch       # Watch mode for tests
npm run test:coverage    # Generate coverage reports

Project Structure

src/
├── cli/                 # Command-line interface
├── config.ts           # Configuration management (Zod validation)
├── domain/             # Pure business logic (no I/O)
├── infra/              # Infrastructure integrations (FFmpeg, filesystem)
├── logger.ts           # Structured logging setup
├── services/           # Orchestration/use-cases
└── test-utils/         # Testing helpers and mocks

Architecture Principles

• Domain-driven: Business logic separated from infrastructure
• Dependency injection: Explicit dependencies passed through layers
• Error boundaries: Proper error handling with custom error types
• Resource safety: Automatic cleanup of temporary files and processes
• Security-first: Input validation and safe FFmpeg usage

Contributing

1. Setup: Follow the installation steps above
2. Development: Use npm run dev for development with hot-reload
3. Testing: Write tests for new features, aim for 90%+ coverage
4. Linting: Run npm run lint:fix before committing
5. Commits: Use conventional commits (feat:, fix:, chore:)

Guidelines

• TypeScript: Strict mode enabled, no any types
• Imports: Use explicit imports, prefer type-only imports for types
• Error handling: Use Result<T, E> pattern for recoverable operations
• FFmpeg safety: Never build shell strings, always use spawn with args array
• Performance: Bound concurrency, stream processing, memory monitoring

Testing

# Unit tests (fast, no I/O)
npm test

# Integration tests (requires FFmpeg)
FFMPEG_RUN_TESTS=1 npm test

# Coverage report
npm run test:coverage

Test structure:

• Unit tests: Pure functions, mocks for I/O
• Integration tests: Real FFmpeg pipelines with small fixtures
• E2E tests: CLI command testing

License

MIT License - see LICENSE file for details.

Output Structure

When processing segments, the tool creates a directory structure with all processing artifacts:

output/
└── 2025-09-12T00-18-53-078Z/  # Or custom --id name
    ├── input_segments.json        # Original input JSON (preserved)
    ├── processing.log             # Detailed processing logs
    ├── segment_000.mp4            # First segment (raw extraction)
    ├── segment_000_overlay.png    # Downloaded/saved overlay image (if overlay used)
    ├── segment_000.v2.mp4         # First segment with effects applied
    ├── segment_001_image.png      # Downloaded/saved gap image (if gap segment)
    ├── segment_001.mp4            # Gap segment (static image + audio)
    ├── segment_002.mp4            # Third segment
    ├── ...
    └── final_output.mp4           # Final concatenated video

File Types:

• input_segments.json: Original input preserved for reproducibility
• processing.log: Structured logs with timestamps and progress data
• segment_XXX.mp4: Individual segments (raw extraction from sources)
• segment_XXX.v2.mp4: Segments with effects (fades, overlays) applied
• segment_XXX_image.png/jpg: Gap segment images (data URLs converted or HTTP(S) URLs downloaded)
• segment_XXX_overlay.png/jpg: Overlay images (data URLs converted or HTTP(S) URLs downloaded)
• final_output.mp4: Final concatenated video with all segments joined

Asset Management: All images (gap backgrounds and overlay graphics) are automatically saved to the output directory:

• Data URLs: Base64-encoded images are decoded and saved as PNG/JPG files
• HTTP(S) URLs: Remote images are downloaded and cached locally
• Local paths: File paths are used as-is (no copying)
• Smart extensions: File extensions determined from MIME types or URL extensions
• QA-friendly: All source assets preserved for debugging and verification

All outputs are standardized to:

• Resolution: 1920x1080 (1080p)
• Aspect Ratio: 16:9 (letterboxed if needed)
• Frame Rate: 30 fps
• Video Codec: H.264 (libx264) with optimized encoding
• Audio Codec: AAC @ 192k with sync correction
• Container: MP4 with fast-start
• Keyframes: Forced every 0.5 seconds for perfect concatenation
• Seeking: Decode-based seeking for HLS streams (faster and more accurate)
• Concatenation: Filter-based concatenation with integrated scaling for seamless joins

Roadmap

✅ Completed Features

• [x] Basic segment extraction and concatenation
• [x] Mixed content support (video clips + static image gaps)
• [x] Advanced effects (fade-in/out, overlay graphics)
• [x] JSON-based segment definition with validation
• [x] HLS stream support with decode-based seeking
• [x] Parallel processing with configurable concurrency (1-8 jobs)
• [x] Custom video specifications (resolution, fps, format)
• [x] Custom directory naming (--id option)
• [x] Dual logging (console + file output)
• [x] Input preservation (original JSON saved with results)
• [x] Comprehensive asset management (automatic image saving for gaps and overlays)
• [x] Data URL support (base64-encoded images automatically decoded and saved)
• [x] HTTP(S) image downloading (remote images downloaded and cached locally)
• [x] Standardized video output format (1080p 16:9 30fps)
• [x] CLI interface with validation and dry-run
• [x] Perfect concatenation with keyframe alignment
• [x] Progress tracking and structured logging
• [x] Error handling and timeout management

🚀 Future Enhancements

• [ ] Transition effects between segments (crossfade, wipe, etc.)
• [ ] Audio processing (volume normalization, crossfading)
• [ ] Batch processing multiple segment files
• [ ] Web UI for visual segment editing
• [ ] Real-time preview capabilities
• [x] Advanced overlay positioning (9 anchor points: corners, edges, center)
• [ ] Overlay animations
• [ ] Template system for common remix patterns

Technical Optimizations

HLS Stream Processing

• Decode-based seeking: Processes streams more efficiently by seeking after decoding rather than before input
• Network optimization: Minimizes chunk downloads and handles network timeouts intelligently
• Accurate seeking: Uses -accurate_seek for frame-perfect positioning in streams

Video Concatenation

• Filter-complex approach: Uses FFmpeg's filter graph for seamless joins instead of file-based concatenation
• Keyframe management: Forces keyframes every 0.5 seconds to prevent "stuck frame" issues during joins
• Integrated scaling: Applies scaling during concatenation for consistent output quality
• Audio sync correction: Includes audio resampling (aresample=async=1000) to prevent sync drift

Performance Features

• Parallel processing: Configurable concurrent segment processing with semaphore-based control
• Progress monitoring: Real-time progress tracking with structured logging and event emission
• Resource management: Bounded concurrency, timeout management, and memory monitoring
• Automatic cleanup: Temporary file and process cleanup with graceful error handling
• Intelligent timeouts: Different timeout strategies for network vs processing operations
• Order preservation: Maintains segment order despite parallel processing using Promise.all
• Asset caching: Downloaded images cached locally to avoid redundant network requests
• Reliable processing: Local file usage eliminates FFmpeg data URL processing issues

Architecture Benefits

• TypeScript strict mode: Full type safety with comprehensive error checking
• Domain-driven design: Clean separation between business logic and infrastructure
• Comprehensive logging: Structured logging with Pino for observability and debugging
• Configuration validation: Environment-based config with Zod schema validation