@camera.ui/rust-detector

A high-performance Rust-based motion detection library with GPU acceleration using WebGPU, optimized for real-time video surveillance and computer vision applications.

Features

• 🚀 GPU Acceleration - Hardware-accelerated motion detection using WebGPU compute shaders
• 📹 Motion Detection - Advanced frame differencing with Gaussian blur and morphological operations
• ⚡ SIMD Optimization - CPU-based SIMD acceleration for fallback operations
• 🎯 Bounding Box Detection - Automatic detection and tracking of moving objects
• 🔧 Multi-Backend Support - Vulkan, Metal, DirectX12, and OpenGL
• 📊 Real-time Performance - Optimized for streaming and live video processing
• 🔄 Adaptive Processing - Automatic fallback from GPU to CPU when needed
• 🎛️ Configurable Parameters - Adjustable threshold, blur, and dilation settings

Installation

npm install @camera.ui/rust-detector

Quick Start

CPU-based Motion Detection

import { ImageProcessor } from '@camera.ui/rust-detector';

const processor = new ImageProcessor(1920, 1080);

// Process frame and detect motion
const boundingBoxes = processor.processImage(
  frameData,      // Uint8Array grayscale frame
  30,             // threshold (0-255)
  5,              // blur kernel size
  3,              // dilation size
  100             // minimum area
);

// Each bounding box: [x, y, x+width, y+height]
boundingBoxes.forEach(box => {
  console.log(`Motion detected at: x=${box[0]}, y=${box[1]}, w=${box[2]-box[0]}, h=${box[3]-box[1]}`);
});

GPU-accelerated Motion Detection

import { GpuImageProcessor } from '@camera.ui/rust-detector';

const processor = new GpuImageProcessor(1920, 1080);

// Initialize with preferred GPU backend
const adapterDetails = await processor.initialize('vulkan');
console.log(`Using GPU: ${adapterDetails.name} (${adapterDetails.backend})`);

// Process frame with GPU acceleration
const boundingBoxes = processor.processImage(
  frameData,      // grayscale frame buffer
  30,             // threshold
  5,              // blur kernel size
  3,              // dilation size
  100,            // minimum area
  false           // debug mode (optional)
);

Real-time Video Surveillance Pipeline

import { GpuImageProcessor, ImageProcessor, getGpuBackends } from '@camera.ui/rust-detector';

class MotionDetector {
  private gpuProcessor?: GpuImageProcessor;
  private cpuProcessor: ImageProcessor;
  private useGpu = false;

  constructor(width: number, height: number) {
    this.cpuProcessor = new ImageProcessor(width, height);
  }

  async initialize(preferredBackend: string = 'auto') {
    try {
      const availableBackends = getGpuBackends();
      console.log('Available GPU backends:', availableBackends);

      this.gpuProcessor = new GpuImageProcessor(1920, 1080);
      const adapter = await this.gpuProcessor.initialize(preferredBackend);
      
      console.log(`GPU acceleration enabled: ${adapter.name}`);
      this.useGpu = true;
      return adapter;
    } catch (error) {
      console.warn('GPU initialization failed, falling back to CPU:', error);
      this.useGpu = false;
      return null;
    }
  }

  detectMotion(frame: Uint8Array, config: MotionConfig) {
    const processor = this.useGpu ? this.gpuProcessor : this.cpuProcessor;
    
    if (!processor?.initialized) {
      throw new Error('Processor not initialized');
    }

    return processor.processImage(
      frame,
      config.threshold,
      config.blurKernel,
      config.dilationSize,
      config.minArea,
      config.debug
    );
  }

  cleanup() {
    this.gpuProcessor?.resetState();
    this.cpuProcessor.resetState();
  }
}

interface MotionConfig {
  threshold: number;      // Motion sensitivity (0-255)
  blurKernel: number;     // Gaussian blur kernel size
  dilationSize: number;   // Morphological dilation size
  minArea: number;        // Minimum area for motion detection
  debug?: boolean;        // Enable debug output
}

API Reference

GPU Image Processor

Initialization

class GpuImageProcessor {
  readonly processorType: string;  // "GPU"
  readonly initialized: boolean;
  
  constructor(width: number, height: number);
  
  // Initialize GPU with preferred backend
  initialize(preferredBackend?: string): Promise<AdapterDetails>;
  
  // Reset GPU state and free resources
  resetState(): void;
}

Motion Detection

// GPU-accelerated motion detection
processImage(
  frame: Buffer,           // Grayscale frame data
  threshold: number,       // Motion threshold (0-255)
  kernelSize: number,      // Gaussian blur kernel size
  dilateSize: number,      // Morphological dilation size
  minArea: number,         // Minimum area for detection
  debug?: boolean          // Enable debug output
): number[][];             // Array of bounding boxes [x1, y1, x2, y2]

CPU Image Processor

Initialization

class ImageProcessor {
  readonly processorType: string;  // "CPU"
  readonly initialized: boolean;
  
  constructor(width: number, height: number);
  
  // Reset processor state
  resetState(): void;
}

Motion Detection

// CPU-based motion detection
processImage(
  frame: Uint8Array,       // Grayscale frame data
  threshold: number,       // Motion threshold (0-255)
  kernelSize: number,      // Gaussian blur kernel size
  dilateSize: number,      // Morphological dilation size
  minArea: number          // Minimum area for detection
): number[][];             // Array of bounding boxes [x1, y1, x2, y2]

Configuration Types

GPU Backend Options

type GpuBackend = 'vulkan' | 'metal' | 'directx12' | 'opengl' | 'auto';

// Get available GPU backends
getGpuBackends(): string[];

Adapter Details

interface AdapterDetails {
  name: string;           // GPU name
  vendor: number;         // Vendor ID
  device: number;         // Device ID
  deviceType: string;     // Device type
  driver: string;         // Driver name
  driverInfo: string;     // Driver info
  backend: string;        // Backend type
}

Bounding Box Format

type BoundingBox = [number, number, number, number];  // [x1, y1, x2, y2]

Algorithm Details

Motion Detection Pipeline

1. Gaussian Blur - Reduces noise and smooths the image
2. Frame Differencing - Compares current frame with previous blurred frame
3. Thresholding - Converts differences to binary motion mask
4. Morphological Dilation - Fills gaps and expands motion regions
5. Connected Component Analysis - Finds connected motion regions
6. Bounding Box Extraction - Calculates bounding boxes for each region

GPU Implementation

The GPU implementation uses WebGPU compute shaders with optimized workgroup sizes:

• Horizontal Blur Pass - Separable Gaussian blur (horizontal)
• Vertical Blur Pass - Separable Gaussian blur (vertical)
• Difference & Dilate Pass - Combined frame differencing and morphological dilation

CPU Implementation

The CPU implementation uses SIMD optimization for:

• Frame Differencing - Vectorized pixel comparison
• Connected Components - Efficient flood-fill algorithm
• Gaussian Blur - SIMD-accelerated convolution

Platform Support

Supported Platforms

• Windows - x64 (DirectX12, Vulkan, OpenGL)
• macOS - Universal (Metal, Vulkan via MoltenVK)
• Linux - x64, ARM64 (Vulkan, OpenGL)
• FreeBSD - x64 (Vulkan, OpenGL)

GPU Backend Availability

| Platform | Vulkan | Metal | DirectX12 | OpenGL | |----------|--------|-------|-----------|--------| | Windows | ✅ | ❌ | ✅ | ✅ | | macOS | ✅* | ✅ | ❌ | ✅ | | Linux | ✅ | ❌ | ❌ | ✅ | | FreeBSD | ✅ | ❌ | ❌ | ✅ |

*Vulkan on macOS requires MoltenVK

Requirements

• Modern GPU with WebGPU support (for GPU acceleration)
• Platform-specific requirements:
  • Windows: DirectX 12 or Vulkan drivers
  • macOS: Metal support (macOS 10.14+)
  • Linux: Vulkan or OpenGL drivers

Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests to our repository.

License

MIT

*Part of the camera.ui ecosystem - A comprehensive camera management solution