seg-cam

An easy-to-use drop-in library that wraps around TensorFlow.js BodyPix. Using it is as simple as passing a video reference to the JerseyDetector component.

seg-cam was created for real-time video processing—optimized for high-performance jersey and person segmentation in browser environments.

Features

• Live Segmentation: High-performance person and body part segmentation.
• Self-Contained Worker: Fully bundled Web Worker for background processing, ensuring zero UI lag.
• CSP Compliant: No external CDNs or importScripts used at runtime.
• Highly Configurable: Control architecture, thresholds, and target body parts.
• React Ready: Optimized for React with a simple component and hook API.

Installation

1. Install the Library

npm install seg-cam
# or
pnpm add seg-cam

2. Setup the Web Worker

Because the heavy ML processing happens in a Web Worker, you must provide the worker file. When you build the project or install it, the worker bundle is available at: node_modules/seg-cam/public/jersey-detector-worker.js (for dev) or node_modules/seg-cam/dist/jersey-detector-worker.js (for production).

Option A: Copy to your public folder (Recommended) Copy jersey-detector-worker.js to your project's public/ folder so it can be served at exactly /jersey-detector-worker.js.

Option B: Custom Hosting Host the worker file anywhere and pass the URL to the component using the workerUrl prop.

Usage

import { useRef, useState } from 'react';
import { JerseyDetector } from 'seg-cam';

export function CameraApp() {
  const videoRef = useRef<HTMLVideoElement>(null);
  const [stats, setStats] = useState(null);

  return (
    <div>
      <video ref={videoRef} autoPlay muted playsInline />
      
      <JerseyDetector 
        videoRef={videoRef}
        workerUrl="/jersey-detector-worker.js" // Path to your hosted worker file
        onStatsUpdate={(newStats) => setStats(newStats)}
        threshold={0.6}
        targetPartId={2}
      />

      {stats && (
        <div>
          Jerseys Found: {stats.jerseyCount} | FPS: {stats.fps}
        </div>
      )}
    </div>
  );
}

API Reference

JerseyDetector Props

| Prop | Type | Default | Description | | :--- | :--- | :--- | :--- | | videoRef | React.RefObject<HTMLVideoElement> | Required | Reference to the video source to process. | | workerUrl | string | '/jersey-detector-worker.js' | URL to the bundled worker JS file. | | threshold | number | 0.5 | Confidence threshold for detection (0 to 1). | | targetPartId | number | 12 | BodyPix Part ID to isolate (12 = Torso). | | imagesToReturn | 'mask' \| 'mask_with_image' \| 'none' | 'mask' | Controls output: 'mask' (overlay only), 'mask_with_image' (overlay + frame), 'none' (data only). | | onStatsUpdate| (stats: DetectionStats) => void | - | Throttled (500ms). Callback triggered with detection metrics. | | onWorkerReady| (ready: boolean) => void | - | Callback triggered when the TFJS model is loaded. | | onWorkerError| (error: string \| null) => void | - | Callback triggered if the worker fails to initialize. | | onSegmentedImage| (images: ImageBitmap[], bboxes?: BoundingBox[]) => void| - | Throttled (500ms). Returns individual bitmaps and bounding boxes of detected objects (tlbr format). | | bodyPixArchitecture| 'MobileNetV1' \| 'ResNet50' | 'MobileNetV1' | Performance vs Accuracy trade-off. | | verbose | boolean | false | Enable detailed console logging. |

DetectionStats Type

interface DetectionStats {
  jerseyCount: number;      // Number of unique jerseys detected
  confidence: number;       // Average confidence score
  processingTime: number;   // Time spent on CPU in ms per frame
  fps: number;              // Real-time processing FPS
}

JerseyDetectorHandle (Inverted Ref)

You can use a ref on the JerseyDetector component to access its internal state imperatively:

const detectorRef = useRef<JerseyDetectorHandle>(null);

// Access current state:
const isReady = detectorRef.current?.isReady;
const currentStats = detectorRef.current?.stats;

Advanced Configuration

seg-cam uses BodyPix under the hood. You can fine-tune the detection and rendering by passing these props:

Model Parameters

• bodyPixArchitecture: ('MobileNetV1' | 'ResNet50') MobileNet is faster; ResNet is more accurate.
• bodyPixMultiplier: (0.5, 0.75, 1.0) The internal width of the model's layers. Smaller is faster.
• bodyPixStride: (8, 16, 32) The output stride. Smaller is more accurate but slower.
• bodyPixQuantBytes: (1, 2, 4) Quantization level. 2 is a good balance.
• multiSegmentation: (boolean, Default true) Whether to detect multiple people simultaneously.
• segmentBodyParts: (boolean, Default true) Whether to compute specific body part IDs.

Rendering & Masking

• targetPartId: (number, Default 12) The BodyPix part ID to treat as the "Jersey". (e.g., 12=torso, 2=full upper body).
• backgroundShade: (number, 0-255, Default 0) The alpha/opacity for non-jersey pixels.
• shirtShade: (number, 0-255, Default 170) The alpha/opacity for the detected jersey pixels.
• imagesToReturn: ('mask' | 'mask_with_image' | 'none', Default 'mask')
  • 'mask': Returns only the overlay mask (clears canvas before drawing).
  • 'mask_with_image': Draws the overlay on top of the existing canvas (may trail if not cleared externally).
  • 'none': Skips bitmap generation entirely. Use this if you only need onSegmentedImage or stats, improving performance.
• threshold: (number, 0.0-1.0, Default 0.5) The minimum confidence required for a person pixel to be considered valid.

Performance Tips

To get the most out of seg-cam v1.0.0:

1. Use imagesToReturn="none" if you only need the data (stats, segmented images, or bboxes). This bypasses the creation of the full-screen overlay bitmap.
2. Optimize your videoRef dimensions: Processing smaller video streams (e.g., 640x480) is significantly faster than 1080p.
3. Architecture Choice: MobileNetV1 with a multiplier of 0.75 or 0.5 is the sweet spot for mobile devices.
4. Quantization: Stick to bodyPixQuantBytes={2} for a good balance between model size and accuracy.
5. Notification Throttling: By default, onStatsUpdate and onSegmentedImage are throttled to 500ms to ensure the main thread remains responsive. They will still fire immediately if the number of detected jerseys changes.

Acknowledgements

This project is powered by TensorFlow.js and the BodyPix model. Detailed attributions for third-party software and models can be found in the NOTICE file.

License

Copyright © 2026 [Seg-Cam Team]. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).