virtual-background-ai
v1.0.0
Published
AI-powered virtual background replacement using TensorFlow.js and Robust Video Matting
Maintainers
rajat0206Readme
Virtual Background AI
AI-powered virtual background replacement using TensorFlow.js and Robust Video Matting. This package provides easy-to-use methods for applying virtual backgrounds to video elements in real-time.
🚀 Features
- Real-time Background Replacement: Instant AI-powered background removal and replacement
- Multiple Background Types: Color backgrounds, image backgrounds, and blur effects
- TypeScript Support: Full TypeScript definitions included
- Memory Efficient: Proper tensor disposal and resource management
- Easy Integration: Simple API for quick implementation
📦 Installation
npm install virtual-background-aiNote: This package requires @tensorflow/tfjs as a peer dependency. Make sure to install it:
npm install @tensorflow/tfjs🎯 Quick Start
<!DOCTYPE html>
<html>
<head>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.min.js"></script>
</head>
<body>
<video id="video" autoplay muted playsinline></video>
<div id="output"></div>
<script type="module">
import { applyColorBackground } from 'virtual-background-ai';
const video = document.getElementById('video');
const output = document.getElementById('output');
// Get webcam stream
navigator.mediaDevices.getUserMedia({ video: true })
.then(stream => {
video.srcObject = stream;
// Apply red background after video loads
video.onloadedmetadata = async () => {
const canvas = await applyColorBackground(video, '#ff0000');
output.appendChild(canvas);
};
});
</script>
</body>
</html>📚 API Reference
Class: VirtualBackgroundAI
Main class for managing virtual background operations.
Constructor
new VirtualBackgroundAI(options?: VirtualBackgroundOptions)Options:
modelPath?: string- Path to the model files (default: 'model/model.json')downsampleRatio?: number- Downsample ratio for processing (default: 0.5)
Methods
initialize(): Promise<void>
Initialize the AI model. Called automatically when needed.
applyColorBackground(videoElement: HTMLVideoElement, color: string): Promise<HTMLCanvasElement>
Apply a solid color background.
Parameters:
videoElement- HTML video elementcolor- Color in hex format (e.g., '#ff0000') or rgb format (e.g., 'rgb(255, 0, 0)')
Returns: Promise resolving to canvas with applied background
applyImageBackground(videoElement: HTMLVideoElement, imageUrl: string): Promise<HTMLCanvasElement>
Apply an image background.
Parameters:
videoElement- HTML video elementimageUrl- URL of the background image
Returns: Promise resolving to canvas with applied background
applyBlurBackground(videoElement: HTMLVideoElement): Promise<HTMLCanvasElement>
Apply a blur background effect.
Parameters:
videoElement- HTML video element
Returns: Promise resolving to canvas with applied blur background
dispose(): void
Clean up resources and dispose of tensors.
Convenience Functions
For one-time operations, you can use these convenience functions:
applyColorBackground(videoElement, color, options?)
import { applyColorBackground } from 'virtual-background-ai';
const canvas = await applyColorBackground(video, '#00ff00');applyImageBackground(videoElement, imageUrl, options?)
import { applyImageBackground } from 'virtual-background-ai';
const canvas = await applyImageBackground(video, 'https://example.com/background.jpg');applyBlurBackground(videoElement, options?)
import { applyBlurBackground } from 'virtual-background-ai';
const canvas = await applyBlurBackground(video);🔧 Advanced Usage
Reusable Instance
For multiple operations, create a reusable instance:
import VirtualBackgroundAI from 'virtual-background-ai';
const vbg = new VirtualBackgroundAI({
modelPath: '/path/to/model',
downsampleRatio: 0.5
});
// Initialize once
await vbg.initialize();
// Apply different backgrounds
const redCanvas = await vbg.applyColorBackground(video, '#ff0000');
const imageCanvas = await vbg.applyImageBackground(video, 'background.jpg');
const blurCanvas = await vbg.applyBlurBackground(video);
// Clean up when done
vbg.dispose();Real-time Processing
import VirtualBackgroundAI from 'virtual-background-ai';
const vbg = new VirtualBackgroundAI();
await vbg.initialize();
const outputCanvas = document.getElementById('output');
async function processFrame() {
const canvas = await vbg.applyColorBackground(video, '#00ff00');
// Replace previous canvas
outputCanvas.innerHTML = '';
outputCanvas.appendChild(canvas);
// Continue processing
requestAnimationFrame(processFrame);
}
processFrame();📁 Model Files
This package requires the Robust Video Matting model files. You need to provide:
model.json- Model configurationgroup1-shard1of1.bin- Model weights
Place these files in a model/ directory relative to your HTML file, or specify a custom path in the options.
🌐 Browser Support
- Chrome 67+
- Firefox 60+
- Safari 11+
- Edge 79+
📄 License
MIT License
🙏 Acknowledgments
- PeterL1n for the original RobustVideoMatting TensorFlow.js implementation
- This package is based on the RobustVideoMatting TensorFlow.js implementation by PeterL1n. The code and pre-trained model are sourced from the original repository.
- TensorFlow.js team for the amazing framework
👨💻 Developer
Developed by Rajat Shrivastava