Streamoji SDK

A React SDK for avatar-based video meetings with real-time face tracking and peer-to-peer communication using MediaPipe face landmarks.

Installation

npm install streamoji-sdk-react

Quick Start

1. Wrap your app with StreamojiProvider

import { StreamojiProvider } from 'streamoji-sdk-react';

function App() {
  return (
    <StreamojiProvider>
      <YourApp />
    </StreamojiProvider>
  );
}

2. Authentication

const { isAuthenticated, streamojiSignIn } = useStreamoji();

useEffect(() => {
   // STEP 1: Sign in with your app's credentials
  await streamojiSignIn(
    userId, // your app's userID
    clientName, // your app's client name
    clientSecret, // your app's client secret 
    userProfile?.avatar // optional: avatar data if you have it
  );
})

if(!isAuthenticated) return null; // only render when isAuthencated is true

3. Creating Meeting

const meetingId = await createMeeting();

4.Rendering Avatars

    <StreamojiView
        meetingId={meetingId}
        userProfile={{
            userId: userProfile?.userId || "",
            avatar: userProfile?.avatar || undefined, // you will have to store the avatar data on your end
        }}
        videoSize={videoSize}
    />

5. Next.js Configuration (Required for WebWorker)

If you're using Next.js, you need to configure your Next.js config file to properly handle the webworker file.

Option A: CommonJS (if your project doesn't use ES modules)

Create or update next.config.js:

/** @type {import('next').NextConfig} */

const path = require('path');
const fs = require('fs');

// Copy worker file to public directory
const copyWorkerFile = () => {
  try {
    const workerSource = path.join(__dirname, 'node_modules/streamoji-sdk-react/dist/workers/faceLandmarker.worker.js');
    const workerDest = path.join(__dirname, 'public/workers/faceLandmarker.worker.js');
    
    // Create workers directory if it doesn't exist
    const workersDir = path.join(__dirname, 'public/workers');
    if (!fs.existsSync(workersDir)) {
      fs.mkdirSync(workersDir, { recursive: true });
    }
    
    // Copy the file if it exists
    if (fs.existsSync(workerSource)) {
      fs.copyFileSync(workerSource, workerDest);
      console.log('✓ Worker file copied to public directory');
    } else {
      console.warn('⚠ Worker file not found at:', workerSource);
    }
  } catch (error) {
    console.error('Error copying worker file:', error);
  }
};

// Copy worker file on config load
copyWorkerFile();

const nextConfig = {
  output: 'export',
  reactStrictMode: false, // NOTE: important! for making this work
  webpack: (config, { isServer }) => {
    if (!isServer) {
      // Alias the worker path to public directory
      config.resolve.alias = {
        ...config.resolve.alias,
        '../workers/faceLandmarker.worker.js': path.resolve(__dirname, 'public/workers/faceLandmarker.worker.js'),
      };
    }
    // Make streamoji-sdk-react external during SSR to avoid ES module issues
    if (isServer) {
      config.externals = config.externals || [];
      if (Array.isArray(config.externals)) {
        config.externals.push('streamoji-sdk-react');
      } else {
        config.externals = [config.externals, 'streamoji-sdk-react'];
      }
    }
    return config;
  },
};

module.exports = nextConfig;

Option B: ES Modules (if your project uses "type": "module" in package.json)

Create or update next.config.mjs (or rename next.config.js to next.config.mjs):

/** @type {import('next').NextConfig} */

import path from 'path';
import fs from 'fs';
import { fileURLToPath } from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

// Copy worker file to public directory
const copyWorkerFile = () => {
  try {
    const workerSource = path.join(__dirname, 'node_modules/streamoji-sdk-react/dist/workers/faceLandmarker.worker.js');
    const workerDest = path.join(__dirname, 'public/workers/faceLandmarker.worker.js');
    
    // Create workers directory if it doesn't exist
    const workersDir = path.join(__dirname, 'public/workers');
    if (!fs.existsSync(workersDir)) {
      fs.mkdirSync(workersDir, { recursive: true });
    }
    
    // Copy the file if it exists
    if (fs.existsSync(workerSource)) {
      fs.copyFileSync(workerSource, workerDest);
      console.log('✓ Worker file copied to public directory');
    } else {
      console.warn('⚠ Worker file not found at:', workerSource);
    }
  } catch (error) {
    console.error('Error copying worker file:', error);
  }
};

// Copy worker file on config load
copyWorkerFile();

const nextConfig = {
  output: 'export',
  reactStrictMode: false, // NOTE: important! for making this work
  webpack: (config, { isServer }) => {
    if (!isServer) {
      // Alias the worker path to public directory
      config.resolve.alias = {
        ...config.resolve.alias,
        '../workers/faceLandmarker.worker.js': path.resolve(__dirname, 'public/workers/faceLandmarker.worker.js'),
      };
    }
    // Make streamoji-sdk-react external during SSR to avoid ES module issues
    if (isServer) {
      config.externals = config.externals || [];
      if (Array.isArray(config.externals)) {
        config.externals.push('streamoji-sdk-react');
      } else {
        config.externals = [config.externals, 'streamoji-sdk-react'];
      }
    }
    return config;
  },
};

export default nextConfig;

Note: If you're getting a "require is not defined" error, use Option B (ES Modules) with next.config.mjs.

6. Add Worker File to .gitignore

Since the worker file is automatically copied from node_modules during the build process, you should add it to your .gitignore to avoid committing generated files:

# Streamoji SDK worker file (auto-generated)
public/workers/faceLandmarker.worker.js

Alternatively, you can ignore the entire workers directory:

# Streamoji SDK worker files (auto-generated)
public/workers/

Example (all in one)

Before rendering your app, you must authenticate the user using the streamojiSignIn method. Do not render your main app/UI until authentication is complete.

Example:

import { useStreamoji } from 'streamoji-sdk-react';

function AppWrapper() {
  const { isAuthenticated, streamojiSignIn } = useStreamoji();
  const [meetingId, setMeetingId] = useState();

  React.useEffect(() => {
    const authenticateUser = async () => {
      try {
        // STEP 1: Sign in with your app's credentials
        await streamojiSignIn(
          userId, // your app's userID
          clientName, // your app's client name
          clientSecret, // your app's client secret
          userProfile?.avatar // optional: avatar data if you have it
        );

        // STEP 2: Create or join a meeting
        const meetingId = await createMeeting();
        setMeetingId(meetingId);
      } catch (error) {
        console.error('Authentication failed:', error);
      }
    };

    authenticateUser();
  }, [streamojiSignIn]);

  if (!isAuthenticated || !meetingId) {
    return <div>Loading...</div>; // Or your loading/auth UI
  }

  return <div>
            <StreamojiView
                meetingId={meetingId}
                userProfile={{
                    userId: userProfile?.userId || "",
                    avatar: userProfile?.avatar || undefined, // you will have to store the avatar data on your end
                }}
                videoSize={videoSize}
            />
        </div>;
}

Important Note on Data Storage: Streamoji does not store any user information or avatar data. All user data, including avatar configurations, must be stored and managed by your application. We only facilitate the real-time avatar meetings. You are responsible for:

• Storing and managing user profiles
• Persisting avatar configurations
• Managing authentication tokens
• Handling user sessions

When joining meetings, you'll need to provide the avatar data that you've stored on your end. This gives you complete control over your users' data and ensures compliance with your data handling policies.

Core Components

StreamojiView

The main component for avatar-based meetings with face tracking.

CameraAvatarCreator

Component for creating avatars using camera input and face landmarks.

useStreamoji() Hook

StreamojiProvider and useStreamoji

The StreamojiProvider context and useStreamoji hook provide access to core Streamoji functionality:

const {
  userProfile,
    remoteUsers,
    currentMeetingId,
    createMeeting,
    joinMeeting,
    leaveMeeting,
    endMeeting,
    onPeerJoined,
    onPeerLeft,
    onMeetingEnded,
    removeOnPeerJoined,
    removeOnPeerLeft,
    removeOnMeetingEnded,
    clearAllOnPeerJoinedCallbacks,
    clearAllOnPeerLeftCallbacks,
    clearAllOnMeetingEndedCallbacks,
    clearAllOnMeetingCreatedCallbacks,
    clearMeetingSubscriptions,
    isAuthenticated,
    streamojiSignIn,
    onAvatarChanged,
    removeOnAvatarChanged,
    clearAllOnAvatarChangedCallbacks,
} = useStreamoji()

Types

• StreamojiUserProfile: User profile interface
• MeetingParticipant: Meeting participant interface
• StreamojiMeeting: Meeting interface
• StreamojiAvatar: Avatar data type