@convai/web-sdk

Real-time conversational AI characters for the web.

TypeScript-first SDK for embedding Convai AI characters into React and vanilla JS applications. Voice, text, lipsync, emotions, video, and screen share — all in one package.

Features

• React & vanilla JS — useConvaiClient hook, ConvaiWidget, and a framework-agnostic core
• Real-time audio/video — full-duplex WebRTC with echo cancellation, camera, and screen share
• Lipsync — ARKit and MetaHuman blendshape streams for facial animation
• Emotions — per-turn emotion detection with intensity scale
• Dynamic context — inject game state, user preferences, or scene data mid-session
• Long-term memory — persistent cross-session memories scoped to each end user
• File upload — send images to the character during a live session
• WebSocket transport — opt-in alternative to WebRTC for constrained networks
• Auth tokens — server-side token exchange for production deployments

Installation

npm install @convai/web-sdk
# or
pnpm add @convai/web-sdk
# or
yarn add @convai/web-sdk

React peer dependencies: react and react-dom ^18 || ^19

Runtime requirement: secure context (https:// or http://localhost) for microphone/camera access.

Quick start

React

import { useConvaiClient, ConvaiWidget } from '@convai/web-sdk';

export function App() {
  const client = useConvaiClient({
    apiKey: import.meta.env.VITE_CONVAI_API_KEY,
    characterId: import.meta.env.VITE_CONVAI_CHARACTER_ID,
  });

  return <ConvaiWidget convaiClient={client} />;
}

Vanilla TypeScript

import { ConvaiClient, createConvaiWidget } from '@convai/web-sdk/vanilla';

const client = new ConvaiClient({
  apiKey: import.meta.env.VITE_CONVAI_API_KEY,
  characterId: import.meta.env.VITE_CONVAI_CHARACTER_ID,
});

const widget = createConvaiWidget(document.body, { convaiClient: client });

window.addEventListener('beforeunload', () => {
  widget.destroy();
  void client.disconnect();
});

Documentation

Full documentation is at docs.convai.com.

| Guide | Description | |---|---| | Quick Start | First working integration in under 5 minutes | | Configuration | All ConvaiConfig options | | React Integration | useConvaiClient, ConvaiWidget, and React-specific patterns | | Vanilla JS | ConvaiClient, createConvaiWidget, and audio setup | | Events | Full event reference — botReady, stateChange, messagesChange, interactionCreated, and more | | Context Management | Dynamic context, updateContext, file upload, session management | | Emotions | Per-turn emotion detection with provider options | | Lipsync | ARKit / MetaHuman blendshape streams and BlendshapeQueue API | | Actions | Trigger character behaviors and scene actions | | Memory | Long-term memory scoped to end users | | Audio & Video | Microphone, camera, screen share controls | | Error Handling | error, disconnect, serverResponse, retry patterns | | Auth Tokens | Server-side token exchange for production | | WebSocket Transport | Alternative transport for WebRTC-constrained environments |

Package entry points

| Import path | Contents | |---|---| | @convai/web-sdk | React hooks, components, and re-exported core types | | @convai/web-sdk/react | Same as default (React-explicit alias) | | @convai/web-sdk/vanilla | ConvaiClient, createConvaiWidget, AudioRenderer | | @convai/web-sdk/core | Framework-agnostic ConvaiClient, managers, and all types | | @convai/web-sdk/lipsync-helpers | Blendshape format utilities and queue helpers | | @convai/web-sdk/vanilla/websocket | Opt-in. Registers the WebSocket transport. Import alongside /vanilla when using transport: "websocket". |

WebSocket transport

The default transport is WebRTC (LiveKit). A WebSocket-based transport is available for environments where WebRTC is unavailable or undesirable.

// Add this import once (e.g. in your app entry file)
import '@convai/web-sdk/vanilla/websocket';
import { ConvaiClient } from '@convai/web-sdk/vanilla';

const client = new ConvaiClient({
  apiKey: '...',
  characterId: '...',
  transport: 'websocket',
});

The WebSocket packages (@pipecat-ai/client-js, @pipecat-ai/websocket-transport) are excluded from your bundle unless you import the /vanilla/websocket subpath — so LiveKit-only apps pay no bundle cost.

See the WebSocket Transport guide for the full feature comparison.

Example app

A full React Three Fiber integration demo is in examples/react-three-fiber. It demonstrates lipsync on a 3D character, emotion display, dynamic context, and file upload.

License

Licensed under the Apache License 2.0. Copyright 2025 Convai.