@voqalize/avatar

A talking head for AI voice calls. Give it your live PipecatClient and an element, and a character in that element speaks the bot's audio, lip-synced, and shows the state of the call: listening, thinking, being interrupted.

This package is the loader. The avatar itself (the characters, the renderer and everything that follows the call) is the Voqalize avatar runtime, which this package imports from https://cdn.voqalize.com when the first avatar mounts. Each release of this package is pinned to the runtime it was released with, so upgrading the package is how you upgrade the avatar.

Install

npm install @voqalize/avatar

The package is ESM, has no runtime dependencies and ships its own types. @pipecat-ai/client-js (>=1.4 <2) and React (>=18) are optional peers, and React is used only by @voqalize/avatar/react. The client you pass is a PipecatClient, or any client with the members the exported AvatarClientLike type names; the package imports nothing from pipecat, not even its types.

Mount a character

Mount it wherever your app renders the bot's tile:

import { createAvatar } from "@voqalize/avatar";

const avatar = createAvatar({
  mount: document.querySelector("#bot-tile"),
  client: pipecatClient,
  character: "<name>",
});

// when the call ends, or the tile goes away
avatar.destroy();

createAvatar returns at once, and the face appears when the runtime and the character have loaded. The avatar owns the contents of mount and sizes itself to it. destroy() is safe to call at any time, including before the face has appeared.

character names one of the characters you can meet at https://voqalize.com/demos/avatar. Each release of this package can mount a fixed set of them. A name it does not have mounts nothing: createAvatar still returns, and the error goes to the console, listing the names it does have. listCharacters() returns that set:

import { listCharacters } from "@voqalize/avatar";

for (const c of await listCharacters()) {
  // c.name, c.still (an image URL), c.gender, c.age.min, c.age.max,
  // c.ethnicity[], c.tags[], c.suggestedVoices[], c.languages[]
}

Use it to build a picker or a filter rather than writing the names down. The words in gender, ethnicity and tags come from closed lists, so a filter you build from one call matches the next. suggestedVoices lists the voices that suit the face, best first, as Voqalize voice ids such as omnivoice/gauri. It is advice: the avatar never speaks. languages lists the languages the first voice speaks, default first ("en" or "hi"). Send one as the language of every speech request: without it the voice falls back to English, and Hindi text comes out with an English accent.

The gains are optional, from 0 to 2, and 1 is as authored:

• mouthGain sets how far the mouth moves.
• gestureGain sets the size of nods and other gestures.
• motionGain sets the amount of idle and speaking motion: the head, trunk and shoulders. Left out, it is 1, or 0.5 when the reader's system asks for reduced motion.

The client must be connected to a Voqalize session that has the avatar turned on. Nothing in the transport changes: the avatar draws in the browser from the session's data-channel messages, so there is no video track to turn on, and video_out_enabled (the path for server-rendered video avatars) stays off.

Preload before the call

Building a face takes the browser a moment: the runtime, the character, a GPU context and its shaders. On a slow phone that is over a second after the call connects before the face appears. preloadAvatar does that work before the call, off screen, so the mount that follows only moves the finished face onto the page:

import { preloadAvatar, unloadAvatar } from "@voqalize/avatar";

// when the visitor is about to talk: the page with the call button, or hover on it
const face = preloadAvatar("<name>");

// later, the ordinary mount takes the preloaded face
createAvatar({ mount, client, character: "<name>" });

// or, if the visitor leaves without calling
unloadAvatar(face);

• One face is held at a time. Preloading the same character again returns the same handle; preloading another frees the one held.
• A mount of that character, through createAvatar or <Avatar>, takes the face. After that the handle holds nothing and unloadAvatar does nothing, and destroy() frees the face as usual.
• face.ready settles when the face is built, or when it cannot be (no WebGL 2, for example) and the mount will build it the ordinary way. You never need to wait for it before mounting. It rejects only for a name that is no character.
• unloadAvatar frees the GPU context and the character's memory at once, and is safe to call at any time.

React

import { Avatar } from "@voqalize/avatar/react";

<Avatar client={client} character="<name>" className="bot-tile" />

Nothing mounts while client is null. A new client or character rebuilds the avatar. The gains are read when the avatar mounts. Every other prop goes to the <div> the avatar renders into.

Content Security Policy

If your page sets a Content-Security-Policy, add our host to these directives:

script-src  https://cdn.voqalize.com
connect-src https://cdn.voqalize.com
img-src     https://cdn.voqalize.com

• script-src, because the runtime is a module imported from there.
• connect-src, because the runtime fetches the character from there.
• img-src, for the still photograph shown where a browser has no WebGL 2, as a call whose other side has turned their camera off. Without it that tile shows an empty circle; nothing else is affected.

The runtime needs nothing else: no blob:, no data:, no workers, no inline styles and no eval. Safari before 17 and Firefox before 98 try one image from a blob: URL first; the policy refuses it, the face draws all the same, and a policy with a report-to gets one report from each such page. Your page's own call (its connect-src for the voice service, its microphone permission) is not part of this list.

A policy applies to the page that sends it, so only the pages that mount an avatar need these lines. To try them before you enforce them, send the policy as Content-Security-Policy-Report-Only: the browser reports what it would refuse and refuses nothing.

Upgrading from 0.6

Releases before 0.7 load from https://avatar.voqalize.com and asked for blob: as well. That host serves the same files and stays up, and a policy written for it keeps working with this release: the runtime is imported from whichever name your policy allows (if your policy has a report-to, expect one report for the name it refuses). Change your policy after you upgrade the package, not before: a 0.6 package under the policy above is refused its runtime. Once every page runs 0.7 or later you can drop avatar.voqalize.com and blob: from your policy.

Bundlers

The runtime is imported with a literal import() of its URL, marked so that Vite and webpack leave it alone. Your bundle carries only this package, a few kilobytes, and the browser fetches the runtime once per page however many avatars mount.

Licence

This package is MIT; see LICENSE. The runtime it loads is not open source. Its terms are at https://cdn.voqalize.com/LICENSE.