@talking-avatar/player

<TalkingAvatar>: a React component that plays one line of a talking-avatar character folder with synced mouth shapes and a caption. MIT license.

pnpm add @talking-avatar/[email protected]

react 18 or 19 is a peer dependency.

Usage

Serve the character folder (asset contract v1) as static files, e.g. at /avatars/keeper/. The server must send .m4a as audio/mp4 and answer Range requests with 206 (iPad Safari needs both).

import { avatarManifestSchema } from "@talking-avatar/formats";
import { TalkingAvatar } from "@talking-avatar/player";

const manifest = avatarManifestSchema.parse(await (await fetch("/avatars/keeper/manifest.json")).json());

<TalkingAvatar
  manifest={manifest}
  lineId="ts-keeper-001"
  baseUrl="/avatars/keeper/"
  onEnd={() => next()}
/>;

| Prop | | | --- | --- | | manifest | parsed manifest.json | | lineId | line to play; changing it plays the next line on the same audio element | | baseUrl | URL of the character folder; manifest paths resolve against it | | onEnd | called once when the line's audio ends | | onError | optional; unknown lineId, timing that fails to load or parse, audio errors, unsupported mouthMode | | autoPlay | default true | | className, style | on the outer <figure> |

Behaviour:

• Every animation frame samples audio.currentTime and shows the mouth of the latest cue at or before it. REST before the first cue, after a cue's end, and once the audio has ended. Pause and seek follow currentTime.
• mouthMode: "frame": each mouth file is a full portrait; the visible frame is swapped in the portrait's box and scales with it. overlay is not supported yet (portrait and caption only, plus onError).
• All mouth frames and every line's timing load when the component mounts; during playback only the audio is fetched.
• Caption: the line text with [pause N] removed. captionTheme is ignored for now.
• Browsers block audio without a user gesture. When play() is refused, the avatar shows a "Tap to play" button. Once a tap has started it, later lines on the same component play on their own, iPad Safari included.

Example page

pnpm --filter @talking-avatar/player example          # http://localhost:5173
pnpm --filter @talking-avatar/player example --host   # reachable from an iPad on the same network

It plays the Keeper fixture from @talking-avatar/formats (placeholder art and voice). Each mouth frame is labelled with its shape name.