@threadlane/agent-avatar
v0.3.0
Published
Plush animated AI agent avatars for React: any shape, any colour, lip-synced speaking, and animations an LLM can write as safe JSON.
Maintainers
Readme
@threadlane/agent-avatar
From Threadlane.
Plush, animated AI agent avatars for React. Pick any agent, or make your own: 143 shapes, 21 materials, any colour and any transparency. It talks too, lip-synced to text or a live voice. It can act out 69 jobs and show any feeling, drawn on a canvas with no images or WebGL. React 18 or later is the only requirement.
npm install @threadlane/agent-avatarimport { AgentAvatar } from '@threadlane/agent-avatar'
<AgentAvatar agent="guardian" size={96} /> // a ready-made agent
<AgentAvatar shape="dog" color="#4c8df6" material="clay" play="happy" /> // your own look
<AgentAvatar shape="cat" play="doctor:sad" state="working" /> // a job with a feeling, on a loop
<AgentAvatar shape="bear" play="chef+dancing" emotion="very excited" /> // mix anything
<AgentAvatar shape="fox" emotion="a bit nervous but very hopeful" /> // any feeling, in words
<AgentAvatar shape="bean" state="speaking" speech="Hello! How can I help?" /> // talks, lip-syncedSee every shape and animation, and build yours, in Agent Maker.
What it can play
- Ready-made agents:
guardian,calculator,fx,payout,audit,dispute,memory,bookkeeper. - Moods and moves (30):
happy,sad,running,dancing,crying,dizzyand more (MOODS). - Jobs (69, across 13 sectors):
doctor,chef,shopping-spree,powdering,runway-modeland more (JOBS). - A job with a feeling:
doctor:sad.jobPack('doctor')lists a job's 17 ready-made plays. - Any feeling:
emotiontakes words or a blend such as{ joy: 0.6, sadness: 0.5 }.
Props
| Prop | Default | What it does |
|---|---|---|
| agent | | A ready-made agent's id, or your own definition with its animations |
| play | | What to play: a name, job:feeling, or several joined with + |
| emotion | | A feeling, alone or while doing play |
| intensity | 1 | How strongly emotion shows, 0 to 1 |
| shape | circle | Body shape (SHAPE_NAMES) |
| color | #2C6BED | Body colour, #rrggbb |
| material | plush | plush, fluffy, clay, vinyl, balloon, glass, metal, knit, pixel, sticker, sketch, neon, pastel, velvet, jelly, pearl, porcelain, satin, frosted, holo or rubber |
| transparency | 0 | How see-through the body is, 0 (solid) to 1. The face stays clear |
| size | 64 | Size in pixels |
| state | idle | idle, working (loops the animation), sleeping or speaking (lip-synced talking) |
| speech | | While speaking: the words it says. The mouth lip-syncs them once; set new words to say more |
| speechRate | 1 | Speaking speed (1 is about 12 letters a second). Match it to your voice |
| viseme | | While speaking: the mouth shape right now, from a TTS viseme stream. Wins over speech |
| onSpeechEnd | | Called when the mouth has finished saying speech |
| speaker | | A realtime voice from useSpeaker(): the avatar talks whenever it talks |
| interactive | true | Click, Enter or Space replays it |
| trackCursor | true | The head follows the cursor |
| label | agent's name | Accessible name |
Speaking
state="speaking" makes the agent talk. Give it speech and the mouth lip-syncs the words: a shape for each vowel (A, E, I, O, U) and for the consonants that look different (MBP lips closed, FV teeth on lip, L tongue up, TH tongue out, CH lips pushed, S teeth together, N tongue tap), blending smoothly between them. Without speech it chats along, handy while audio plays.
To match a voice exactly, drive the mouth yourself: pass viseme from your TTS engine's viseme events, or line the text up with your audio using textToVisemes(text, rate), visemeAt(cues, seconds) and speechSeconds(text, rate). MOUTHS describes every shape.
<AgentAvatar shape="bean" state="speaking" speech={reply} speechRate={1.1} onSpeechEnd={() => setTalking(false)} />Realtime agents
Voice agents stream audio, and sometimes a transcript, and users interrupt them. Give the avatar a speaker and feed it whatever your pipeline has; the avatar talks whenever the speaker talks, in any state.
import { AgentAvatar, useSpeaker } from '@threadlane/agent-avatar'
const speaker = useSpeaker()
<AgentAvatar shape="mochi" material="jelly" speaker={speaker} />
// live audio, from any voice engine (OpenAI Realtime over WebRTC, ElevenLabs, Gemini Live, …)
pc.ontrack = (e) => { audioEl.srcObject = e.streams[0]; speaker.listen(e.streams[0]) }
speaker.listen(audioEl) // an <audio> element
speaker.listen(outputNode, { context: ctx }) // your own Web Audio graph (PCM chunks you play yourself)
// or text as it streams in, or exact timings from a TTS engine
speaker.say(transcriptDelta)
speaker.cues(fromAlignment(elevenLabsAlignment), { start: performance.now() / 1000 })
synthesizer.visemeReceived = (_, e) => speaker.viseme(fromAzureViseme(e.visemeId))
speaker.stop() // the user interrupted: the mouth closes at onceLive audio sets the mouth from the voice itself: loudness opens the jaw, low formants round it (o, oo), higher ones spread it (e, ee) and a hiss shows the teeth (s, sh). It learns the room's noise so a steady hum doesn't count as talking. In Chrome, a WebRTC stream also needs to play in an <audio> element for Web Audio to hear it.
Your own animations
Animations are plain JSON, never code, so an LLM can write them safely. Check one with parseAnimation, then put it in your agent's animations. ANIMATION_GUIDE tells an LLM how to write one, and the MCP server lets it do this directly.
Good to know
- It renders in the browser. On the server it outputs an empty box, and the canvas starts drawing once the page loads.
- All avatars share one animation loop, skip drawing when off screen, and stand still under
prefers-reduced-motion(a speaking mouth still moves, since it carries meaning).
Support the project
It's free to use. If it helps you, you can chip in from $5.
Licence
MIT
