@shiit/avatars

XState-powered avatar system with Zdog 3D rendering.

Installation

npm install @shiit/avatars gsap zdog

Quick Start

import { Avatar } from "@shiit/avatars";
import { baseAvatarConfig } from "@shiit/avatars/configs";

const canvas = document.getElementById("avatar-canvas");
const avatar = new Avatar(canvas, baseAvatarConfig, {
  body: "#74b9ff",
  eye: "#2d3436",
  mouth: "#2d3436",
  blush: "#fd79a8",
  dead: "#636e72",
});

// Change state
avatar.setState("speaking");

// Change expression
avatar.setExpression("happy");

Running the Demo

From the package root:

# 1. Build the package
pnpm build

# 2. Run the demo
pnpm demo

The demo lets you:

• Switch between Mushroom, Cat, and Base avatars
• Trigger all 13 expressions with emoji buttons
• Change states (idle, speaking, thinking, listening, processing, error)
• See social-sized avatar preview
• Watch micro-movements when expressions change

React Usage

import { useAvatar } from "@shiit/avatars/hooks/useAvatar";

function MyAvatar() {
  const { avatarRef, setState, setExpression } = useAvatar(config, colors);

  return (
    <div>
      <canvas ref={avatarRef} width={400} height={400} />
      <button onClick={() => setState("speaking")}>Speak</button>
    </div>
  );
}

Available States

• idle - Gentle bobbing animation
• speaking - Mouth cycling animation
• thinking - Head tilt animation
• listening - Lean forward
• processing - Surprised expression
• error - Error state with custom geometry

Available Expressions

• normal - Default face
• happy - Smiling with blush
• sad - Downcast eyes
• angry - Furrowed brows
• surprised - Wide eyes
• dead - X X eyes
• confused - Raised eyebrow
• sleepy - Closed eyes
• nervous - Sweat drop
• shy - Sideways eyes
• wink - Random eye wink
• cute - Big shiny anime eyes
• pleading - Kawaii pout face

Creating Custom Avatars

Using the Factory (Recommended)

import { createAvatar } from "@shiit/avatars";

const myConfig = createAvatar({
  name: "my-avatar",
  states: {
    idle: {
      motion: { translate: { y: 2 } },
    },
  },
});

Extending the Base Class

import { Avatar } from "@shiit/avatars";

class MyAvatar extends Avatar {
  buildBody() {
    // Add custom body geometry
  }
}

Usage with Ultravox (Voice AI)

When using the avatar with real-time voice agents like Ultravox, you need to handle state (animations) and expression (face) as separate layers:

import { useEffect, useRef, useState } from "react";
import { UltravoxSession } from "ultravox-client";
import {
  MushroomAvatar,
  mushroomConfig,
  MUSHROOM_COLORS,
} from "@shiit/avatars/mushroom";

function VoiceAvatar() {
  const canvasRef = useRef<HTMLCanvasElement>(null);
  const avatarRef = useRef<MushroomAvatar | null>(null);
  const [status, setStatus] = useState("idle"); // idle | speaking | listening | thinking
  const [emailCaptured, setEmailCaptured] = useState(false);

  // Initialize avatar
  useEffect(() => {
    if (!canvasRef.current) return;
    avatarRef.current = new MushroomAvatar(
      canvasRef.current,
      { ...mushroomConfig, zoom: 2 },
      MUSHROOM_COLORS,
    );
  }, []);

  // CRITICAL: Always set state AND expression separately
  useEffect(() => {
    if (!avatarRef.current) return;

    // 1. Set state for animations (speaking mouth, idle bobbing, etc.)
    const state = status === "idle" ? "idle" : status;
    avatarRef.current.setState(state);

    // 2. Set expression for face (layered on top of state)
    if (emailCaptured) {
      avatarRef.current.setExpression("happy"); // Happy face while talking
    } else if (status === "idle") {
      avatarRef.current.setExpression("sleepy"); // Sleepy when idle
    } else {
      avatarRef.current.setExpression("normal"); // Normal face during call
    }
  }, [status, emailCaptured]);

  // Connect to Ultravox
  useEffect(() => {
    const session = new UltravoxSession();

    session.addEventListener("status", () => {
      setStatus(session.status); // speaking | listening | thinking | idle
    });

    session.registerToolImplementation("captureEmail", () => {
      setEmailCaptured(true);
      return "Email captured";
    });

    session.joinCall(joinUrl);
  }, []);

  return <canvas ref={canvasRef} width={80} height={80} />;
}

Key Pattern:

• setState() controls animations (mouth moving when speaking, head bobbing when idle)
• setExpression() controls face geometry (happy eyes, sleepy eyes, etc.)
• Always call both - they work as layers, not alternatives

Features

• Micro-movements: Each expression has a tiny animation when triggered
• Blush levels: 0-1 intensity (0.5 = medium, 1 = big)
• Custom expressions: Add your own with createAvatar
• Zod validation: Type-safe configs with validation
• Social size: Built-in zoom support for profile pictures

License

MIT