Hedra plugin for LiveKit Agents

Support for avatar generation and animation with Hedra.

See https://docs.livekit.io/agents/integrations/avatar/hedra/ for more information.

Installation

npm install @livekit/agents-plugin-hedra
# or
pnpm add @livekit/agents-plugin-hedra
# or
yarn add @livekit/agents-plugin-hedra

Pre-requisites

You'll need an API key from Hedra. It can be set as an environment variable: HEDRA_API_KEY

Usage

Using an Avatar ID

import { AvatarSession } from '@livekit/agents-plugin-hedra';

// Create an avatar session with an avatar ID
const avatar = new AvatarSession({
  avatarId: 'your-avatar-id',
  apiKey: 'your-hedra-api-key', // or set HEDRA_API_KEY env var
});

// Start the avatar session after creating your agent session
await avatar.start(agentSession, room);

Using a Custom Avatar Image

import { AvatarSession } from '@livekit/agents-plugin-hedra';
import fs from 'node:fs';

// Read an image file
const imageBuffer = fs.readFileSync('path/to/avatar.jpg');

// Create an avatar session with a custom image
const avatar = new AvatarSession({
  avatarImage: {
    data: imageBuffer,
    mimeType: 'image/jpeg',
    filename: 'avatar.jpg',
  },
  apiKey: 'your-hedra-api-key',
});

// Start the avatar session
await avatar.start(agentSession, room);

Full Example

import {
  type JobContext,
  type JobProcess,
  WorkerOptions,
  cli,
  defineAgent,
  voice,
} from '@livekit/agents';
import * as deepgram from '@livekit/agents-plugin-deepgram';
import * as hedra from '@livekit/agents-plugin-hedra';
import * as openai from '@livekit/agents-plugin-openai';
import * as silero from '@livekit/agents-plugin-silero';

export default defineAgent({
  prewarm: async (proc: JobProcess) => {
    proc.userData.vad = await silero.VAD.load();
  },
  entry: async (ctx: JobContext) => {
    const vad = ctx.proc.userData.vad as silero.VAD;

    const assistant = new voice.Agent({
      instructions: 'You are a helpful voice AI assistant.',
    });

    const session = new voice.AgentSession({
      vad,
      stt: new deepgram.STT(),
      llm: new openai.LLM(),
      tts: new openai.TTS(),
    });

    // Create and start the Hedra avatar
    const avatar = new hedra.AvatarSession({
      avatarId: 'your-avatar-id',
    });

    await session.start({
      agent: assistant,
      room: ctx.room,
    });

    // Start the avatar session after connecting
    await avatar.start(session, ctx.room);

    session.generateReply({
      instructions: 'Greet the user and offer your assistance.',
    });
  },
});

Configuration Options

AvatarSessionOptions

| Option | Type | Description | |--------|------|-------------| | avatarId | string | The Hedra avatar ID to use. Either avatarId or avatarImage must be provided. | | avatarImage | AvatarImage | A custom avatar image. Either avatarId or avatarImage must be provided. | | apiUrl | string | The Hedra API URL. Defaults to HEDRA_API_URL env var or the default Hedra API endpoint. | | apiKey | string | The Hedra API key. Defaults to HEDRA_API_KEY env var. | | avatarParticipantIdentity | string | The identity of the avatar participant in the room. Defaults to 'hedra-avatar-agent'. | | avatarParticipantName | string | The name of the avatar participant in the room. Defaults to 'hedra-avatar-agent'. | | connOptions | APIConnectOptions | Connection options for API requests (retry count, timeout, etc.). |

AvatarImage

| Property | Type | Description | |----------|------|-------------| | data | Buffer | The raw image data. | | mimeType | string | The MIME type of the image (e.g., 'image/jpeg', 'image/png'). | | filename | string | Optional filename for the image. |

Environment Variables

| Variable | Description | |----------|-------------| | HEDRA_API_KEY | Your Hedra API key | | HEDRA_API_URL | Custom Hedra API URL (optional) | | LIVEKIT_URL | Your LiveKit server URL | | LIVEKIT_API_KEY | Your LiveKit API key | | LIVEKIT_API_SECRET | Your LiveKit API secret |