@keyframelabs/elements

Framework-agnostic primitives for Keyframe Labs.

Which package should I use?

• @keyframelabs/sdk (high control)
  • You implement the UI, state management, and agent/llm binding yourself
• @keyframelabs/elements (custom UI)
  • You implement the UI; we handle the state and agent/llm binding (framework-agnostic)
• @keyframelabs/react: (drop-in)
  • We handle the UI, state, and agent/llm binding

Install

pnpm add @keyframelabs/elements

Usage

This package provides three integration options depending on your strategy:

1. PersonaEmbed: Fully managed. Uses a publishable key. Best for rapid frontend integration.
2. PersonaView: Bring your own backend. Uses session tokens generated by your server.
3. <kfl-embed>: Drop-in web component. Wraps PersonaEmbed with a complete widget UI (states, controls, positioning). Zero framework dependencies.

Option A: PersonaEmbed (managed)

Use this if you have configured an embed in the Keyframe platform dashboard.

import { PersonaEmbed } from '@keyframelabs/elements';

const embed = new PersonaEmbed({
  container: document.getElementById('persona-container'),
  publishableKey: 'kfl_pk_live_...',
  videoFit: 'cover',
  onStateChange: (status) => console.log('Status:', status),
  onAgentStateChange: (state) => console.log('Agent:', state),
  onDisconnect: () => console.log('Disconnected'),
  onError: (err) => console.error(err),
});

await embed.connect();

// Later...
embed.disconnect();

Option B: PersonaView (bring your own backend)

Use this when you want to manage authentication through your own backend.

import { PersonaView } from '@keyframelabs/elements';

const view = new PersonaView({
  container: document.getElementById('persona-container')!,
  sessionDetails,
  voiceAgentDetails,
  videoFit: 'contain',
  onStateChange: (status) => console.log('Status:', status),
  onAgentStateChange: (state) => console.log('Agent:', state),
  onDisconnect: () => console.log('Disconnected'),
  onError: (err) => console.error(err),
});

await view.connect();

// Later...
view.disconnect();

Option C: <kfl-embed> Web Component

A self-registering custom element that wraps PersonaEmbed with a widget UI shell: minimized/active/hidden states, "Join call" button, minimize/expand toggle, corner positioning, and button color styling. Drop it into any page with zero framework dependencies.

<kfl-embed
  publishable-key="kfl_pk_live_..."
  initial-state="minimized"
  corner="bottom-right"
  minimized-width="144"
  minimized-height="216"
  active-width="252"
  active-height="377"
  button-color="#919191"
  button-color-opacity="0.3"
  video-fit="cover"
  preview-image="https://example.com/avatar.png"></kfl-embed>
<script type="module" src="https://unpkg.com/@keyframelabs/elements/dist/kfl-embed.js"></script>

Attributes

| Attribute | Type | Default | Description | | -------------------------------- | ---------------------------------------------------------- | ---------------- | ------------------------------------------------------------------ | | publishable-key | string | Required | Your publishable embed key. | | api-base-url | string | Keyframe default | Base URL for the Keyframe API. | | initial-state | 'minimized' \| 'active' \| 'hidden' | 'minimized' | Widget state on first load. | | controlled-widget-state | 'minimized' \| 'active' \| 'hidden' | — | Externally control the widget state (overrides internal state). | | corner | 'bottom-right' \| 'bottom-left' \| 'top-right' \| 'top-left' | 'bottom-right' | Which corner to anchor the widget (fixed mode only). | | inline | boolean attribute | — | Use position: relative instead of fixed. | | minimized-width | number | 144 | Width in px when minimized. | | minimized-height | number | 216 | Height in px when minimized. | | active-width | number | 252 | Width in px when active (in-call). | | active-height | number | 377 | Height in px when active (in-call). | | hide-ui | boolean attribute | — | Hides all overlay controls. Useful for building your own UI shell. | | show-minimize-button | 'true' \| 'false' | 'true' | Show the X/minimize button on hover. | | controlled-show-minimize-button| 'true' \| 'false' | — | Externally control the minimize button visibility. | | button-color | hex color string | '#919191' | Color of the "Join call" button. | | button-color-opacity | number (0–1) | 0.3 | Opacity of the "Join call" button background. | | video-fit | 'cover' \| 'contain' | 'cover' | Video scaling mode. | | preview-image | URL string | — | Image shown in the widget before a call starts. |

Events

| Event | detail | Description | | ------------------ | --------------------------------------- | -------------------------------------- | | statechange | { status: EmbedStatus } | Connection status changed. | | agentstatechange | { state: AgentState } | Avatar playback state changed. | | widgetstatechange| { state: 'minimized' \| 'active' \| 'hidden' } | Widget UI state changed. | | disconnected | — | Session disconnected. | | error | { error: Error } | A fatal error occurred. |

JavaScript API

const el = document.querySelector('kfl-embed');

await el.micOn();     // Start session if needed, then unmute mic
await el.micOff();    // Mute mic
el.isMicOn();         // boolean

await el.mute();      // Mute speaker audio
await el.unmute();    // Unmute speaker audio
el.isMuted();         // boolean

Build

The web component is built as a self-contained ES module via vite.config.embed.ts:

pnpm build:embed   # → dist/kfl-embed.js

Import the auto-registering entry point (registers <kfl-embed> on customElements):

import '@keyframelabs/elements/kfl-embed';

Or import the class directly for manual registration:

import { KflEmbedElement } from '@keyframelabs/elements';
customElements.define('my-embed', KflEmbedElement);

Supported agents and real-time LLMs

Supports ElevenLabs and OpenAI Realtime.

For PersonaEmbed, this is determined by the values you set in the Keyframe platform dashboard.

For PersonaView, this is determined by voiceAgentDetails.

Emotion Controls

The avatar can display emotional expressions (neutral, angry, sad, happy) that affect its facial expression and demeanor. All supported voice agents can drive emotions automatically via tool/function calling.

Agent Events

The emotion event is emitted when any agent triggers a set_emotion tool call:

agent.on('emotion', (emotion) => {
  console.log('Emotion changed:', emotion); // 'neutral' | 'angry' | 'sad' | 'happy'
});

When using PersonaEmbed or PersonaView, emotion events are automatically wired to the avatar session -- no extra code is needed.

ElevenLabs

Emotions are driven by a client tool call named set_emotion. The agent parses incoming client_tool_call WebSocket messages and sends a client_tool_result back.

Setup: Create a set_emotion client tool in the ElevenLabs dashboard for your agent with a single emotion parameter (enum: neutral, angry, sad, happy). Then instruct your agent (via its system prompt) to call set_emotion on each turn.

OpenAI Realtime

The set_emotion function is automatically declared in the OpenAI Realtime session setup. The model calls it via Realtime function calling (response.done with function_call output items), and the client responds by creating a function_call_output conversation item before asking the model to continue.

Setup: No additional dashboard configuration is needed. Instruct the model via its system prompt to call set_emotion on each turn to reflect the tone of its response.

Manual Emotion Control

For custom emotion logic outside of tool calling, you can access the underlying session directly:

import { createClient } from '@keyframelabs/sdk';

const session = createClient({ ... });
await session.setEmotion('happy');

API

PersonaEmbed

Options

| Option | Type | Default | Description | | -------------------- | ------------------------------- | -------------------------------- | ---------------------------------------------- | | container | HTMLElement | Required | Target container for video and audio elements. | | publishableKey | string | Required | Your publishable embed key. | | apiBaseUrl | string | 'https://api.keyframelabs.com' | Base URL for the Keyframe API. | | videoFit | 'cover' \| 'contain' | 'cover' | Video scaling mode (object-fit). | | onStateChange | (status: EmbedStatus) => void | — | Fired when connection status changes. | | onAgentStateChange | (state: AgentState) => void | — | Fired when avatar playback state changes. Signaled by the GPU node via RPC, not the voice agent. | | onDisconnect | () => void | — | Fired when the session disconnects. | | onError | (err: Error) => void | — | Fired on fatal errors. |

Methods

| Method | Signature | Description | | ------------ | --------------------- | -------------------------------------------------- | | connect | () => Promise<void> | Starts the session and establishes the connection. | | disconnect | () => void | Disconnects the session and cleans up resources. | | toggleMute | () => void | Toggles the user's microphone mute state. |

Properties

| Property | Type | Description | | -------------- | ------------------ | -------------------------------------------------------------------------------------- | | status | EmbedStatus | Current connection status: 'connecting' \| 'connected' \| 'disconnected' \| 'error'. | | agentState | AgentState | Avatar playback state: 'listening' \| 'speaking'. Set by the GPU node, not the voice agent. | | isMuted | boolean | Whether the microphone is currently muted. | | videoElement | HTMLVideoElement | The underlying video element used for rendering. | | audioElement | HTMLAudioElement | The underlying audio element used for playback. |

PersonaView

Options

| Option | Type | Default | Description | | -------------------- | ------------------------------- | --------- | ---------------------------------------------- | | container | HTMLElement | Required | Target container for video and audio elements. | | sessionDetails | SessionDetails | Required | Session configuration from your backend. | | voiceAgentDetails | VoiceAgentDetails | Required | Voice agent configuration from your backend. | | videoFit | 'cover' \| 'contain' | 'cover' | Video scaling mode (object-fit). | | onStateChange | (status: EmbedStatus) => void | — | Fired when connection status changes. | | onAgentStateChange | (state: AgentState) => void | — | Fired when agent state changes. | | onDisconnect | () => void | — | Fired when the session disconnects. | | onError | (err: Error) => void | — | Fired on fatal errors. |

Methods

Same as PersonaEmbed.

Properties

Same as PersonaEmbed.

Types

type SessionDetails = {
  server_url: string;
  participant_token: string;
  agent_identity: string;
};

type VoiceAgentDetails = {
  type: 'elevenlabs' | 'openai';
  token?: string;        // For openai (ephemeral client secret)
  agent_id?: string;     // For elevenlabs
  signed_url?: string;   // For elevenlabs
  system_prompt?: string; // For openai
  voice?: string;         // For openai
};

type Emotion = 'neutral' | 'angry' | 'sad' | 'happy';

License

MIT