Anam AI JavaScript SDK

This is the official JavaScript SDK for integrating Anam AI realtime digital personas into your product. It provides a simple and intuitive API to interact with Anam AI's services.

Introduction

The Anam AI JavaScript SDK is designed to help developers integrate Anam AI's digital personas into their JavaScript applications. The SDK provides a set of APIs and utilities to make it easier to create, manage, and interact with digital personas in a realtime environment.

Documentation

Full documentation is available at docs.anam.ai.

Examples

Check out our example projects for implementation samples.

Prerequisites

An Anam AI account

To create a free account head to the Anam Lab and sign up.

An Anam API key

To use the SDK you first need an API key. Follow the instructions here to create one.

Getting Started

First, install the SDK in your project

npm install @anam-ai/js-sdk

Deprecation Notice

Important: The brainType field in PersonaConfig is deprecated and will be removed in a future version. Please use llmId instead. If you are currently using brainType, you will see a deprecation warning in the console. Both fields are supported during the transition period.

Local development

The quickest way to start testing the SDK is to use your API key directly with our SDK and the example persona config shown below. To use the SDK you first need to create an instance of AnamClient. For local development you can do this using the unsafe_createClientWithApiKey method.

import { unsafe_createClientWithApiKey } from '@anam-ai/js-sdk';

const anamClient = unsafe_createClientWithApiKey('your-api-key', {
  personaId: '<PERSONA ID HERE>',
  name: 'Cara',
  avatarId: '30fa96d0-26c4-4e55-94a0-517025942e18',
  voiceId: '6bfbe25a-979d-40f3-a92b-5394170af54b',
  brainType: 'ANAM_GPT_4O_MINI_V1',
  systemPrompt:
    "[STYLE] Reply in natural speech without formatting. Add pauses using '...' and very occasionally a disfluency. [PERSONALITY] You are Cara, a helpful assistant.",
});

NOTE: the method unsafe_createClientWithApiKey is unsafe for production use cases because it requires exposing your api key to the client. When deploying to production see production usage first.

Once you have an instance of the Anam client initialised you can start a session by streaming to audio and video elements in the DOM.

await anamClient.streamToVideoElement('video-element-id');

This will start a new session using the pre-configured persona id and start streaming video element in the DOM with the matching element id.

To stop a session use the stopStreaming method.

anamClient.stopStreaming();

Usage in production

When deploying to production it is important not to publicly expose your API key. To avoid this issue you should first exchange your API key for a short-lived session token on the server side. Session tokens can then be passed to the client and used to initialise the Anam SDK.

From the server

const response = await fetch(`https://api.anam.ai/v1/auth/session-token`, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify({
    personaConfig: {
      name: 'Cara',
      avatarId: '30fa96d0-26c4-4e55-94a0-517025942e18',
      voiceId: '6bfbe25a-979d-40f3-a92b-5394170af54b',
      llmId: '<LLM ID HERE>',
      systemPrompt:
        "[STYLE] Reply in natural speech without formatting. Add pauses using '...' and very occasionally a disfluency. [PERSONALITY] You are Cara, a helpful assistant.",
    },
  }),
});
const data = await response.json();
const sessionToken = data.sessionToken;

Once you have a session token you can use the createClient method of the Anam SDK to initialise an Anam client instance.

import { createClient } from '@anam-ai/js-sdk';

const anamClient = createClient('your-session-token');

Regardless of whether you initialise the client using an API key or session token the client exposes the same set of available methods for streaming.

See here for an example sequence diagram of starting a session in production environments.

Talk streams

createTalkMessageStream sends text straight to TTS for the persona to speak, one chunk at a time. All chunks in the stream share a correlation id, which is what an interruption correlates against.

const stream = anamClient.createTalkMessageStream();
for (const chunk of llmChunks) {
  await stream.streamMessageChunk(chunk.text, chunk.isLast);
}

You can start a talk stream as soon as streamToVideoElement() resolves. Chunks sent before the signalling connection first opens are queued and sent when it does.

Delivery failures

Talk-stream chunks travel over the session's signalling connection. On engines that answer heartbeats (see older engines below), the SDK checks that connection with heartbeats, and the answer to a heartbeat sent after a chunk confirms the chunk arrived. The SDK does not resend chunks. If a chunk can't be sent because the connection has dropped or stopped answering, or it isn't confirmed within about 3 seconds, the stream stops accepting chunks, stream.getState() returns TalkMessageStreamState.DELIVERY_FAILED, and the client emits MESSAGE_DELIVERY_FAILED with the stream's correlation id. The SDK reconnects on its own; deciding what to do with the answer is up to you. One option is to fall back to talk() with the full text:

import { AnamEvent } from '@anam-ai/js-sdk';

interface Answer {
  correlationId: string;
  text: string;
  complete: boolean;
  failed: boolean;
}

// Only the latest answer falls back: once a new one starts, the old one is stale.
let latest: Answer | null = null;

// Register once per client, not once per stream.
anamClient.addListener(AnamEvent.MESSAGE_DELIVERY_FAILED, (failure) => {
  const answer = latest;
  if (!answer || failure.correlationId !== answer.correlationId) return;
  answer.failed = true;
  // If the failure arrives after the answer is complete, speak it now.
  if (answer.complete) anamClient.talk(answer.text).catch(console.error);
});

async function speak(llmChunks: AsyncIterable<{ text: string; isLast: boolean }>) {
  const stream = anamClient.createTalkMessageStream();
  const answer: Answer = {
    correlationId: stream.getCorrelationId(),
    text: '',
    complete: false,
    failed: false,
  };
  latest = answer;
  for await (const chunk of llmChunks) {
    answer.text += chunk.text;
    // A failed or interrupted stream rejects further chunks, so keep collecting the
    // answer instead. This also covers a stale answer, whose failure the listener ignores.
    if (stream.isActive()) await stream.streamMessageChunk(chunk.text, chunk.isLast);
  }
  answer.complete = true;
  if (answer.failed && latest === answer) await anamClient.talk(answer.text);
}

A failure means the engine did not confirm the chunks, not that it never received them. Some of the answer may already have been spoken, and talk() repeats it. Once the client has received TALK_STREAM_INTERRUPTED for a stream, it does not emit MESSAGE_DELIVERY_FAILED for it. An interruption sent while the connection was down can be missed, though, so the fallback can still repeat an answer the user cut off. The reverse can also happen: the engine can deliver that interruption after the client reconnects, so TALK_STREAM_INTERRUPTED can arrive for a stream that has already failed, after your fallback has started.

Ending the session is not a delivery failure. After stopStreaming(), or when the client emits CONNECTION_CLOSED because the server ended the session or the media connection failed, unconfirmed chunks are lost without MESSAGE_DELIVERY_FAILED, and chunks sent afterwards are dropped. Treat CONNECTION_CLOSED as the end of every open stream. The one exception is the signalling connection closing for good on its own: unconfirmed chunks then fail before CONNECTION_CLOSED.

Only talk streams are confirmed. Audio sent through createAgentAudioInputStream() is not, so a connection that stops working without closing can lose it without any event.

Older engines

Confirming chunks needs an engine that answers heartbeats. Engines that do say so when the session becomes ready, with sendsHeartbeatAcks: true in the sessionready message. On an older engine, the SDK stops tracking chunks once the session is ready and sends them as before, queuing them while the connection reconnects. It still emits MESSAGE_DELIVERY_FAILED in three cases:

• A chunk is sent after the connection has closed for good.
• A chunk is queued before the connection first opens, and it never opens.
• A chunk is sent before the session is ready, and the session takes more than about 3 seconds to become ready. Until then the SDK can't tell an older engine from one that has stopped answering, so it fails the stream and reconnects. Waiting for AnamEvent.SESSION_READY before the first chunk avoids this.

Speech across a tool call

A talk stream can stay open while your application runs a tool. Give the speech before and after the tool call separate utterance ids, and the second waits for the first to finish playing.

[!NOTE] Utterance ids need a Cara 4 avatar. Cara 3 avatars run an older engine that drops the field silently: the text is still spoken, but as one utterance, and persona message events carry no utteranceId. Nothing errors or warns, so check whether ids come back rather than waiting for a failure.

[!IMPORTANT] utteranceId marks the start of an utterance, not an individual text chunk. Set it on the first chunk, then omit it from the remaining chunks in that utterance. Set a new id only when the next utterance begins.

const stream = anamClient.createTalkMessageStream();

// Utterance A can begin playing while the tool runs.
await stream.streamMessageChunk('Let me check ', false, crypto.randomUUID());
await stream.streamMessageChunk('that for you.', false); // Continues utterance A.

const toolResultText = await runToolCall();

// Utterance B waits for A to finish, then continues with the result.
await stream.streamMessageChunk(toolResultText, true, crypto.randomUUID());

The tool has to finish quickly. The server closes a talk stream that goes 15 seconds without receiving a chunk that carries text, ends the turn, and rejects anything sent on that correlation id afterwards. Empty chunks do not reset that timer. For a tool that may run longer, let the first utterance end the stream and start a new one when the result arrives.

The same ordering applies when both utterances are ready immediately. Send them with different ids and the second waits for the first to finish playing:

const stream = anamClient.createTalkMessageStream();
await stream.streamMessageChunk('First utterance.', false, crypto.randomUUID());
await stream.streamMessageChunk('Second utterance.', true, crypto.randomUUID());

Ids must be lowercase canonical UUID v4 strings, which is what crypto.randomUUID() returns; anything else throws. They come back on persona message events as utteranceId, so you can match your own ids against what the persona spoke.

Director Notes (Cara 4)

On Cara 4 avatars you can add Director Notes to guide the avatar's performance. Provide either a built-in presetStyle or a free-form customStylePrompt — the two are mutually exclusive (enforced by the DirectorNotes type) — plus an optional expressivity value, normalized from 0 to 1, controlling how expressively the style is played (lower values are steadier; higher values increase style and speech-driven motion together; omit it to use the engine default). Director Notes are forwarded unchanged to session-token creation and are only applied on Cara 4 avatars; on older models the server ignores them and the session proceeds without them.

import { unsafe_createClientWithApiKey } from '@anam-ai/js-sdk';

const anamClient = unsafe_createClientWithApiKey('your-api-key', {
  personaId: '<PERSONA ID HERE>',
  name: 'Cara',
  // Use a Cara 4 avatar — Director Notes are ignored on older models.
  avatarId: '30fa96d0-26c4-4e55-94a0-517025942e18',
  voiceId: '6bfbe25a-979d-40f3-a92b-5394170af54b',
  llmId: '<LLM ID HERE>',
  systemPrompt:
    '[STYLE] Reply in natural speech without formatting. [PERSONALITY] You are Cara, a helpful assistant.',
  directorNotes: {
    presetStyle: 'warm',
    expressivity: 0.5,
  },
});

To use a free-form style instead of a preset, provide customStylePrompt:

directorNotes: {
  customStylePrompt:
    'Warm smile, composed, slightly amused, looking directly at camera',
  expressivity: 0.5,
}

You can also set expressivity on its own to tune how expressively the avatar's default style is played:

directorNotes: {
  expressivity: 0.2,
}

Sending Director Note cues mid-session

On Cara 4 you can use sendDirectorNoteCue to change the performance as a conversation unfolds. Cues use the engine's dedicated cue path, so they do not purge buffered audio or video. Data-channel cues are primarily for audio-passthrough sessions; for Turnkey sessions, prefer inline cue tags in persona speech text.

import { AnamEvent, type DirectorNoteCueTag } from '@anam-ai/js-sdk';

let cueChannelOpen = false;
anamClient.addListener(AnamEvent.DATA_CHANNEL_OPEN, () => {
  cueChannelOpen = true;
});
anamClient.addListener(AnamEvent.CONNECTION_CLOSED, () => {
  cueChannelOpen = false;
});

// Call this later from your TTS timing callback for an active persona turn.
function onTtsCue(tag: DirectorNoteCueTag, atSeconds: number) {
  if (!cueChannelOpen) {
    throw new Error('Director Note cue channel is not open');
  }

  anamClient.sendDirectorNoteCue(tag, { atSeconds });
}

// Wire onTtsCue to your TTS provider's cue/word-timing callback. For example,
// report `warm` at 0 seconds and `surprised` at 1.25 seconds for that turn.

Register the listener before starting the stream so it cannot miss the event. Sending before DATA_CHANNEL_OPEN throws instead of silently dropping the cue. Omitting timing applies the cue immediately. inSeconds is a delay from now during an active response, while atSeconds is an absolute offset from the start of persona speech and is preferred when aligning cues to generated audio; provide at most one. Runtime cue tags also include cue-only styles such as laughter, curious, concerned, and surprised that are not available as session-start presetStyle values.