@stellar-ai/agent-sdk

Stellar Agent SDK for real-time voice conversations with AI agents.

Installation

npm install @stellar-ai/agent-sdk
# or
yarn add @stellar-ai/agent-sdk
# or
pnpm install @stellar-ai/agent-sdk

Usage

import { createStellarClient } from "@stellar-ai/agent-sdk";

const client = createStellarClient();

const conversation = await client.startConversation({
  auth: {
    strategy: "publicToken",
    token: "<your-public-access-token>",
  },
  agentId: "<your-agent-id>",
  variables: {
    userName: "Alice",
    orderId: "12345",
  },
});

conversation
  .on("stateChanged", ({ state }) => {
    console.log("Conversation state:", state);
  })
  .on("error", ({ error, code }) => {
    console.error("Conversation error:", code, error);
  })
  .on("started", ({ clientId }) => {
    console.log("Conversation started with client ID:", clientId);
  });

// Mute/unmute controls
conversation.mute();
conversation.unmute();

// End the conversation
await conversation.end();

Environments

The SDK provides utilities for connecting to different environments:

import {
  createStellarClient,
  Local, // http://localhost:4001
  Staging, // https://agent-staging.stellarcs.ai
  Production, // https://agent.stellarcs.ai (default)
  Environment, // Encore subdomain env (e.g. preview)
  PreviewEnv, // Preview environment by PR number
} from "@stellar-ai/agent-sdk";

// Local development
const localClient = createStellarClient({ baseUrl: Local });

// Staging
const stagingClient = createStellarClient({ baseUrl: Staging });

// Preview environment (PR #123)
const previewClient = createStellarClient({ baseUrl: PreviewEnv(123) });

// Custom environment
const customClient = createStellarClient({ baseUrl: Environment("my-env") });

API

createStellarClient(options?)

Creates a new Stellar client.

Options:

• baseUrl - Optional base URL (defaults to Production)
• logger - Optional logger for debugging
• audioCapture - Optional custom audio capture implementation (see Custom audio)
• audioPlayback - Optional custom audio playback implementation (see Custom audio)

client.startConversation(options)

Starts a new conversation with the agent. Only one conversation can be active at a time; end the current one with conversation.end() before starting another, or a StellarError with code CONVERSATION_ALREADY_ACTIVE is thrown.

Options:

• auth - Authentication configuration ({ strategy: 'publicToken', token: '...' })
• agentId - The agent ID to connect to
• variables - Optional key-value pairs for initial context

Conversation Methods

• conversation.end() - End the conversation
• conversation.mute() - Mute the microphone
• conversation.unmute() - Unmute the microphone
• conversation.isMuted - Check mute state
• conversation.state - Current state (connecting, connected, disconnected, error)

Events

• stateChanged - Connection state changed
• error - An error occurred
• started - Conversation started (includes clientId)
• handover - Agent initiated a handover (includes handoverType, optional phoneNumber, queueId, announcement)
• actionExecuted - Agent executed a tool/action (includes actionName, optional requestBody, responseBody)

Error Codes

• UNAUTHENTICATED - Authentication failed
• TRANSPORT_ERROR - Network/connection issues
• MIC_ACCESS_DENIED - Microphone access denied
• MISSING_AGENT_ID - Agent ID not provided
• CONVERSATION_ALREADY_ACTIVE - A conversation is already active; end it before starting another
• INTERNAL_ERROR - Unexpected SDK error

React Native / Expo

The SDK ships with ready-made audio implementations for Expo via the @stellar-ai/agent-sdk/expo subpath. These wrap @mykin-ai/expo-audio-stream and handle sample-rate conversion automatically.

npx expo install @mykin-ai/expo-audio-stream

import { createStellarClient } from "@stellar-ai/agent-sdk";
import { ExpoAudioCapture, ExpoAudioPlayback } from "@stellar-ai/agent-sdk/expo";

const client = createStellarClient({
  audioCapture: new ExpoAudioCapture(),
  audioPlayback: new ExpoAudioPlayback(),
});

That's it — microphone capture and audio playback are fully handled.

Custom audio

The SDK uses the Web Audio API by default (browsers). For other environments, you can provide custom implementations of IAudioCapture and IAudioPlayback:

import { createStellarClient } from "@stellar-ai/agent-sdk";
import type { IAudioCapture, IAudioPlayback } from "@stellar-ai/agent-sdk";

const client = createStellarClient({
  audioCapture: new MyCustomAudioCapture(),
  audioPlayback: new MyCustomAudioPlayback(),
});

IAudioCapture

| Method / Property | Description | | --- | --- | | start(onAudioData: (data: Int16Array) => void) | Start capturing. Call the callback with PCM16 24kHz mono chunks. | | stop() | Stop capturing and release resources. | | mute() | Mute the microphone. | | unmute() | Unmute the microphone. | | muted (getter) | Whether the microphone is currently muted. |

IAudioPlayback

| Method | Description | | --- | --- | | start() | Initialize the audio output. | | stop() | Stop playback and release resources. | | play(pcm16Data: Int16Array) | Queue a PCM16 24kHz mono chunk for playback. | | interrupt() | Stop all current playback immediately (used for barge-in). |

Requirements

• Browser: Modern browser with WebSocket and Web Audio support, HTTPS in production (except localhost)
• React Native / Expo: Install @mykin-ai/expo-audio-stream and use the built-in Expo implementations (see above)