@equos/browser

Browser SDK for building real-time conversational experiences with Equos characters. Provides audio and video pipelines that connect to the Equos platform via WebSocket.

Installation

npm install @equos/browser

Peer dependencies

The following packages must be installed by the consumer:

npm install onnxruntime-web @equos/audio-processor

Quick start

import {
  EquosPipelineFactory,
  EquosPipelineEvents,
  EquosUserAudioRecorder,
} from '@equos/browser';
import type { EquosPipelineConfig } from '@equos/browser';

// 1. Create the pipeline
const pipeline = EquosPipelineFactory.new({
  accessToken: '<your-access-token>',
  audio: { enabled: true },
  video: {
    enabled: true,
    width: 450,
    height: 800,
    containerSelector: '#container',
  },
  debug: true,
});

// 2. Listen for events
pipeline.on(EquosPipelineEvents.UTTERANCE, async (utterance) => {
  console.log(`${utterance.author}: ${utterance.content}`);
});

pipeline.on(EquosPipelineEvents.ERROR, async (error) => {
  console.error(`[${error.code}]: ${error.message}`);
});

// 3. Set up the microphone recorder
const recorder = new EquosUserAudioRecorder((chunk) =>
  pipeline.audio().pushUsrAudio(chunk),
);

// 4. Initialize both
await Promise.all([pipeline.init(), recorder.init()]);

// 5. Start a conversation
await pipeline.start({});
await recorder.start();

// 6. Later — stop and clean up
await recorder.stop();
await pipeline.stop();

await recorder.destroy();
await pipeline.destroy();

API reference

EquosPipelineFactory

Static factory for creating pipeline instances.

const pipeline = EquosPipelineFactory.new(config);

EquosPipelineConfig

Configuration passed to EquosPipelineFactory.new().

| Field | Type | Required | Description | |---|---|---|---| | accessToken | string | Yes | JWT token obtained from the Equos API. | | audio | { enabled: boolean } | Yes | Whether audio output is enabled at creation. | | video | object | No | Video configuration (see below). | | name | string | No | Pipeline instance name (useful for debugging). | | endpoint | string | No | WebSocket endpoint URL. Defaults to the current host. | | debug | boolean | No | Enable debug logging to the console. |

video object

| Field | Type | Description | |---|---|---| | enabled | boolean | Whether video rendering is enabled at creation. | | width | number | Video canvas width in pixels. | | height | number | Video canvas height in pixels. | | containerSelector | string | CSS selector for the DOM element that will host the video. |

EquosPipeline

Manages the full lifecycle of a real-time conversation session.

Lifecycle methods

| Method | Description | |---|---| | init(): Promise<void> | Initialize internal resources. Must be called before start(). | | use(character: string): Promise<void> | Select a character. Only needed when the access token is not scoped to a specific character. | | start(conversation): Promise<void> | Start a conversation session. | | stop(): Promise<void> | Stop the active conversation. The pipeline returns to a ready state and can be started again. | | destroy(): Promise<void> | Tear down all resources. The pipeline cannot be used after this. | | getState(): EquosPipelineState | Returns the current pipeline state. |

conversation parameter

The object passed to start() accepts the following optional fields:

| Field | Type | Description | |---|---|---| | prompt_ctx | string | Additional context injected into the character prompt. | | prompt_vars | Record<string, string> | Key-value pairs substituted into prompt template variables. | | max_seconds | number | Maximum duration of the conversation in seconds. |

Audio & video facades

// Audio
pipeline.audio().enable();
pipeline.audio().disable();
pipeline.audio().pushUsrAudio(chunk); // Push a Float32Array audio chunk

// Video
pipeline.video().enable();
pipeline.video().disable();

Events

Register and unregister listeners with on() and off():

pipeline.on(EquosPipelineEvents.UTTERANCE, callback);
pipeline.off(EquosPipelineEvents.UTTERANCE, callback);

| Event | Callback signature | Description | |---|---|---| | UTTERANCE | (utterance: EquosUtterance) => Promise<void> | Fired when the agent or user produces an utterance. | | INTERRUPT | () => Promise<void> | Fired when the user interrupts the agent. | | ERROR | (error: EquosPipelineError) => Promise<void> | Fired on pipeline errors. |

EquosUtterance

| Field | Type | Description | |---|---|---| | content | string | The utterance text. | | author | 'agent' \| 'user' | Who produced the utterance. | | recordedAt | number | Timestamp of the utterance. |

EquosPipelineState

Enum representing the pipeline lifecycle.

CREATED → INITIALIZED → READY → STARTED → (back to READY on stop)
                                          ↘ DESTROYED

| Value | Description | |---|---| | CREATED | Pipeline has been constructed but not initialized. | | INITIALIZED | init() has completed. | | READY | A character has been selected (via token or use()). Ready to start. | | STARTED | A conversation is active. | | DESTROYED | destroy() has been called. Terminal state. |

EquosUserAudioRecorder

Captures microphone audio and forwards PCM chunks to the pipeline.

const recorder = new EquosUserAudioRecorder((chunk: Float32Array) => {
  pipeline.audio().pushUsrAudio(chunk);
});

await recorder.init();    // Set up the AudioWorklet
await recorder.start();   // Request mic permission and begin capture
await recorder.stop();    // Stop capture and release the mic stream
await recorder.destroy(); // Tear down the AudioContext

EquosPipelineError

Error class with a code property corresponding to EquosPipelineErrors.

pipeline.on(EquosPipelineEvents.ERROR, async (err) => {
  console.error(err.code, err.message);
});

EquosPipelineErrors

Enum of error codes.

| Code | Meaning | |---|---| | MUST_INIT | init() must be called first. | | MUST_USE | use() must be called first (token has no character scope). | | NOT_IDLE | Operation requires the pipeline to be in INITIALIZED or READY state. | | STARTED | Operation cannot be performed while a conversation is active. | | DESTROYED | Operation cannot be performed after destroy(). | | INVALID_TOKEN | The access token could not be decoded. | | CANNOT_USE_CHARACTER | The token is scoped to a specific character; use() cannot override it. | | NO_CONTAINER | The video container element was not found in the DOM. |

Full example

import {
  EquosPipelineFactory,
  EquosPipeline,
  EquosPipelineEvents,
  EquosUserAudioRecorder,
} from '@equos/browser';

let pipeline: EquosPipeline | null = null;
let recorder: EquosUserAudioRecorder | null = null;

// ── Create & initialize ──────────────────────────────────

async function initialize(accessToken: string) {
  pipeline = EquosPipelineFactory.new({
    name: 'example',
    accessToken,
    audio: { enabled: true },
    video: {
      enabled: true,
      width: 450,
      height: 800,
      containerSelector: '#container',
    },
    debug: true,
  });

  pipeline.on(EquosPipelineEvents.UTTERANCE, async (u) => {
    console.log(`Utterance (${u.author}): ${u.content}`);
  });

  pipeline.on(EquosPipelineEvents.INTERRUPT, async () => {
    console.log('Interrupt');
  });

  pipeline.on(EquosPipelineEvents.ERROR, async (err) => {
    console.error(`Error [${err.code}]: ${err.message}`);
  });

  recorder = new EquosUserAudioRecorder((chunk) =>
    pipeline?.audio().pushUsrAudio(chunk),
  );

  await Promise.all([pipeline.init(), recorder.init()]);
}

// ── Start a conversation ─────────────────────────────────

async function start() {
  if (!pipeline || !recorder) return;

  await pipeline.start({
    prompt_ctx: 'The user is a first-time visitor.',
    prompt_vars: { name: 'Alice' },
    max_seconds: 300,
  });

  await recorder.start();
}

// ── Stop the conversation ────────────────────────────────

async function stop() {
  if (!pipeline || !recorder) return;

  await recorder.stop();
  await pipeline.stop();
}

// ── Toggle audio/video at runtime ────────────────────────

async function toggleAudio(enabled: boolean) {
  if (!pipeline) return;
  if (enabled) {
    await pipeline.audio().enable();
  } else {
    await pipeline.audio().disable();
  }
}

async function toggleVideo(enabled: boolean) {
  if (!pipeline) return;
  if (enabled) {
    await pipeline.video().enable();
  } else {
    await pipeline.video().disable();
  }
}

// ── Tear down ────────────────────────────────────────────

async function destroy() {
  await recorder?.stop();
  await recorder?.destroy();
  await pipeline?.destroy();
  pipeline = null;
  recorder = null;
}