AStack Client SDK

Browser SDK for AStack real-time conversational AI — voice input, AI voice + avatar animation output over WebSocket.

Installation

npm install @aether-stack-dev/client-sdk

Quick Start

import { AStackCSRClient } from '@aether-stack-dev/client-sdk';

const client = new AStackCSRClient({
  workerUrl: 'wss://your-worker-url',
  sessionToken: sessionToken, // from your backend via developer-sdk
});

await client.connect();

client.on('transcript', (text) => console.log('User said:', text));
client.on('response', (text) => console.log('AI:', text));
client.on('blendshapeUpdate', (bs) => {
  // 52 ARKit blendshape values for avatar animation
});

await client.startCall();

// When done
client.stopCall();
client.disconnect();

Constructor

new AStackCSRClient(config: AStackCSRConfig)

| Option | Type | Default | Description | |--------|------|---------|-------------| | workerUrl | string | required | WebSocket URL of the worker container | | sessionToken | string | '' | Auth token from developer SDK (required) | | sampleRate | number | 24000 | Audio playback sample rate | | fps | number | 30 | Blendshape animation frame rate | | enableImageCapture | boolean | true | Capture webcam frames for vision | | imageCaptureInterval | number | 5000 | Image capture interval (ms) | | autoReconnect | boolean | true | Auto-reconnect on disconnect | | maxRetries | number | 5 | Max reconnection attempts | | reconnectDelay | number | 1000 | Base delay between reconnects (ms) | | audioProcessorUrl | string | '/audio-processor.js' | Path to AudioWorklet processor | | debug | boolean | false | Enable debug logging |

Methods

| Method | Description | |--------|-------------| | connect(): Promise<void> | Connect to the worker via WebSocket and authenticate | | startCall(): Promise<void> | Start microphone capture and begin the conversation | | stopCall(): void | Stop the call and release media resources | | sendText(message: string): void | Send a text message during a call | | disconnect(): void | Close WebSocket connection | | destroy(): Promise<void> | Disconnect, clean up audio player, remove all listeners | | isConnected(): boolean | Check if WebSocket is open | | getCallStatus(): CallStatus | Get current call status (idle | starting | active | stopping | error) | | getCurrentBlendshapes(): number[] | Get the latest 52 blendshape values |

Events

| Event | Payload | Description | |-------|---------|-------------| | connected | — | WebSocket connected and authenticated | | disconnected | — | WebSocket closed | | reconnecting | attempt: number | Attempting reconnection | | reconnected | — | Successfully reconnected | | error | Error | Connection or call error | | callStarted | — | Call is active, mic streaming | | callStopped | — | Call ended | | interim | text: string | Interim ASR transcript | | transcript | text: string | Final ASR transcript | | response | text: string | AI response text (streamed) | | responseComplete | — | AI response fully delivered | | speechStarted | — | User started speaking | | utteranceEnd | — | End of user utterance detected | | asrError | message: string | ASR processing error | | blendshapeUpdate | number[] | 52 ARKit blendshape values | | playbackStarted | — | Audio playback started | | playbackEnded | — | Audio playback ended | | modelStatus | { model_loaded?: boolean; blendshape_count?: number } | Worker model status |

React Hook

import { useAStackCSR } from '@aether-stack-dev/client-sdk/react';

function CallUI() {
  const {
    isConnected, callStatus, transcript, response,
    blendshapes, error, connect, startCall, stopCall, sendText
  } = useAStackCSR({
    workerUrl: 'wss://your-worker-url',
    sessionToken: sessionToken,
  });

  return (
    <div>
      <p>Status: {callStatus}</p>
      <p>Transcript: {transcript}</p>
      <p>Response: {response}</p>
      <button onClick={() => connect().then(() => startCall())}>
        Start Call
      </button>
      <button onClick={stopCall}>Stop Call</button>
    </div>
  );
}

The hook returns:

| Field | Type | Description | |-------|------|-------------| | client | AStackCSRClient \| null | Underlying client instance | | isConnected | boolean | WebSocket connection state | | callStatus | CallStatus | idle | starting | active | stopping | error | | blendshapes | number[] | Current blendshape values | | transcript | string | Latest user transcript | | response | string | Latest AI response | | error | Error \| null | Latest error | | connect | () => Promise<void> | Connect to worker | | disconnect | () => void | Disconnect | | startCall | () => Promise<void> | Start call | | stopCall | () => void | Stop call | | sendText | (message: string) => void | Send text input |

Pass autoConnect: true in options to connect automatically on mount.

The hook also accepts a character config for custom avatars (see below).

Custom Characters

AStack supports custom VRM characters. Any VRM model with ARKit blendshapes will animate with the server's Audio2Face output.

VRMAvatar Props

| Prop | Type | Default | Description | |------|------|---------|-------------| | blendshapes | number[] | required | 52 ARKit blendshape values | | modelUrl | string | Supabase-hosted VRM | URL of the VRM model | | blendshapeMap | Record<string, string> | — | Custom blendshape name mapping (ARKit → model) | | expressionOverrides | Record<string, number> | — | Manual VRM expression values (0–1) | | onModelLoad | (report: ModelCompatibilityReport) => void | — | Callback with compatibility report after load | | maxModelSize | number | 31457280 (30MB) | Max model file size in bytes (0 to disable) | | width | number | 400 | Canvas width | | height | number | 400 | Canvas height | | backgroundColor | number \| null | 0xffffff | Background color, or null for transparent | | cameraPosition | [number, number, number] | [0, 1.4, 1.2] | Camera position in world space | | cameraTarget | [number, number, number] | [0, 1.3, 0] | Camera look-at target | | cameraFov | number | 30 | Camera field of view (degrees) | | lightIntensity | number | 1 | Multiplier for all scene lights | | idleAnimationUrl | string | — | URL of a .vrma animation clip for idle loop | | postProcessing | PostProcessingConfig | — | Post-processing effects (bloom, ao, dof) | | environmentUrl | string | — | URL of an .hdr environment map | | environmentIntensity | number | 1 | HDRI lighting intensity | | environmentBlur | number | 0 | HDRI background blur | | environmentZoom | number | 1 | HDRI background zoom level | | onEnvironmentLoad | () => void | — | Callback when HDRI environment finishes loading | | animationSpeed | number | 1 | Playback speed multiplier for idle animation clip | | animationWeight | number | 1 | Blend weight for idle animation clip (0–1) | | animationCrossfade | number | 0.5 | Crossfade duration (seconds) when switching animations | | orbitAngle | number | — | Camera azimuth angle (radians) for orbit control | | orbitElevation | number | — | Camera elevation angle (radians) for orbit control | | avatarRotation | number | — | Avatar Y-axis rotation (radians) | | onOrbitChange | (angle: number, elevation: number) => void | — | Callback when camera orbit changes | | features | VRMAvatarFeatures | — | Feature toggles (see below) |

VRMAvatarFeatures

Control which built-in behaviors are active:

interface VRMAvatarFeatures {
  idleAnimation?: boolean;     // Subtle breathing/swaying (default: true)
  microExpressions?: boolean;  // Random blinks, smiles, brow raises (default: true)
  springBones?: boolean;       // Hair/clothing physics simulation (default: true)
  postProcessing?: PostProcessingConfig;
  expressionPresets?: boolean; // VRM expression preset support (default: true)
}

Custom Model URL

<VRMAvatar blendshapes={blendshapes} modelUrl="https://cdn.example.com/my-character.vrm" />

Blendshape Mapping

For models with non-standard blendshape names, provide a mapping from ARKit names to your model's names:

import { VROID_BLENDSHAPE_MAP } from '@aether-stack-dev/client-sdk';

// Use the built-in VRoid Studio preset
<VRMAvatar blendshapes={blendshapes} blendshapeMap={VROID_BLENDSHAPE_MAP} />

// Or provide a custom mapping
<VRMAvatar blendshapes={blendshapes} blendshapeMap={{ jawOpen: 'mouth_open', eyeBlinkLeft: 'blink_L' }} />

Advanced Rendering

VRMAvatar supports expression presets, animation clips, HDRI environments, post-processing (bloom, ambient occlusion, depth of field), and camera control. See the Rendering Features documentation for full details and examples.

<VRMAvatar
  blendshapes={blendshapes}
  expressionOverrides={{ happy: 0.5 }}
  idleAnimationUrl="https://cdn.example.com/idle.vrma"
  environmentUrl="https://cdn.example.com/studio.hdr"
  postProcessing={{ bloom: 0.3, ao: 0.5, dof: true }}
/>

Compatibility Report

<VRMAvatar
  blendshapes={blendshapes}
  modelUrl="/custom.vrm"
  onModelLoad={(report) => {
    console.log(`${report.supported}/52 blendshapes supported`);
    console.log('Missing:', report.missing);
    console.log('Warnings:', report.warnings);
    console.log('Stats:', report.modelStats);
  }}
/>

Hook Character Config

const { blendshapes, characterConfig } = useAStackCSR({
  workerUrl, sessionToken,
  character: {
    modelUrl: 'https://cdn.example.com/my-character.vrm',
    blendshapeMap: VROID_BLENDSHAPE_MAP,
    onModelLoad: (report) => console.log(report),
  },
});

// Spread characterConfig directly on VRMAvatar
<VRMAvatar blendshapes={blendshapes} {...characterConfig} />

See the full character documentation for VRM requirements, ecosystem links, and integration guides.

Exports

Types

| Export | Description | |--------|-------------| | AStackCSRConfig | Constructor config interface | | AStackCSREvents | Event name/payload map | | CallStatus | 'idle' \| 'starting' \| 'active' \| 'stopping' \| 'error' | | VRMAvatarProps | VRMAvatar component props (react subpath) | | VRMAvatarFeatures | Feature toggle interface (react subpath) | | ModelCompatibilityReport | Model load report (react subpath) | | PostProcessingConfig | Post-processing config (react subpath) | | ARKitBlendshapeName | Union of 52 ARKit blendshape name strings | | AudioChunk | Audio player chunk interface | | AudioPlayerEvents | Audio player event map | | CharacterConfig | Character config from useAStackCSR (react subpath) | | UseAStackCSROptions | Hook options (react subpath) | | UseAStackCSRReturn | Hook return type (react subpath) | | AStackError | Error class with error code | | ErrorCodes | Error code constants |

Constants

| Export | Description | |--------|-------------| | ARKIT_BLENDSHAPES | Array of 52 ARKit blendshape name strings | | BLENDSHAPE_COUNT | 52 | | CRITICAL_BLENDSHAPES | ['jawOpen', 'eyeBlinkLeft', 'eyeBlinkRight'] | | DEFAULT_BLENDSHAPE_MAP | Identity map (ARKit name → same name) | | VROID_BLENDSHAPE_MAP | VRoid Studio naming preset (react subpath) |

License

MIT