JS SDK for clients to connect to voice-chat ws server.

Features

• 🎙️ Real-time Audio Streaming: Capture and stream microphone input with configurable sample rates
• 🔊 Audio Playback: Queue and play audio chunks with checkpoint synchronization
• ⚛️ React Integration: Ready-to-use hooks for React applications
• 🌐 Framework Agnostic: Core classes work in any JavaScript environment
• 📦 Type Safe: Full TypeScript support with complete type definitions
• 🎯 WebSocket Protocol: Standardized protocol for voice AI server communication

Installation

yarn add @hunar.ai/voice-chat-client

Quick Start

Note: This library is primarily supports React applications. Vanilla JavaScript/HTML support will be added in the future.

import React from "react";
import {
  RealtimeProvider,
  useRealtimeContext,
  useMicrophone,
  useMicrophoneSelector,
  useMicrophonePermission,
} from "@hunar.ai/voice-chat-client";

function VoiceChat() {
  const { client, clientState, connect, disconnect } = useRealtimeContext();
  const { currentDevice, audioDevices, changeAudioDevice } =
    useMicrophoneSelector();
  const { hasMicrophonePermission, requestMicrophonePermission } =
    useMicrophonePermission();

  const { isRecording, startRecording, stopRecording } = useMicrophone({
    currentDevice,
    onChunk: (audio, contentType, sampleRate) => {
      client.sendAudio(audio, contentType, sampleRate);
    },
  });

  React.useEffect(() => {
    connect({
      onConnect: () => console.log("Connected"),
      onError: (error) => console.error("Error:", error),
      onDisconnect: () => console.log("Disconnected"),
    });

    return () => disconnect();
  }, [connect, disconnect]);

  return (
    <div>
      <div>Status: {clientState}</div>

      {!hasMicrophonePermission && (
        <button onClick={requestMicrophonePermission}>Enable Microphone</button>
      )}

      <select onChange={(e) => changeAudioDevice(e.target.value)}>
        {Object.values(audioDevices).map((device) => (
          <option key={device.deviceId} value={device.deviceId}>
            {device.label}
          </option>
        ))}
      </select>

      <button onClick={isRecording ? stopRecording : startRecording}>
        {isRecording ? "Stop" : "Start"}
      </button>
    </div>
  );
}

function App() {
  return (
    <RealtimeProvider wsUrl="wss://your-server.com">
      <VoiceChat />
    </RealtimeProvider>
  );
}

Audio Processing

The library uses an Audio Worklet for efficient real-time audio processing and PCM conversion. The worklet is automatically bundled with the package - no additional setup required.

The audio processing pipeline:

• Captures microphone input at configurable sample rates (default: 24kHz)
• Converts audio to 16-bit PCM format
• Streams audio chunks to your WebSocket server

Note: Audio worklets require HTTPS in production environments.

API Reference

Export Path Options:

• @hunar.ai/voice-chat-client - All exports (core classes, React hooks, and providers)

Core Classes

RealtimeClient<TAudioPlayer>

WebSocket client for voice AI communication.

const client = new RealtimeClient<TAudioPlayer>(
  url: string, createAudioPlayer: () => TAudioPlayer
);

// Connect to server
client.connect({
  onConnect: () => void,
  onError: (error: Error) => void,
  onDisconnect: () => void,
});

// Send audio data
client.sendAudio(
  audio: ArrayBuffer | Int16Array,
  contentType?: string,    // default: 'audio/pcm'
  sampleRate?: number      // default: 24000
);

// Send context updates
client.sendContextUpdate(context: Record<string, unknown>);

// Send markers
client.sendMark(marker: string);

// Disconnect
client.disconnect();

RealtimeClientState

Connection state enumeration for the RealtimeClient.

enum RealtimeClientState {
  CONNECTING = "connecting",
  CONNECTED = "connected",
  DISCONNECTED = "disconnected",
}

AudioManager<TCheckpoint, TAudioPlayer>

Manages audio playback queue with checkpoint synchronization.

// TCheckpoint is the type of the checkpoint data. For example it could be a string or an object.
// TAudioPlayer extends BaseAudioPlayer for custom audio player implementations.
const audioManager = new AudioManager<TCheckpoint, TAudioPlayer>({
  onCheckpointPlayed: (checkpoint: TCheckpoint) => void,
  createAudioPlayer: () => TAudioPlayer
});

// Add audio chunk to queue
audioManager.addChunk(audioData: ArrayBuffer);

// Add checkpoint marker
audioManager.addCheckpoint(checkpointData: TCheckpoint);

// Control playback
audioManager.stop();

BaseAudioPlayer

Abstract base class for audio playback implementations.

abstract class BaseAudioPlayer {
  abstract isPlaying: boolean;
  abstract get canPlay(): boolean;
  abstract play(props: AudioPlayerPlayProps): void;
  abstract stop(): void;
}

BrowserAudioPlayer

Web Audio API implementation of BaseAudioPlayer for browser environments.

const audioPlayer = new BrowserAudioPlayer();

// Play audio buffer
audioPlayer.play({
  buffer: audioData,
  onended: () => console.log("Playback finished"),
});

// Stop playback
audioPlayer.stop();

// Check playback state
console.log(audioPlayer.isPlaying);

React Hooks

useRealtimeContext

Provides access to the RealtimeClient instance and connection state within a RealtimeProvider.

const {
  client, // RealtimeClient instance
  clientState, // RealtimeClientState (CONNECTED | DISCONNECTED | CONNECTING)
  connect, // (props: {onConnect, onError, onDisconnect}) => void
  disconnect, // () => void
  isPlaying, // boolean: current playback state
} = useRealtimeContext();

RealtimeProvider

Context provider that manages a RealtimeClient instance and tracks connection state.

<RealtimeProvider wsUrl="wss://your-server.com">
  <YourApp />
</RealtimeProvider>

The provider automatically manages the WebSocket connection lifecycle and provides access to the current connection state via useRealtimeContext.

useMicrophone

Manages microphone recording with audio streaming.

const {
  isRecording,              // boolean: current recording state
  startRecording,           // () => Promise<void>: start recording
  stopRecording,            // () => void: stop recording and cleanup
  toggleMicrophoneCapture,  // () => void: pause/resume recording without cleanup
} = useMicrophone({
  currentDevice: string | null,
  onChunk: (audio, contentType?, sampleRate?) => void,
  sampleRate?: number,      // default: 24000
});

useMicrophoneSelector

Lists and manages audio input devices.

const {
  audioDevices, // AudioDeviceMap
  currentDevice, // string | null
  changeAudioDevice, // (deviceId: string | null) => Promise<void>
  handleDeviceChangeEvent, // () => void
} = useMicrophoneSelector({
  workletPath: string,
});

useMicrophonePermission

Handles microphone permission requests.

const {
  hasMicrophonePermission, // boolean
  permissionError, // string
  checkMicrophonePermission, // () => Promise<void>
  requestMicrophonePermission, // () => Promise<boolean>
} = useMicrophonePermission();

useMedia

Low-level hook for media device access and audio recording.

const {
  getAudioDevices, // () => Promise<AudioDeviceMap>
  getAudioDevice, // (deviceId: string) => Promise<MediaStream>
  createAudioRecorder, // (onChunk, options?) => MediaRecorder
} = useMedia({
  workletPath: string,
});

WebSocket Protocol

The client follows this protocol for communication:

Client → Server Messages

Start Stream

{
  "event": "start",
  "start": { "streamId": "uuid" }
}

Send Audio

{
  "event": "media",
  "streamId": "uuid",
  "media": {
    "contentType": "audio/pcm",
    "sampleRate": 24000,
    "payload": "base64-encoded-audio"
  }
}

Context Update

{
  "event": "contextUpdate",
  "streamId": "uuid",
  "context": { "key": "value" }
}

Mark Played

{
  "event": "playedStream",
  "streamId": "uuid",
  "name": "marker-name"
}

Server → Client Messages

Play Audio

{
  "event": "playAudio",
  "media": {
    "contentType": "audio/pcm",
    "sampleRate": 24000,
    "payload": "base64-encoded-audio"
  }
}

Checkpoint

{
  "event": "checkpoint",
  "streamId": "uuid",
  "name": "checkpoint-name"
}

Clear Audio

{
  "event": "clearAudio",
  "streamId": "uuid"
}

TypeScript Support

The library is written in TypeScript and includes complete type definitions. All exports are fully typed.

import type {
  // Audio Manager types
  QueuedItemType,
  QueuedAudioItem,
  QueuedCheckpointItem,
  QueueItem,

  // Audio Player types
  AudioPlayerPlayProps,

  // Realtime Client types
  RealtimeClientEvent,
  RealtimeServerEvent,
  RealtimeClientState,
  RealtimeClientMessageProps,
  RealtimeServerMessageProps,
  RealtimeStartMessageProps,
  RealtimeContextUpdateMessageProps,
  RealtimeMediaMessageProps,
  RealtimeMarkMessageProps,
  RealtimeStopMessageProps,
  RealtimePlayAudioMessageProps,
  RealtimeClearAudioMessageProps,
  RealtimeClearedAudioMessageProps,
  RealtimeCheckpointMessageProps,

  // React Hook types
  AudioDevice,
  AudioDeviceMap,
  UseMicrophoneOptions,
} from "@hunar.ai/voice-chat-client";

Browser Compatibility

• Chrome/Edge: ✅ Full support
• Firefox: ✅ Full support
• Safari: ✅ Full support (14.1+)
• Opera: ✅ Full support

Requires:

• WebSocket API
• Web Audio API
• AudioWorklet API
• MediaDevices API

Examples

TBD

License

MIT

Attributions

This project includes code derived from external sources:

Audio Processor Worklet

The src/core/audio-processor-worker.js file is derived from the Pipecat Client Web Transports library:

• Source: https://github.com/pipecat-ai/pipecat-client-web-transports
• Original file: lib/wavtools/lib/worklets/audio_processor.js
• Raw URL: https://raw.githubusercontent.com/pipecat-ai/pipecat-client-web-transports/ad419d77b1221528ca1ea8ccaa973c0a8176f81a/lib/wavtools/lib/worklets/audio_processor.js
• Date accessed: October 24, 2025
• License: BSD 2-Clause License (Copyright © 2024, Daily)

Contributing

This is a source available project. Direct PR for contributions are not accepted. Please open issues for any bugs or feature requests.