2 way audio

Expo native module for reliable two-way audio communication with echo cancellation on iOS and Android.

Uses Apple's AVAudioEngine with Voice Processing for reliable, system-level echo cancellation.

Features

• PCM Audio Streaming: Record and stream audio as PCM16 at native sample rate (48kHz)
• Echo Cancellation (AEC): Built-in voice processing prevents speaker audio from being picked up by the microphone
• Sound Effects (SFX): Play sound effects without interrupting recording - no conflicts!
• Low Latency: Optimized for real-time voice communication
• Volume Monitoring: Real-time input and output volume levels
• Interruption Handling: Automatic handling of phone calls and other audio interruptions
• iOS Microphone Modes: Support for iOS 15+ microphone modes (Voice Isolation, Wide Spectrum)
• Session Recovery: Automatic and manual recovery from audio session interruptions

Installation

yarn add clox-two-way-audio

iOS Setup

Add to your Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app needs microphone access for two-way audio communication.</string>

Android Setup

Add to your AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />

Quick Start

import {
  initialize,
  cleanup,
  toggleRecording,
  playPCMData,
  addCloxTwoWayAudioEventListener,
  requestMicrophonePermissionsAsync,
  tearDown,
} from 'clox-two-way-audio';

// Request permission
await requestMicrophonePermissionsAsync();

// Clean up any previous session and initialize
cleanup();
await initialize();

// Listen for microphone data
const subscription = addCloxTwoWayAudioEventListener('onMicrophoneData', (event) => {
  // event.data is Uint8Array of PCM16 audio (native sample rate, mono)
  sendToServer(event.data);
});

// Start recording
toggleRecording(true);

// Play received audio (with echo cancellation active)
function onAudioFromServer(pcmData: Uint8Array) {
  // For OpenAI TTS (24kHz):
  playPCMData(pcmData, 24000);
  
  // For 16kHz audio:
  // playPCMData(pcmData, 16000);
  
  // For native 48kHz (no conversion):
  // playPCMData(pcmData);
}

// Clean up when done
subscription.remove();
tearDown();

Sound Effects (No Conflicts!)

Play sound effects through the same audio engine - recording continues uninterrupted:

import {
  preloadSoundFiles,
  playSoundFile,
  clearSoundCache,
} from 'clox-two-way-audio';

// Preload sounds on app startup (handles remote URLs)
await preloadSoundFiles([
  'https://example.com/click.wav',
  'https://example.com/success.wav',
  'https://example.com/notification.wav',
]);

// Play instantly - won't interrupt recording!
playSoundFile('https://example.com/click.wav', 0.8); // volume 0-1

// Clear cache when done
clearSoundCache();

Why Use Built-in SFX?

Using external audio libraries like expo-audio will reconfigure the shared AVAudioSession, causing:

• Recording to stop
• Need for full reinitialization

Our built-in SFX uses the same AVAudioEngine, so no conflicts occur.

API Reference

Core Functions

initialize(): Promise<boolean>

Initialize the audio engine. Must be called before any other functions.

cleanup(): void

Clean up any existing audio sessions. Safe to call even if not initialized. Call this on app startup before initialize() to ensure a clean state.

toggleRecording(val: boolean): boolean

Start or stop recording. Returns the current recording state.

playPCMData(audioData: Uint8Array, sampleRate?: number): void

Play PCM16 audio data through the speaker with echo cancellation active.

• sampleRate - Optional sample rate of the input audio. Common values:
  • 24000 - OpenAI TTS output
  • 16000 - Common speech rate
  • 48000 - Native hardware rate (default if not specified)

flushPlaybackQueue(): void

Stop playback and clear all scheduled audio buffers. Use to interrupt playback.

isRecording(): boolean

Check if recording is currently active.

isPlaying(): boolean

Check if audio is currently playing.

tearDown(): void

Release all resources. Call when done with audio.

restart(): void

Restart audio after an interruption.

getSampleRate(): number

Get the native sample rate (usually 48000Hz). Audio is captured and played at this rate.

Session Recovery Functions

reconfigureAudioSession(): boolean

Reconfigure audio session after another app takes over. Returns true if successful.

fullReinitialize(): Promise<boolean>

Full reinitialization when the engine is in a bad state. Fires onReinitialize event.

Voice Processing Controls

bypassVoiceProcessing(bypass: boolean): void

Disable echo cancellation (for testing only).

setAGCEnabled(enabled: boolean): void

Enable or disable Automatic Gain Control. Disabling can reduce residual echo.

getVoiceProcessingStatus(): Record<string, any>

Get current voice processing status flags and sample rate.

Sound Effects (SFX) Functions

preloadSoundFile(urlString: string): Promise<boolean>

Preload a single sound file. Handles both local and remote URLs.

preloadSoundFiles(urlStrings: string[]): Promise<number>

Preload multiple sound files in parallel. Returns count of successfully loaded files.

playSoundFile(urlString: string, volume?: number): void

Play a sound file immediately. Volume is 0.0 to 1.0 (default 1.0).

stopSoundEffect(): void

Stop the currently playing sound effect.

clearSoundCache(): void

Clear all cached sound files to free memory.

Permission Functions

getMicrophonePermissionsAsync(): Promise<PermissionResponse>

Get current microphone permission status.

requestMicrophonePermissionsAsync(): Promise<PermissionResponse>

Request microphone permission.

iOS-Specific Functions

getMicrophoneModeIOS(): MicrophoneMode | ''

Get the current iOS microphone mode ('standard', 'voiceIsolation', 'wideSpectrum').

setMicrophoneModeIOS(): void

Show iOS system UI for microphone mode selection.

Events

onMicrophoneData

Emitted when audio data is available from the microphone.

{ data: Uint8Array } // PCM16 audio data at native sample rate

onInputVolumeLevelData

Emitted with current input (microphone) volume level.

{ data: number } // 0-1 normalized level

onOutputVolumeLevelData

Emitted with current output (speaker) volume level.

{ data: number } // 0-1 normalized level

onRecordingChange

Emitted when recording state changes (including when interrupted by system).

{ data: boolean } // true if recording

onAudioInterruption

Emitted when audio is interrupted.

{ data: 'began' | 'ended' | 'blocked' }

onReinitialize

Emitted after a full reinitialization.

{ success: boolean, reason: string }

React Hooks

import {
  useMicrophoneData,
  useInputVolumeLevel,
  useOutputVolumeLevel,
  useRecordingChange,
  useAudioInterruption,
  useInputVolumeLevelState,
  useOutputVolumeLevelState,
  useRecordingState,
} from 'clox-two-way-audio';

// Callback hooks
useMicrophoneData((event) => { /* handle audio data */ });
useInputVolumeLevel((event) => { /* handle input level */ });

// State hooks (auto-updating)
const inputLevel = useInputVolumeLevelState(); // 0-1
const outputLevel = useOutputVolumeLevelState(); // 0-1
const isRecording = useRecordingState(); // boolean

How Echo Cancellation Works

This module uses AVAudioEngine with iOS's built-in Voice Processing:

1. AVAudioEngine - Apple's native audio graph API
2. AVAudioSession - Configured with .playAndRecord category and .voiceChat mode
3. Voice Processing - setVoiceProcessingEnabled(true) on the input node provides:

• Acoustic Echo Cancellation (AEC)
• Noise Suppression
• Automatic Gain Control (AGC)

AEC Adaptation Period

When playback starts for the first time, the AEC algorithm needs a brief moment to adapt. The module automatically discards the first 500ms of microphone input after playback begins to prevent initial echo.

iOS 18.2+ Enhanced AEC

On iOS 18.2+, the module enables setPrefersEchoCancelledInput(true) for enhanced echo cancellation.

Audio Format

| Property | Value | | ----------- | ------------------------ | | Sample Rate | Native (typically 48kHz) | | Channels | Mono (1 channel) | | Bit Depth | 16-bit signed integers | | Format | PCM16 (Little-endian) |

Note: Audio is captured and played at the native hardware sample rate (usually 48kHz) with no format conversion, ensuring optimal AEC performance.

Handling External Audio Libraries

If you must use external audio libraries (like expo-audio), they will interrupt recording. Handle this with:

import { addCloxTwoWayAudioEventListener, fullReinitialize } from 'clox-two-way-audio';

// Listen for recording state changes
addCloxTwoWayAudioEventListener('onRecordingChange', (event) => {
  if (!event.data) {
    console.log('Recording stopped (possibly by external audio)');
  }
});

// After external audio finishes, reinitialize
await fullReinitialize();

Better approach: Use built-in playSoundFile() for sound effects to avoid conflicts entirely.

Requirements

• iOS 15.0+
• Android API 24+
• Expo SDK 54+

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     JavaScript Layer                         │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────────────┐ │
│  │   Events    │  │   Hooks      │  │   Core Functions    │ │
│  └─────────────┘  └──────────────┘  └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                    Native Module Layer                       │
│  ┌─────────────────────────────────────────────────────────┐│
│  │              CloxTwoWayAudioModule                       ││
│  │   - Permission handling                                  ││
│  │   - Event emission                                       ││
│  │   - JS <-> Native bridge                                 ││
│  └─────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                     AudioEngine Layer                        │
│  ┌─────────────────────────────────────────────────────────┐│
│  │                  AVAudioEngine                           ││
│  │   ┌───────────┐  ┌─────────────┐  ┌─────────────────┐   ││
│  │   │  Mixer    │  │Speech Player│  │  Input Node     │   ││
│  │   │           │  │(PCM Stream) │  │(Voice Process)  │   ││
│  │   │           │  ├─────────────┤  │  - AEC          │   ││
│  │   │           │  │ SFX Player  │  │  - Noise Supp.  │   ││
│  │   │           │  │(Sound FX)   │  │  - AGC          │   ││
│  │   └───────────┘  └─────────────┘  └─────────────────┘   ││
│  └─────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────┘

Troubleshooting

Recording stops when playing sounds

Use the built-in playSoundFile() instead of external audio libraries.

Echo not fully cancelled

1. Ensure voiceChat mode is active: check getVoiceProcessingStatus()
2. Try disabling AGC: setAGCEnabled(false)
3. On iOS 15+, show the mic mode picker: setMicrophoneModeIOS() and select "Voice Isolation"

Can't start recording after interruption

Call fullReinitialize() to completely reset the audio engine.

Sound effects have latency

Preload sounds on app startup with preloadSoundFiles() for instant playback.

License

MIT