@vexyl.ai/aivg-sdk

AI Voice Gateway Browser SDK - WebSocket-based voice assistant for browsers.

What is Vexyl AI Voice Gateway?

Vexyl is an ** production-ready voice AI gateway** that enables real-time conversational AI for telephone systems, contact centers, and web applications. It acts as an intelligent middleware between your telephony infrastructure (Asterisk, FreeSWITCH, SIP) and modern AI services (OpenAI, Gemini, Sarvam, Deepgram).

The Problem We Solve

Building voice AI systems traditionally requires:

• Complex integration with multiple AI providers (STT, TTS, LLM)
• Managing real-time audio streaming and processing
• Handling telephony protocols (SIP, RTP, AudioSocket)
• Optimizing latency for natural conversations
• Supporting multiple languages and accents
• Implementing features like barge-in, call transfer, and sentiment analysis
• Managing costs across different AI services

Connect your web applications to AI-powered voice assistants with real-time speech-to-text, LLM processing, and text-to-speech.

Installation

npm install @vexyl.ai/aivg-sdk

Or via CDN:

<script src="https://unpkg.com/@vexyl.ai/aivg-sdk"></script>

Quick Start

import AIVoiceGateway from '@vexyl.ai/aivg-sdk';

const voice = new AIVoiceGateway({
    serverUrl: 'wss://your-server.com:8082',
    language: 'en-IN',
    onTranscript: (text, { isFinal }) => {
        console.log('User said:', text, isFinal ? '(final)' : '(partial)');
    },
    onResponse: (text) => {
        console.log('AI said:', text);
    },
    onError: (error) => {
        console.error('Error:', error.message);
    }
});

// Connect and start listening
await voice.connect();
await voice.startListening();

Usage

ES Modules

import AIVoiceGateway from '@vexyl.ai/aivg-sdk';

CommonJS

const AIVoiceGateway = require('@vexyl.ai/aivg-sdk');

Browser (CDN)

<script src="https://unpkg.com/@vexyl.ai/aivg-sdk"></script>
<script>
    const voice = new AIVoiceGateway({ serverUrl: 'wss://...' });
</script>

Configuration Options

const voice = new AIVoiceGateway({
    // Required
    serverUrl: 'wss://your-server.com:8082',

    // Optional
    language: 'en-IN',              // Language code (default: 'en-IN')
    apiKey: 'your-api-key',         // API key for authentication
    metadata: {                      // Custom session metadata
        botId: 'sales-bot',
        callerName: 'John Doe',
        department: 'sales'
    },
    autoGreet: false,               // AI speaks first on connect
    greetingMessage: 'hi',          // Message for auto-greeting
    autoReconnect: true,            // Auto-reconnect on disconnect
    maxReconnectAttempts: 5,        // Max reconnection attempts
    reconnectDelay: 1000,           // Delay between attempts (ms)

    // Callbacks
    onConnect: ({ uuid }) => {},
    onDisconnect: ({ code, reason }) => {},
    onTranscript: (text, { isFinal }) => {},
    onResponse: (text) => {},
    onAudio: (arrayBuffer) => {},
    onStatus: (status) => {},
    onError: ({ code, message }) => {},
    onHangup: ({ reason }) => {}
});

API Methods

Connection

// Connect to server
await voice.connect();

// Disconnect
voice.disconnect();

Audio Control

// Start listening (requests microphone permission)
await voice.startListening();

// Stop listening
voice.stopListening();

// Mute/unmute microphone
voice.mute();
voice.unmute();

Configuration

// Change language
voice.setLanguage('hi-IN');

// Update metadata (e.g., change bot mid-session)
voice.updateMetadata({ botId: 'support-bot' });

Status

// Get current status
const status = voice.getStatus();
// { isConnected, isListening, isMuted, uuid, language }

// Check if audio is playing
if (voice.isAudioPlaying()) {
    console.log('AI is speaking');
}

Audio Visualization

// Get Web Audio API analyser for visualization
const analyser = voice.getAnalyser();

// Get frequency data
const frequencyData = voice.getFrequencyData();
if (frequencyData) {
    // Use for visualization (e.g., waveform, spectrum)
}

Examples

Basic Voice Chat

const voice = new AIVoiceGateway({
    serverUrl: 'wss://voice.example.com:8082',
    onTranscript: (text, { isFinal }) => {
        document.getElementById('transcript').textContent = text;
    },
    onResponse: (text) => {
        document.getElementById('response').textContent = text;
    }
});

document.getElementById('start-btn').onclick = async () => {
    await voice.connect();
    await voice.startListening();
};

document.getElementById('stop-btn').onclick = () => {
    voice.disconnect();
};

With Dynamic Bot Selection

const voice = new AIVoiceGateway({
    serverUrl: 'wss://voice.example.com:8082',
    metadata: {
        botId: 'sales-bot',      // Select which bot to use
        callerName: 'John',
        department: 'sales'
    }
});

AI Speaks First (Auto-Greeting)

const voice = new AIVoiceGateway({
    serverUrl: 'wss://voice.example.com:8082',
    autoGreet: true,
    greetingMessage: 'Hello, how can I help you today?'
});

Audio Visualization

const voice = new AIVoiceGateway({
    serverUrl: 'wss://voice.example.com:8082'
});

await voice.connect();
await voice.startListening();

// Visualize audio
function visualize() {
    const data = voice.getFrequencyData();
    if (data) {
        // Draw waveform or spectrum
        const avg = data.reduce((a, b) => a + b, 0) / data.length;
        console.log('Audio level:', avg);
    }
    requestAnimationFrame(visualize);
}
visualize();

Supported Languages

The SDK supports any language configured on your AI Voice Gateway server:

• Indian Languages (via Sarvam): Hindi, Malayalam, Tamil, Telugu, Kannada, Bengali, etc.
• International Languages (via Groq/Gemini): 90+ languages

Common language codes:

• en-IN - English (India)
• hi-IN - Hindi
• ml-IN - Malayalam
• ta-IN - Tamil
• te-IN - Telugu
• en-US - English (US)

Error Handling

const voice = new AIVoiceGateway({
    serverUrl: 'wss://voice.example.com:8082',
    onError: (error) => {
        switch (error.code) {
            case 'WS_ERROR':
                console.error('WebSocket error');
                break;
            case 'MIC_ERROR':
                console.error('Microphone access denied');
                break;
            case 'RECORDER_ERROR':
                console.error('Recording error');
                break;
            default:
                console.error('Error:', error.message);
        }
    }
});

Browser Support

• Chrome 66+
• Firefox 60+
• Safari 14.1+
• Edge 79+

Requires:

• WebSocket API
• MediaRecorder API
• Web Audio API
• getUserMedia API

License

MIT

Links

• GitHub Repository
• Documentation
• Issues