Telnyx Voice AI Widget

Overview

The Telnyx Voice AI Widget is a web component that allows you to easily integrate voice capabilities into your web applications. It provides a simple interface for making and receiving calls, as well as handling voice interactions using Telnyx's Voice AI services.

Usage

To use the Telnyx Voice AI Widget, you need to include the widget script in your HTML file and initialize it with your Telnyx AI Assistant ID

<telnyx-ai-agent agent-id="assistant-xxx"></telnyx-ai-agent>
<script async src="https://unpkg.com/@telnyx/ai-agent-widget"></script>

Call Options

You can customize the call options by adding attributes to the <telnyx-ai-agent> tag:

• call-destination-number
• call-caller-number
• call-caller-name
• call-custom-headers
• call-audio

VAD (Voice Activity Detection) Options

Configure Voice Activity Detection for speech detection and latency measurement by passing a JSON object to the vad attribute:

<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"volumeThreshold": 10, "silenceDurationMs": 500, "minSpeechDurationMs": 100}'
></telnyx-ai-agent>

Available VAD options:

| Option | Type | Default | Description | | --------------------- | -------- | ----------- | --------------------------------------------------------------------------------------------------------------------------- | | volumeThreshold | number | 10 | Volume threshold (0-255) for detecting speech. Audio levels above this value are considered speech. | | silenceDurationMs | number | 1000 | Duration of silence (ms) before triggering "thinking" state. Lower values = faster response but may cut off natural pauses. | | minSpeechDurationMs | number | 100 | Minimum speech duration (ms) to count as real user speech. Filters out brief noise spikes. | | maxLatencyMs | number | undefined | Maximum latency (ms) to report. Values above this are considered stale and won't be reported. |

Example configurations:

<!-- Fast-paced conversation (aggressive turn detection) -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"silenceDurationMs": 500, "minSpeechDurationMs": 80}'
></telnyx-ai-agent>

<!-- Thoughtful conversation (tolerant of pauses) -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"silenceDurationMs": 1500, "minSpeechDurationMs": 150}'
></telnyx-ai-agent>

<!-- Noisy environment -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"volumeThreshold": 20, "minSpeechDurationMs": 200}'
></telnyx-ai-agent>

Latency Display Options

Control whether latency measurements are included in transcript messages. Both options default to false.

| Attribute | Type | Default | Description | | ----------------------------- | --------- | ------- | ------------------------------------------------------------------------------------------------------ | | show-user-perceived-latency | boolean | false | Include userPerceivedLatencyMs in transcript items (time from user stop speaking to agent response). | | show-greeting-latency | boolean | false | Include greetingLatencyMs in transcript items (time for initial agent greeting). |

<!-- Enable latency tracking in transcript messages -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  show-user-perceived-latency
  show-greeting-latency
></telnyx-ai-agent>

When enabled, latency values are attached to assistant transcript items and can be accessed via the transcript.item event:

widget.addEventListener('transcript.item', function (event) {
  const { role, content, userPerceivedLatencyMs, greetingLatencyMs } =
    event.detail;
  if (role === 'assistant') {
    if (userPerceivedLatencyMs !== undefined) {
      console.log('Response latency:', userPerceivedLatencyMs, 'ms');
    }
    if (greetingLatencyMs !== undefined) {
      console.log('Greeting latency:', greetingLatencyMs, 'ms');
    }
  }
});

Events and Callbacks

The widget emits DOM CustomEvents that you can listen to for tracking analytics, updating UI, or integrating with your application logic. Event names match @telnyx/ai-agent-lib for consistency.

<script>
  document.addEventListener('DOMContentLoaded', function () {
    const widget = document.querySelector('telnyx-ai-agent');

    // Listen for call lifecycle events
    widget.addEventListener('conversation.update', function (event) {
      const { callState } = event.detail;
      if (callState === 'active') {
        console.log('Voice call started');
      }
      if (callState === 'destroy') {
        console.log('Voice call ended');
      }
    });

    widget.addEventListener('transcript.item', function (event) {
      console.log('Message received:', event.detail);
      // Process message, update state, etc.
    });

    widget.addEventListener('conversation.agent.state', function (event) {
      const { state, userPerceivedLatencyMs, thinkingStartedAt } = event.detail;
      console.log('Agent state:', state);
      if (userPerceivedLatencyMs) {
        console.log('Response latency:', userPerceivedLatencyMs, 'ms');
      }
    });

    widget.addEventListener('agent.error', function (event) {
      console.error('Widget error:', event.detail);
      // Handle errors, show fallback UI, etc.
    });
  });
</script>

Available Events

| Event | Description | Detail | | -------------------------- | ------------------------------------------------- | -------------------------------------------------------- | | agent.connected | Agent connected to platform | - | | agent.disconnected | Agent disconnected from platform | - | | agent.error | Error occurred | { message, name } | | transcript.item | Transcript message received | { id, role, content, timestamp, attachments? } | | conversation.update | Conversation state updated | { type, callState } | | conversation.agent.state | Agent state changed (listening/speaking/thinking) | { state, userPerceivedLatencyMs?, thinkingStartedAt? } | | agent.audio.mute | Agent audio muted/unmuted | { muted } |

Development

To develop the Telnyx Voice AI Widget, you can clone the repository and run the following commands:

yarn install --immutable
yarn dev