Telnyx Voice AI Widget

Overview

The Telnyx Voice AI Widget is a web component that allows you to easily integrate voice capabilities into your web applications. It provides a simple interface for making and receiving calls, as well as handling voice interactions using Telnyx's Voice AI services.

Installation

Via CDN (recommended for quick start)

<telnyx-ai-agent agent-id="assistant-xxx"></telnyx-ai-agent>
<script async src="https://unpkg.com/@telnyx/ai-agent-widget"></script>

Via npm/yarn

npm install @telnyx/ai-agent-widget
# or
yarn add @telnyx/ai-agent-widget

Then import in your application:

import '@telnyx/ai-agent-widget';

Usage

To use the Telnyx Voice AI Widget, add the custom element to your HTML:

<telnyx-ai-agent agent-id="assistant-xxx"></telnyx-ai-agent>

Call Options

You can customize the call options by adding attributes to the <telnyx-ai-agent> tag:

• call-destination-number
• call-caller-number
• call-caller-name
• call-custom-headers
• call-audio
• chat-mode — enable text-only mode (no microphone). See Chat Mode below.

Chat Mode

Enable text-only interaction (no microphone) with the chat-mode attribute. When enabled:

• No microphone permission is requested — the WebRTC peer negotiates audio as recvonly
• The conversation auto-starts immediately after the client connects (no "Start Call" button)
• The widget starts in the expanded state to show the transcript
• The audio visualizer, mute button, and <audio> element are hidden
• Agent audio tracks are disabled so no speech is played back
• The user interacts via sendConversationMessage() (text input in the transcript UI)

<telnyx-ai-agent agent-id="assistant-xxx" chat-mode></telnyx-ai-agent>

Toggling chat mode mid-conversation

The chat-mode attribute is a watched attribute — changing it at runtime triggers a full client teardown and recreation. Toggling from voice to chat mode (or vice versa) destroys the current WebRTC session and starts a new one:

• The active call is hung up
• The underlying WebRTC connection is disconnected
• A new TelnyxAIAgent client is created with the updated chatMode value
• A fresh connection is established and a new conversation begins

This means the transcript and conversation context are not preserved across a toggle. If you need to maintain conversation continuity, consider using the conversation-id attribute so the AI agent can resume context on the new session:

<telnyx-ai-agent
  agent-id="assistant-xxx"
  conversation-id="my-conversation-123"
></telnyx-ai-agent>

<script>
  const widget = document.querySelector('telnyx-ai-agent');

  // Toggle to chat mode — destroys the current session and starts fresh
  widget.setAttribute('chat-mode', '');

  // Toggle back to voice mode — another full teardown/reconnect
  widget.removeAttribute('chat-mode');
</script>

After a conversation ends in chat mode, the widget will not auto-restart even if chat-mode remains set. The user must interact again or you must remove and re-add the element.

Region

Pin signalling to a specific region with the region attribute. When set, the WebRTC SDK routes the connection through <region>.rtc.telnyx.com instead of the default anycast rtc.telnyx.com. Useful when anycast DNS routes a client to a sub-optimal datacenter.

<telnyx-ai-agent agent-id="assistant-xxx" region="apac"></telnyx-ai-agent>

Known regions: us-east, us-central, us-west, ca-central, eu, apac, south-asia.

Environment

Select which Telnyx environment the widget connects to with the environment attribute. It is forwarded to the underlying @telnyx/webrtc SDK, which picks the WebRTC signalling host accordingly. Defaults to production.

| Value | WebRTC signalling host | When to use | | ---------------------- | ------------------------ | --------------------------------------------------------------------------------------------- | | production (default) | wss://rtc.telnyx.com | Normal usage against Telnyx production. | | development | wss://rtcdev.telnyx.com | Testing an agent/backend deployed to the Telnyx dev environment (e.g. pre-release AI features). |

<telnyx-ai-agent agent-id="assistant-xxx" environment="development"></telnyx-ai-agent>

development routes signalling to rtcdev.telnyx.com. Only use it when your assistant and the backend changes you are testing are deployed to Telnyx dev — production assistants are not reachable there. environment and region are independent: region only rewrites the production host and does not apply to development.

VAD (Voice Activity Detection) Options

Configure Voice Activity Detection for speech detection and latency measurement by passing a JSON object to the vad attribute:

<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"volumeThreshold": 10, "silenceDurationMs": 500, "minSpeechDurationMs": 100}'
></telnyx-ai-agent>

Available VAD options:

| Option | Type | Default | Description | | --------------------- | -------- | ----------- | --------------------------------------------------------------------------------------------------------------------------- | | volumeThreshold | number | 10 | Volume threshold (0-255) for detecting speech. Audio levels above this value are considered speech. | | silenceDurationMs | number | 1000 | Duration of silence (ms) before triggering "thinking" state. Lower values = faster response but may cut off natural pauses. | | minSpeechDurationMs | number | 100 | Minimum speech duration (ms) to count as real user speech. Filters out brief noise spikes. | | maxLatencyMs | number | undefined | Maximum latency (ms) to report. Values above this are considered stale and won't be reported. |

Example configurations:

<!-- Fast-paced conversation (aggressive turn detection) -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"silenceDurationMs": 500, "minSpeechDurationMs": 80}'
></telnyx-ai-agent>

<!-- Thoughtful conversation (tolerant of pauses) -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"silenceDurationMs": 1500, "minSpeechDurationMs": 150}'
></telnyx-ai-agent>

<!-- Noisy environment -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"volumeThreshold": 20, "minSpeechDurationMs": 200}'
></telnyx-ai-agent>

Latency Display Options

Control whether latency measurements are included in transcript messages. Both options default to false.

| Attribute | Type | Default | Description | | ----------------------------- | --------- | ------- | ------------------------------------------------------------------------------------------------------ | | show-user-perceived-latency | boolean | false | Include userPerceivedLatencyMs in transcript items (time from user stop speaking to agent response). | | show-greeting-latency | boolean | false | Include greetingLatencyMs in transcript items (time for initial agent greeting). |

<!-- Enable latency tracking in transcript messages -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  show-user-perceived-latency
  show-greeting-latency
></telnyx-ai-agent>

When enabled, latency values are attached to assistant transcript items and can be accessed via the transcript.item event:

widget.addEventListener('transcript.item', function (event) {
  const { role, content, userPerceivedLatencyMs, greetingLatencyMs } =
    event.detail;
  if (role === 'assistant') {
    if (userPerceivedLatencyMs !== undefined) {
      console.log('Response latency:', userPerceivedLatencyMs, 'ms');
    }
    if (greetingLatencyMs !== undefined) {
      console.log('Greeting latency:', greetingLatencyMs, 'ms');
    }
  }
});

Events and Callbacks

The widget emits DOM CustomEvents that you can listen to for tracking analytics, updating UI, or integrating with your application logic. Event names match @telnyx/ai-agent-lib for consistency.

<script>
  document.addEventListener('DOMContentLoaded', function () {
    const widget = document.querySelector('telnyx-ai-agent');

    // Listen for call lifecycle events
    widget.addEventListener('conversation.update', function (event) {
      const { callState } = event.detail;
      if (callState === 'active') {
        console.log('Voice call started');
      }
      if (callState === 'destroy') {
        console.log('Voice call ended');
      }
    });

    widget.addEventListener('transcript.item', function (event) {
      console.log('Message received:', event.detail);
      // Process message, update state, etc.
    });

    widget.addEventListener('conversation.agent.state', function (event) {
      const { state, userPerceivedLatencyMs, thinkingStartedAt } = event.detail;
      console.log('Agent state:', state);
      if (userPerceivedLatencyMs) {
        console.log('Response latency:', userPerceivedLatencyMs, 'ms');
      }
    });

    widget.addEventListener('agent.error', function (event) {
      console.error('Widget error:', event.detail);
      // Handle errors, show fallback UI, etc.
    });
  });
</script>

Available Events

| Event | Description | Detail | | -------------------------- | ---------------------------------------------------- | -------------------------------------------------------- | | agent.connected | Agent connected to platform | - | | agent.disconnected | Agent disconnected from platform | - | | agent.error | Error occurred | { message, name } | | transcript.item | Transcript message received | { id, role, content, timestamp, attachments? } | | conversation.update | Conversation state updated | { type, callState } | | conversation.agent.state | Agent state changed (listening/speaking/thinking) | { state, userPerceivedLatencyMs?, thinkingStartedAt? } | | agent.audio.mute | Agent audio muted/unmuted | { muted } | | client.tool.invoked | A client-side tool was invoked by the agent | { callId, toolName } | | client.tool.completed | A client-side tool produced an output for the agent | { callId, toolName, isError } | | client.tool.error | A client-side tool failed; a safe error was returned | { callId, toolName, reason } |

Client-side Tools

The widget can execute client-side tools requested by the AI agent (for example, looking something up in the page, reading local state, or triggering a UI action) and return the result to the agent. This implements the PR-531 client_side_tool flow on top of @telnyx/ai-agent-lib.

Because tool handlers are JavaScript functions, they cannot be passed as HTML attributes. Instead, register them imperatively on the widget element:

<telnyx-ai-agent agent-id="assistant-xxx"></telnyx-ai-agent>

<script>
  const widget = document.querySelector('telnyx-ai-agent');

  // Register a handler. `args` is the parsed JSON arguments object the agent
  // sent; return any JSON-serializable value (or a string) as the result.
  widget.registerClientTool('get_cart_total', async (args, context) => {
    // context = { callId, toolName, rawArguments }
    return { total: 42.5, currency: 'USD' };
  });

  // Inspect or remove handlers later:
  widget.getClientTools(); // ['get_cart_total']
  widget.unregisterClientTool('get_cart_total'); // true

  // Observe execution via DOM events (payloads never include raw
  // arguments/outputs — only safe correlation fields):
  widget.addEventListener('client.tool.invoked', (e) =>
    console.log('tool invoked', e.detail),
  );
  widget.addEventListener('client.tool.completed', (e) =>
    console.log('tool completed', e.detail),
  );
  widget.addEventListener('client.tool.error', (e) =>
    console.warn('tool error', e.detail),
  );
</script>

Notes:

• Register handlers as early as you like — registrations are durable and are automatically re-applied across the widget's internal reconnects.
• The library always returns a function_call_output to the agent (a safe error payload on unknown tool, invalid arguments, handler throw, or timeout), so a missing or failing handler never hangs the conversation.
• Raw tool arguments and outputs are never logged.

Development

To develop the Telnyx Voice AI Widget, you can clone the repository and run the following commands:

yarn install --immutable
yarn dev