Telnyx Voice AI Widget

Overview

The Telnyx Voice AI Widget is a web component that allows you to easily integrate voice capabilities into your web applications. It provides a simple interface for making and receiving calls, as well as handling voice interactions using Telnyx's Voice AI services.

Installation

Via CDN (recommended for quick start)

<telnyx-ai-agent agent-id="assistant-xxx"></telnyx-ai-agent>
<script async src="https://unpkg.com/@telnyx/ai-agent-widget"></script>

Via npm/yarn

npm install @telnyx/ai-agent-widget
# or
yarn add @telnyx/ai-agent-widget

Then import in your application:

import '@telnyx/ai-agent-widget';

Usage

To use the Telnyx Voice AI Widget, add the custom element to your HTML:

<telnyx-ai-agent agent-id="assistant-xxx"></telnyx-ai-agent>

Call Options

You can customize the call options by adding attributes to the <telnyx-ai-agent> tag:

• call-destination-number
• call-caller-number
• call-caller-name
• call-custom-headers
• call-audio
• trickle-ice — enable/disable trickle ICE. Defaults to true; set trickle-ice="false" to disable.
• prefetch-ice-candidates — enable/disable ICE candidate prefetching. Defaults to true; set prefetch-ice-candidates="false" to disable.
• chat-mode — hide the audio but still place a call. See Chat Mode below.
• text-only — a typing-only widget with no call at all. See Text-Only Mode below.

Text-Only Mode

text-only is the right choice when the visitor should only ever type. There is no call: no microphone permission prompt, no audio, no WebRTC, and no voice minutes. Turns go to the assistant over a WebSocket to voice-sdk-proxy and the answer streams back as text.

<telnyx-ai-agent agent-id="assistant-xxx" text-only></telnyx-ai-agent>

The widget opens straight into the transcript panel — there is no "Start Call" button and no collapsed state, since a visitor who is typing is already engaged and collapsing would only offer something this mode cannot do.

The page still needs nothing but the assistant id. The assistant chat API requires a Telnyx API key, which public page JavaScript cannot hold, so voice-sdk-proxy performs the same trust move it already performs for anonymous voice logins and no credential reaches the browser.

Requirements

The assistant must have telephony_settings.supports_unauthenticated_web_calls enabled — the same opt-in the voice widget requires. It is what marks an assistant as reachable by anonymous web visitors.

Events

The documented DOM events fire on a text-only widget the same as on a voice one: agent.connected (its detail carries the conversationId rather than the voice path's dc / region / callReportId, which have no counterpart here), agent.disconnected, agent.error, transcript.item and conversation.agent.state.

client.tools.configured is deliberately not dispatched: the assistant chat API is request/response, so a tool announced on this path could never be invoked.

Connection loss

If the connection to the assistant drops, the widget shows its error state with a retry control rather than sitting on a loading spinner. Retrying reconnects and resumes the same conversation.

text-only vs chat-mode

| | chat-mode | text-only | | -------------------------- | ----------- | ----------- | | Microphone prompt | no | no | | Audio played | no | no | | Places a WebRTC call | yes | no | | Voice minutes / TTS billed | yes | no | | Image attachments | yes | no | | Client-side tools | yes | no | | Barge-in / interrupt | yes | n/a |

Use chat-mode when the visitor may switch to speaking, or when you need attachments or client-side tools. Use text-only when they never will.

Chat Mode

Enable text-only interaction (no microphone) with the chat-mode attribute. When enabled:

• No microphone permission is requested — the WebRTC peer negotiates audio as recvonly
• The conversation auto-starts immediately after the client connects (no "Start Call" button)
• The widget starts in the expanded state to show the transcript
• The audio visualizer, mute button, and <audio> element are hidden
• Agent audio tracks are disabled so no speech is played back
• The user interacts via sendConversationMessage() (text input in the transcript UI)

<telnyx-ai-agent agent-id="assistant-xxx" chat-mode></telnyx-ai-agent>

Toggling chat mode mid-conversation

The chat-mode attribute is a watched attribute — changing it at runtime triggers a full client teardown and recreation. Toggling from voice to chat mode (or vice versa) destroys the current WebRTC session and starts a new one:

• The active call is hung up
• The underlying WebRTC connection is disconnected
• A new TelnyxAIAgent client is created with the updated chatMode value
• A fresh connection is established and a new conversation begins

This means the transcript and conversation context are not preserved across a toggle. If you need to maintain conversation continuity, consider using the conversation-id attribute so the AI agent can resume context on the new session:

<telnyx-ai-agent
  agent-id="assistant-xxx"
  conversation-id="my-conversation-123"
></telnyx-ai-agent>

<script>
  const widget = document.querySelector('telnyx-ai-agent');

  // Toggle to chat mode — destroys the current session and starts fresh
  widget.setAttribute('chat-mode', '');

  // Toggle back to voice mode — another full teardown/reconnect
  widget.removeAttribute('chat-mode');
</script>

After a conversation ends in chat mode, the widget will not auto-restart even if chat-mode remains set. The user must interact again or you must remove and re-add the element.

Note that chat-mode still places a full WebRTC call underneath — the assistant's speech is synthesized and then discarded. If the visitor will only ever type, text-only avoids that entirely.

Region

Pin signalling to a specific region with the region attribute. When set, the WebRTC SDK routes the connection through <region>.rtc.telnyx.com instead of the default anycast rtc.telnyx.com. Useful when anycast DNS routes a client to a sub-optimal datacenter.

<telnyx-ai-agent agent-id="assistant-xxx" region="apac"></telnyx-ai-agent>

Known regions: us-east, us-central, us-west, ca-central, eu, apac, south-asia.

Environment

Select which Telnyx environment the widget connects to with the environment attribute. It is forwarded to the underlying @telnyx/webrtc SDK, which picks the WebRTC signalling host accordingly. Defaults to production.

| Value | WebRTC signalling host | When to use | | ---------------------- | ------------------------- | ----------------------------------------------------------------------------------------------- | | production (default) | wss://rtc.telnyx.com | Normal usage against Telnyx production. | | development | wss://rtcdev.telnyx.com | Testing an agent/backend deployed to the Telnyx dev environment (e.g. pre-release AI features). |

<telnyx-ai-agent
  agent-id="assistant-xxx"
  environment="development"
></telnyx-ai-agent>

development routes signalling to rtcdev.telnyx.com. Only use it when your assistant and the backend changes you are testing are deployed to Telnyx dev — production assistants are not reachable there. environment and region are independent: region only rewrites the production host and does not apply to development.

VAD (Voice Activity Detection) Options

Configure Voice Activity Detection for speech detection and latency measurement by passing a JSON object to the vad attribute:

<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"volumeThreshold": 10, "silenceDurationMs": 500, "minSpeechDurationMs": 100}'
></telnyx-ai-agent>

Available VAD options:

| Option | Type | Default | Description | | --------------------- | -------- | ----------- | --------------------------------------------------------------------------------------------------------------------------- | | volumeThreshold | number | 10 | Volume threshold (0-255) for detecting speech. Audio levels above this value are considered speech. | | silenceDurationMs | number | 1000 | Duration of silence (ms) before triggering "thinking" state. Lower values = faster response but may cut off natural pauses. | | minSpeechDurationMs | number | 100 | Minimum speech duration (ms) to count as real user speech. Filters out brief noise spikes. | | maxLatencyMs | number | undefined | Maximum latency (ms) to report. Values above this are considered stale and won't be reported. |

Example configurations:

<!-- Fast-paced conversation (aggressive turn detection) -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"silenceDurationMs": 500, "minSpeechDurationMs": 80}'
></telnyx-ai-agent>

<!-- Thoughtful conversation (tolerant of pauses) -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"silenceDurationMs": 1500, "minSpeechDurationMs": 150}'
></telnyx-ai-agent>

<!-- Noisy environment -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  vad='{"volumeThreshold": 20, "minSpeechDurationMs": 200}'
></telnyx-ai-agent>

Latency Display Options

Control whether latency measurements are included in transcript messages. Both options default to false.

| Attribute | Type | Default | Description | | ----------------------------- | --------- | ------- | ------------------------------------------------------------------------------------------------------ | | show-user-perceived-latency | boolean | false | Include userPerceivedLatencyMs in transcript items (time from user stop speaking to agent response). | | show-greeting-latency | boolean | false | Include greetingLatencyMs in transcript items (time for initial agent greeting). |

<!-- Enable latency tracking in transcript messages -->
<telnyx-ai-agent
  agent-id="assistant-xxx"
  show-user-perceived-latency
  show-greeting-latency
></telnyx-ai-agent>

When enabled, latency values are attached to assistant transcript items and can be accessed via the transcript.item event:

widget.addEventListener('transcript.item', function (event) {
  const { role, content, userPerceivedLatencyMs, greetingLatencyMs } =
    event.detail;
  if (role === 'assistant') {
    if (userPerceivedLatencyMs !== undefined) {
      console.log('Response latency:', userPerceivedLatencyMs, 'ms');
    }
    if (greetingLatencyMs !== undefined) {
      console.log('Greeting latency:', greetingLatencyMs, 'ms');
    }
  }
});

Events and Callbacks

The widget emits DOM CustomEvents that you can listen to for tracking analytics, updating UI, or integrating with your application logic. Event names match @telnyx/ai-agent-lib for consistency.

<script>
  document.addEventListener('DOMContentLoaded', function () {
    const widget = document.querySelector('telnyx-ai-agent');

    // Listen for call lifecycle events
    widget.addEventListener('conversation.update', function (event) {
      const { callState } = event.detail;
      if (callState === 'active') {
        console.log('Voice call started');
      }
      if (callState === 'destroy') {
        console.log('Voice call ended');
      }
    });

    widget.addEventListener('transcript.item', function (event) {
      console.log('Message received:', event.detail);
      // Process message, update state, etc.
    });

    widget.addEventListener('conversation.agent.state', function (event) {
      const { state, userPerceivedLatencyMs, thinkingStartedAt } = event.detail;
      console.log('Agent state:', state);
      if (userPerceivedLatencyMs) {
        console.log('Response latency:', userPerceivedLatencyMs, 'ms');
      }
    });

    widget.addEventListener('agent.error', function (event) {
      console.error('Widget error:', event.detail);
      // Handle errors, show fallback UI, etc.
    });
  });
</script>

Available Events

| Event | Description | Detail | | -------------------------- | ---------------------------------------------------- | -------------------------------------------------------- | | agent.connected | Agent connected to platform | - | | agent.disconnected | Agent disconnected from platform | - | | agent.error | Error occurred | { message, name } | | transcript.item | Transcript message received | { id, role, content, timestamp, attachments? } | | conversation.update | Conversation state updated | { type, callState } | | conversation.agent.state | Agent state changed (listening/speaking/thinking) | { state, userPerceivedLatencyMs?, thinkingStartedAt? } | | agent.audio.mute | Agent audio muted/unmuted | { muted } | | client.tool.invoked | A client-side tool was invoked by the agent | { callId, toolName } | | client.tool.completed | A client-side tool produced an output for the agent | { callId, toolName, isError } | | client.tool.error | A client-side tool failed; a safe error was returned | { callId, toolName, reason } |

Client-side Tools

The widget can execute client-side tools requested by the AI agent (for example, looking something up in the page, reading local state, or triggering a UI action) and return the result to the agent. This implements the PR-531 client_side_tool flow on top of @telnyx/ai-agent-lib.

Because tool handlers are JavaScript functions, they cannot be passed as HTML attributes. Instead, register them imperatively on the widget element:

<telnyx-ai-agent agent-id="assistant-xxx"></telnyx-ai-agent>

<script>
  const widget = document.querySelector('telnyx-ai-agent');

  // Register a handler. `args` is the parsed JSON arguments object the agent
  // sent; return any JSON-serializable value (or a string) as the result.
  widget.registerClientTool('get_cart_total', async (args, context) => {
    // context = { callId, toolName, rawArguments }
    return { total: 42.5, currency: 'USD' };
  });

  // Inspect or remove handlers later:
  widget.getClientTools(); // ['get_cart_total']
  widget.unregisterClientTool('get_cart_total'); // true

  // Observe execution via DOM events (payloads never include raw
  // arguments/outputs — only safe correlation fields):
  widget.addEventListener('client.tool.invoked', (e) =>
    console.log('tool invoked', e.detail),
  );
  widget.addEventListener('client.tool.completed', (e) =>
    console.log('tool completed', e.detail),
  );
  widget.addEventListener('client.tool.error', (e) =>
    console.warn('tool error', e.detail),
  );
</script>

Notes:

• Register handlers as early as you like — registrations are durable and are automatically re-applied across the widget's internal reconnects.
• The library always returns a function_call_output to the agent (a safe error payload on unknown tool, invalid arguments, handler throw, or timeout), so a missing or failing handler never hangs the conversation.
• Raw tool arguments and outputs are never logged.

Development

To develop the Telnyx Voice AI Widget, you can clone the repository and run the following commands:

yarn install --immutable
yarn dev

Linked library development

The library exports dist/index.js. After linking, rebuild the library whenever its source changes; linking alone does not compile it.

# In telnyx-voice-ai-lib
yarn build
npm link

# In telnyx-voice-ai-widget
npm link @telnyx/ai-agent-lib
yarn dev --force

Reload the browser after rebuilding. --force refreshes Vite's dependency cache after switching from an installed package to a link. The development config also excludes the library from dependency optimization so subsequent builds are loaded directly. Use either this link workflow or setup:local-library; that setup command replaces the link with a copied build.

Force an audio RTC gateway

Pass rtc-ip and rtc-port together to select the upstream voice gateway through VSP. These map to the SDK's rtcIp (lowercase p) and numeric rtcPort options, which become rtc_ip and rtc_port in the WebSocket URL. The browser still connects to VSP; these attributes do not replace its public hostname. They apply to audio and legacy chat-mode; text-only conversations do not use a voice gateway.

<telnyx-ai-agent
  agent-id="assistant-xxx"
  environment="production"
  rtc-ip="10.0.0.1"
  rtc-port="14938"
></telnyx-ai-agent>

Omit both attributes for normal gateway discovery. rtc-port must be a valid integer port number. Updating either attribute recreates the voice connection.

Text-only transcript, errors and retry

The typing-only widget renders the library's transcript.updated snapshots. Each DOM transcript.updated event has the complete TranscriptItem[] as its detail; replace the displayed array. It includes final corrections, empty replacements, user revisions and deletions. The existing DOM transcript.item event retains the actual library fragment payload. Do not append snapshots to a fragment-based transcript or consume both events as new messages.

widget.addEventListener('transcript.updated', ({ detail }) => {
  renderTranscript(detail); // Replace the displayed transcript.
});
widget.addEventListener('agent.error', ({ detail }) => {
  console.log(detail.scope, detail.reason, detail.message);
});

For text mode, agent.error detail includes message, name, and scope (connection, response, or send). Response errors also retain the backend reason, including stream_truncated, stream_failed, and assistant_error. These errors leave partial replies visible. Connection failures offer Reconnect inside the chat. Retry reuses the same agent, transcript and server-issued conversation ID; changing the assistant, environment, version, conversation ID, region, debug setting or URL starts a fresh session and clears its draft.

A send clears its draft only after proxy acceptance. A rejected send retains the draft with an error. Further sending is disabled until response.text.done or response.error, not merely until acceptance or the first fragment. Visitors can compose their next draft while a reply streams. Voice and the older chat-mode retain their existing call-based sending behavior.

Ending and starting a text chat

End chat closes the text WebSocket, including during login or an active reply, and shows a compact start button with the Telnyx logo, with no input or live connection. The chat stays ended until the visitor explicitly starts it again; changing attributes while ended does not reconnect it. Ending closes this widget session; it does not delete the server's conversation record. Saying goodbye in the transcript does not automatically end the connection.

Start new chat creates a new agent, clears the displayed transcript and draft, and requests a fresh server conversation ID. If the embed explicitly supplies conversation-id, the control instead says Start chat: the new agent respects that configured ID and resumes its server context with an empty local transcript. Remove or change that attribute to select a different conversation.

Reconnect after an unexpected network interruption still preserves the same agent, server conversation ID, transcript and draft. Intentional ending emits the normal final state/disconnected events once for an established session, without an unexpected-loss error; cancelling an uncompleted login does not announce a connected/disconnected pair.

Local text-only development

The published dependency pin remains unchanged until the library fixes have been released. This branch requires the updated text library API; a plain install of an older package is not sufficient. After installing dependencies in both sibling repositories, run these commands from the widget repository:

yarn install --frozen-lockfile
yarn setup:local-library
yarn test
yarn typecheck
yarn lint:check
yarn build

setup:local-library builds ../telnyx-voice-ai-lib and copies its built package into the widget's node_modules. It does not change either repository's source, the widget's dependency pin, or its lockfile. To use another checkout, pass its path: yarn setup:local-library ../another-library-checkout. Run setup again after a dependency install or library change. The copied package avoids bringing a second React/Preact dependency tree into the bundle. The predev, pretest and prebuild checks reject an installed library missing the required snapshot API. Before merge/release, publish the library fixes and update the widget's dependency pin and lockfile to that real released version. No publication is part of local setup.

To run the widget locally against VSP:

yarn dev --host 127.0.0.1 --port 5173 --strictPort

Open http://127.0.0.1:5173/. In index.html, set agent-id to the assistant being tested and enable text-only="true".

For a DEV run, use a public DEV assistant enabled for anonymous access, set environment="development", and omit version-id. The library uses wss://rtcdev.telnyx.com. That endpoint must run the VSP text-login, keepalive and stream-termination changes and be able to reach the private Assistants service. A successful WebSocket upgrade alone does not establish text-login readiness.

For a configured local or alternate DEV VSP, set text-chat-url to its WebSocket URL. This attribute applies only to text-only and overrides environment/region URL selection. Use an endpoint you control: chat content is sent to that URL. Use wss:// from an HTTPS embedding page; HTTP localhost development can use ws://. A local VSP still needs its private dependencies, TLS and internal-service configuration.