Bridge AI Chat Widget

Embeddable React + TypeScript chat widget for Bridge AI agents, with:

• text chat
• optional realtime voice mode backed by the Lola Socket.IO proxy and xAI
• optional animated 3D avatar (three.js)
• WebSocket live message updates (AnyCable)
• Shadow DOM isolation for host-page style safety

What This Project Is

bridgeapp-ai-chat-widget is an ESM library bundle that host applications instantiate via:

• new ChatWidgetInstance(config)

The instance mounts a React app into a host element's Shadow DOM, fetches static asset mappings from a remote manifest, initializes chat/voice/avatar modules, and exposes a small integration API (setOpen, destroy, setCustomerInteractionSession, events).

Tech Stack

• React 19
• TypeScript
• Vite (dev app + library build)
• Tailwind CSS 4
• Zustand (state stores)
• three.js + FBX assets (avatar rendering/animation)
• AnyCable Web client (live chat transport)
• Socket.IO + AudioWorklet (realtime voice mode through Lola Backend)

Repository Layout

High-level structure:

• src/index.ts - public library entry (ChatWidgetInstance)
• src/index.types.ts - public config/types surface
• src/App.tsx - provider composition root
• src/react/mount.tsx - React root mount helper
• src/ui/components - UI components (chat widget, routes, controls)
• src/domain/message - message store + websocket channel integration
• src/domain/voice - Lola voice session, PCM media, and xAI realtime events
• src/avatar - three.js avatar scene, animation manager, blend shapes
• src/api - HTTP clients for messages/voice/session-related calls
• src/context/ChatWidgetContext.tsx - widget-level UI state/context
• src/dev - local/dev/demo stand bootstrap (non-library integration path)
• src/assets - local assets used by the widget and avatar (not used directly, rather this whole folder is getting pushed to assets repo)
• scripts - utility scripts (for example asset sync to remote static repo)

Runtime Architecture

1) Host Integration Layer

ChatWidgetInstance (src/index.ts) is the host-facing API:

• validates mountElementId
• creates shadowRoot on mount element
• injects bundled CSS into shadow root
• fetches ${ASSETS_URL}/manifest.json
• resolves hashed asset paths via resolveAssetPath(path)
• optionally merges remote config (remoteConfig)
• mounts React app and wires AppApi callbacks/events

2) React App + Providers

App composes:

• ChatWidgetContextProvider (widget open state, sizing, config, loading progress)
• CableProvider (AnyCable websocket service + network status)
• HashRouter (voice/text/list routes)

3) Messaging Domain

useMessageStore handles:

• initial thread bootstrap (ListMessages, ListMessagesForThreads)
• optimistic and server-driven message updates
• thread map and per-thread message derivation
• message send (CreateMessage)
• agentic steps merge/update and rendering support

Live updates arrive via AnyCable channel events and are pushed into the store. Versioned messageDelta updates are applied by messageId and increasing sequence; each frame contains the current accumulated text, so duplicate or stale frames cannot append content twice. The UI renders every accepted frame immediately and animates only the newly arrived suffix. See docs/chat-streaming.ru.md for the wire contract, lifecycle, rendering, and scroll behavior.

4) Voice Domain

useVoiceStore manages the Lola voice turn lifecycle:

• creates a canonical POST /voice/sessions session
• captures mono microphone input and converts it to PCM16 24 kHz in an AudioWorklet
• connects to the authenticated /assistant Socket.IO namespace, activates the provider session with voice.connect, and sends sequenced Lola protocol v4 voice.audio.append events with acknowledgements
• plays response.output_audio.delta through a continuous AudioWorklet queue
• unwraps canonical voice.event envelopes, renders Grok realtime transcripts, and keeps avatar talking state in sync
• creates a fresh VoiceSession after an unexpected voice socket disconnect
• closes the session through voice.disconnect with the REST endpoint as a fallback

5) Avatar Domain

Avatar + AnimationManager:

• initializes three.js renderer/camera/lights/materials
• loads model + texture + animation assets from resolved remote paths
• emits loading_progress
• starts animation mixer loops
• maps viseme timeline to morph target animation during speech
• exposes external animation actions (excited, fix_hair, spin, kiss)

When all assets load, widget emits avatar_ready through ChatWidgetInstance.

Public API

Import:

import {
  ChatWidgetInstance,
  type WidgetConfig,
} from "bridgeapp-ai-chat-widget";

ExternalWidgetConfig

• mountElementId: string
• • configName: string - remote config, lives at the same place as other static assets
• customerInteractionSession?: { accessToken; chatId; expiresAt } - if there's no session (separate case for dev/demo mode) - we look for customerId in remote config and create session ourselves.

RemoteWidgetConfig

Actual widget config

• agentId: string
• customerId?: string
• wsUrl: string
• apiUrl: string
• AIAvatar?: boolean - is 3D avatar be used
• voiceEnabled?: boolean - is voice mode enebled
• voiceConfig?: { voice: "ara" | "eve" | "leo" | "rex" | "sal" } - optional xAI voice
• background?: boolean - clear or visible background
• footer?: boolean - is footer rendered
• debug?: boolean (avatar debug GUI)
• manualOpen?: boolean - is widget trigger rendered or should it be open only programatically

Instance Methods

• setOpen(open: boolean): void - open/close widget
• setCustomerInteractionSession(session): void - update session after initialization (on session expire)
• playAnimation(name): void
• destroy(): void

Events

• avatar_ready - emitted once avatar assets are fully loaded
• renew_session - emitted when session is expired or near expiry

Host Usage Example

import { ChatWidgetInstance } from "bridgeapp-ai-chat-widget";

const widget = new ChatWidgetInstance({
  mountElementId: "chat-widget-root",
  customerInteractionSession: CustomerInteractionSession,
  AIAvatar: true,
  background: false,
  voiceEnabled: true,
  footer: false,
  manualOpen: true,
});

widget.addEventListener("avatar_ready", () => {
  widget.setOpen(true);
});

widget.addEventListener("renew_session", async () => {
  const newSession = await fetchNewSessionSomehow();
  widget.setCustomerInteractionSession(newSession);
});

Teardown:

widget.destroy();

Local Development

Install:

npm install

Start dev server:

npm run dev

Notes:

• Vite dev server runs on 5179 by default.
• src/dev/main.tsx is the local bootstrap path (creates session and mounts widget directly).
• vite.config.ts is for app/dev runtime.
• vite.config.lib.ts is for distributable library build.

Build, Publish, and Deployment

Build local dev app

npm run build-dev

Build library artifacts

npm run build-lib

Outputs include:

• dist/bridge-ai-chat-widget.es.js
• dist/index.types.d.ts

Publish flow

• prepublishOnly runs build-lib
• package exports are defined in package.json under "exports"

Other project scripts

The demo app reads its service credential from VITE_SERVICE_ACCOUNT_API_KEY; credentials must not be committed to source or static config files.

• npm run lint
• npm run lint-fix
• npm run format
• npm run format:check
• deployment scripts for S3 buckets (project-specific, to be reworked)
• npm run push-assets-metamediastatic (sync src/assets into remote static assets repo)

Session Handling

Widget relies on customerInteractionSession:

• used as Bearer token for HTTP APIs
• used in websocket query for AnyCable auth
• periodically checked by useSessionExpiryChecks

When session is near expiry or invalid, renew_session event is emitted; host app must provide a fresh session via setCustomerInteractionSession.

Lola Backend conversations

The widget uses Lola's canonical conversation API. Interaction chatId remains the authenticated session identifier; chat history is keyed independently by conversationId (threadId in realtime compatibility messages).

• GET /chat/conversations loads the open chat list.
• GET /chat/conversations/:conversationId/messages loads one history.
• POST /chat/conversations creates and selects a new chat.
• POST /chat/conversations/:conversationId/select selects an existing chat for the current interaction session.
• POST /chat/messages sends conversationId and adopts the canonical conversationId returned by the backend when a chat is resolved or created.

A 409 response means that the selected conversation was closed. It is shown as a chat error and the open conversation list is refreshed; only 401 requests trigger interaction-session renewal.

Lola Backend command runtime

The widget consumes Lola Backend command.created envelopes from the AnyCable commandCreated/uiCommand frame, acknowledges received and a final result, and deduplicates commandId during reconnect replay. Updating the interaction session now reconnects REST and AnyCable without remounting the widget.

The complete frontend integration and rollout guide is available in docs/ui-actions-sdk.ru.md.

The previous callback fields (url, target, commandType, targetDefinition, and modalId) remain as deprecated aliases for a zero-downtime host migration.

Canonical page routes and modal names arrive in data.target; the host only provides typed callbacks:

const widget = new ChatWidgetInstance({
  // Existing widget options...
  customerInteractionSession: {
    accessToken,
    chatId: sessionId,
    sessionId,
    expiresAt,
  },
  handlers: {
    openModal: ({ modalName }) => modalRegistry.open(modalName),
    openPage: ({ route }) => router.push(route),
  },
  // Temporary fallback for legacy commands without data.target.
  uiTargets: {
    deposit_button: {
      kind: "button",
      selector: '[data-lola-action="deposit"]',
    },
    deposit_modal: {
      kind: "modal",
      modalName: "deposit",
    },
    account_page: {
      kind: "page",
      route: "/account",
      query: { source: "lola" },
    },
  },
});

Targets and handlers can also be registered after construction with registerUiTarget(code, target) and registerCommandHandler(name, handler). Both return an unregister callback.

Built-in handlers cover assistant/chat visibility, supported avatar animations, CTA rendering inside the text chat, and element highlighting. Highlight targets fall back to [data-lola="code"] or [data-lola-action="code"]. The default highlight is a fixed overlay attached to document.body, so it is not clipped by an element's overflow or container bounds and follows resize and scroll. A registered page URL fallback uses history.pushState and emits popstate, so an explicitly configured query such as ?lola_modal=deposit can be handled without reloading the host SPA. A registered modal selector can also open a native HTMLDialogElement. data.target is authoritative. Legacy commands without it are resolved through payload.modalId/pageId and uiTargets. If no safe handler/fallback exists, the widget sends unsupported and emits command_unsupported; other lifecycle events are command_received, command_succeeded, command_failed, and command_expired.

Asset Pipeline

Runtime asset resolution:

1. fetch remote manifest from https://metamediastatic.com/manifest.json
2. map logical path (for example animations/idle_1.fbx) to hashed URL
3. load via resolveAssetPath

This allows cache-friendly hashed asset delivery without changing code references.

Troubleshooting

• Widget does not mount:
  • verify mount element exists and mountElementId is correct
• No messages or send failures:
  • verify apiUrl, accessToken, and chatId
• No live updates:
  • verify wsUrl, websocket reachability, and token validity
• Voice fails to start:
  • verify mic permission, Lola apiUrl, Socket.IO reachability, and backend Project.settings.voiceEnabled
• Avatar never becomes ready:
  • verify manifest URL + asset availability under static host