@konpro/widget

Embeddable real-time AI voice-avatar widget. Voice conversation with an avatar (WebRTC lipsync video, or an audio-only fallback), plus display tools, 360° panoramas, virtual tours, 3D maps, kiosk mode and QR document upload.

Framework-agnostic. For React, see @konpro/widget-react.

• Install
• Authentication
• Configuration — every option
• Instance methods
• Callbacks
• Avatar states
• Bubble mode
• Tools
• CSP & permissions
• Deployment notes
• Troubleshooting

Install

npm

npm install @konpro/widget

import { init } from '@konpro/widget';

const widget = init({
  sessionEndpoint: '/api/widget-session',
  containerId: 'konpro-avatar',
});

Ships ESM, CJS and TypeScript types. The module entry sets no globals and touches no DOM at import time, so it is safe to import under SSR (Next.js, Remix) — just call init() on the client.

Script tag (CDN)

<div id="konpro-avatar" style="width:400px;height:600px"></div>
<script src="https://cdn.konpro.ai/widget/index.min.js"></script>
<script>
  KonproWidget.init({
    sessionEndpoint: '/api/widget-session',
    containerId: 'konpro-avatar',
  });
</script>

KonAI still works. The widget was originally published under the global KonAI — a misnaming, since the product is Konpro. window.KonAI remains and points at the same object as window.KonproWidget, so existing embeds keep working unchanged. New integrations should use KonproWidget. The alias will only be removed at a future major version.

Check what's deployed with KonproWidget.version.

Authentication

The widget needs a short-lived session token to open its WebSocket. Choose exactly one method — passing none throws at init().

sessionEndpoint — recommended

Your backend mints the token, so your API key never reaches the browser.

init({ sessionEndpoint: '/api/widget-session' });

The widget POSTs { audioOnly } (plus agenticAvatarId if you set one) and expects the full session response back:

app.post('/api/widget-session', async (req, res) => {
  const r = await fetch('https://api.konpro.ai/v1/widget/sessions', {
    method: 'POST',
    headers: {
      'x-api-key': process.env.KONPRO_API_KEY,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      agentic_avatar_id: process.env.KONPRO_AVATAR_ID,
      settings: { max_duration: 3600, allowed_domains: ['yourdomain.com'] },
    }),
  });
  res.json((await r.json()).data);
});

Return the whole object — { sessionToken, sessionId, expiresAt, expiresIn, websocketUrl, userId, avatar, audioOnly }. In particular return expiresAt or expiresIn: the widget uses them to re-mint before the server's 30-minute TTL lapses. Without them it falls back to assuming the token is stale after 25 minutes, which is safe but less precise. Do not cache and re-serve the same session — mint one per request, or reconnects will fail once it expires.

getSessionToken — custom auth flows

init({
  getSessionToken: async () => {
    const res = await myClient.post('/create-avatar-session');
    return res.data; // same shape as above
  },
});

apiKey — development only

Puts your key in the browser where anyone can read it. Never ship this.

init({ apiKey: 'YOUR_KEY', agenticAvatarId: 'af1250af-...' });

agenticAvatarId is required with apiKey. With the other two methods your backend already picked the avatar, and the widget reads its ID from the session response — passing it in config is optional and ignored if it disagrees.

Configuration

Every option accepted by init({...}).

Authentication

| Option | Type | Default | Notes | |---|---|---|---| | apiKey | string | — | ⚠️ Development only. Requires agenticAvatarId. | | sessionEndpoint | string | — | Recommended. Your token-minting endpoint. | | getSessionToken | () => Promise<SessionData> | — | Custom auth flows. | | agenticAvatarId | string | — | Required with apiKey; optional otherwise. |

Layout

| Option | Type | Default | Notes | |---|---|---|---| | containerId | string | — | Element ID to mount inside. Widget fills it 100%×100%; you control size via that element's CSS. Overrides position. | | position | object | {position:'fixed', bottom:'20px', right:'20px', width:'400px', height:'600px'} | Floating-mode CSS. Used only when containerId and bubble are both omitted. | | bubble | boolean \| object | — | Bubble launcher mode. See Bubble mode. |

Video & audio

| Option | Type | Default | Notes | |---|---|---|---| | audioOnly | boolean | false | Disable lipsync + WebRTC video. Shows an animated circle with the avatar's image or initials. Sets the initial mode; users can change it between calls via showTopButtons.mediaMode. | | enableLipsync | boolean | true | Request server-side lipsync. Degrades gracefully if unavailable. | | enableWebRTC | boolean | true | Use WebRTC/MediaSoup for lowest-latency A/V. Falls back to audio-only on failure. | | usePlaceholderVideo | boolean | false | Audio-only mode only. Opt in to a full-frame idle-loop video plus a floating speaker pill, instead of the default avatar circle with the pulsing ring. | | ringtone | boolean \| string | true | Outgoing-call ringtone while connecting. false disables it; a string overrides the asset URL. The connecting overlay shows Calling <agent name> from the session payload. | | camera | boolean | follows Studio | Camera vision. Omit to follow the agent's Camera Vision toggle in Studio; true/false force it. See Camera vision. | | debug | boolean | false | Print the step-by-step trace to the console. See Debugging. | | videoFit | "contain" \| "cover" | "contain" | CSS object-fit for the avatar video. | | videoPosition | string | "top" | CSS object-position, e.g. "top", "center", "50% 20%". |

Conversation behaviour

| Option | Type | Default | Notes | |---|---|---|---| | allowInterruption | boolean | true | When false: no barge-in, interrupt button hidden, interruptAI() is a no-op, and text send is refused while the avatar speaks. |

Voice barge-in is unavailable in audio-only mode. Echo cancellation works by subtracting a reference signal — what the browser is rendering — from the mic input, and that reference comes from the WebRTC audio pipeline. Audio-only plays the avatar outside that pipeline, so its voice reaches the mic uncancelled and the session interrupts itself. The widget therefore sends silence while the avatar speaks in audio-only, and reports aecCapable: false so your server can disarm barge-in too. Push-to-talk, the interrupt button and text input all still interrupt — none of them depend on the mic. This also applies if WebRTC audio fails mid-call and playback falls back to the streaming player. | pushToTalk | boolean | false | Push-to-talk. Mic starts muted; a hold-to-talk button replaces the mute button. |

Integration keys

| Option | Type | Default | Notes | |---|---|---|---| | googleMapsApiKey | string | — | Required for the city display tool. Needs Maps JavaScript API + Map Tiles API enabled, with billing. |

Advanced

| Option | Type | Default | Notes | |---|---|---|---| | apiBaseUrl | string | "https://api.konpro.ai" | Override for self-hosted / staging backends. | | workletUrl | string | auto | Explicit URL for the mic-capture AudioWorklet. Only needed when your CSP forbids both data: and blob: script sources — serve @konpro/widget/dist/mic-capture.worklet.js yourself and point here. |

theme

theme: {
  primaryColor:   '#667eea',  // gradient start (header, avatar, call button)
  secondaryColor: '#764ba2',  // gradient end
  iconSize:       18,         // control-button icon px (mute renders at iconSize - 2)
}

| Key | Type | Default | |---|---|---| | primaryColor | string | #667eea | | secondaryColor | string | #764ba2 | | iconSize | number | 18 |

uiDetails

Controls what chrome is rendered. Pass false for a whole group to hide it.

| Key | Type | Default | Notes | |---|---|---|---| | showHeader | boolean | true | Top bar with avatar name + "Online". | | showStatus | boolean | true | Live / Listening / Speaking / Processing pill. | | showTranscript | boolean | true | Caption panel. | | showAvatar | boolean | true | The avatar video/image itself. | | showControls | boolean \| object | see below | Bottom control row. | | showTopButtons | boolean \| object | see below | Top button row. |

showControls — bottom row. false hides all; an object merges over the defaults, so you only specify overrides.

| Key | Default | Notes | |---|---|---| | call | true | Start/end call button. | | mute | true | Mic mute. Replaced by the PTT button when pushToTalk is on. | | interrupt | false | Explicit interrupt button. Force-hidden when allowInterruption: false. | | audioMute | false | Speaker mute (WebRTC only). |

showTopButtons — top row. false hides all; an object merges over the defaults.

| Key | Default | Notes | |---|---|---| | fullscreen | true | Fullscreen toggle. | | captions | true | Caption toggle. Hidden anyway if showTranscript: false. | | textInput | true | Text-input toggle. Only visible during an active call. | | inputMode | false | Opt-in dropdown to switch Voice ⇄ Push-to-talk at runtime. | | mediaMode | false | Opt-in dropdown to switch Video ⇄ Audio only. Shown only between calls — see below. |

Why mediaMode hides during a call. Audio-only is negotiated when the session connects: it travels in the WebSocket init payload and determines whether the server builds a video transport at all. There is no hidden video stream to reveal mid-call, so the mode is fixed for the lifetime of a session and the switcher hides while connected rather than sitting there disabled. A selection applies to the next call. The server has the final say — an agent configured audio-only will override the choice, and the switcher updates to show what actually happened.

init({
  sessionEndpoint: '/api/widget-session',
  uiDetails: {
    showHeader: false,
    showControls: { interrupt: true },   // other controls keep their defaults
    showTopButtons: { fullscreen: false },
  },
});

Recipes

Minimalist — just the avatar and a call button

init({
  sessionEndpoint: '/api/widget-session',
  uiDetails: {
    showHeader: false,
    showStatus: false,
    showTranscript: false,
    showTopButtons: false,
    showControls: { call: true, mute: false },
  },
});

Exhibition kiosk — portrait, push-to-talk, strict turn-taking

init({
  sessionEndpoint: '/api/widget-session',
  pushToTalk: true,
  allowInterruption: false,   // the avatar finishes every turn
  videoFit: 'cover',          // full-bleed portrait
  videoPosition: 'top',
  uiDetails: {
    showHeader: false,
    showStatus: false,
    showTopButtons: { fullscreen: true, captions: false, textInput: false },
  },
});

Full-featured support widget — every control, analytics wired up

init({
  sessionEndpoint: '/api/widget-session',
  theme: { primaryColor: '#0ea5e9', secondaryColor: '#6366f1' },
  uiDetails: {
    showControls: { call: true, mute: true, interrupt: true, audioMute: true },
    showTopButtons: { fullscreen: true, captions: true, textInput: true, inputMode: true },
  },
  onTranscription: (text) => analytics.track('user_said', { text }),
  onResponse: (text) => analytics.track('avatar_said', { text }),
});

Instance methods

init() returns the widget instance (or a bubble — see Bubble mode).

| Method | Notes | |---|---| | init() | Fetch session + mount. init(config) from the package calls it for you. | | destroy() | Tear everything down. Call before removing the container from the DOM. | | toggleCall() | Start / end the call. | | startCall() / endCall() | The individual halves. | | toggleMute() | Mute / unmute the mic. | | toggleAudioMute() | Mute / unmute speaker output (WebRTC only). | | toggleTextInput() | Show / hide the text-input bar. Requires an active call. | | toggleCaptions() | Show / hide the transcript at runtime. | | toggleFullscreen() | Enter / exit fullscreen. | | interruptAI() | Interrupt the current response. No-op when allowInterruption: false. | | sendTextMessage(text) | Send text as if typed. Returns boolean — false if refused (avatar speaking with interruption disabled, or socket down). | | setInputMode(mode) | 'vad' or 'ptt' at runtime. | | toggleCamera() | Camera on/off during a call. Turning it off releases the device. | | flipCamera() | Front ⇄ back camera. No-op unless a camera is live. | | setAudioOnly(bool) | Video ⇄ audio-only for the next call. Ignored with a console warning while connected or connecting. | | getAvatarState() | "idle" \| "listening" \| "thinking" \| "speaking". | | getModeInfo() | { audioOnly, usingLipsync, webrtcConnected, webrtcAudioEnabled, avatarState, isFirefox, version, ... }. | | getSessionInfo() | Session metadata from the ready event. | | isWebRTCConnected() | boolean. | | isAudioOnlyMode() | boolean. |

Callbacks

Passed in config.

| Callback | Fires | |---|---| | onReady(widget) | Initialisation finished — session fetched, UI mounted. | | onError(error) | Unrecoverable error (auth, session, socket). Always receives an Error. | | onTranscription(text) | The guest's speech was committed by STT. | | onResponse(text) | The avatar finished a complete response. | | onAvatarStateChange(state) | Any avatar state transition. | | onAudioOnlyChange(bool) | Video ⇄ audio-only changed between calls. Not fired for a server override on ready — read getModeInfo() after connecting. | | onBubbleOpen(widget) | Bubble expanded. Bubble mode only. | | onBubbleClose() | Bubble collapsed. Bubble mode only. |

Avatar states

| State | Meaning | |---|---| | idle | Session live, waiting. | | listening | Guest speech detected. | | thinking | Speech committed; the model is generating. | | speaking | Avatar audio/video is playing. |

Bubble mode

A circular launcher that expands into the widget. Clicking it opens a panel and hides the launcher; closing brings it back. The mic is never touched until the guest presses call.

const bubble = init({
  sessionEndpoint: '/api/widget-session',
  bubble: { label: 'Talk to us' },
});

| Option | Type | Default | Notes | |---|---|---|---| | prefetch | boolean | true | Build the widget and mint its session when the bubble mounts, not on first open. Lets the launcher show the real avatar image and makes opening instant. Costs one session per page view. | | imageUrl | string | avatar image | Launcher image. | | videoUrl | string | — | Looping launcher video (muted, inline). Beats imageUrl. | | size | number | 64 | Launcher diameter, px. | | panelWidth | number | 320 | Expanded panel width, px. | | panelHeight | number | 460 | Expanded panel height, px. | | side | "right" \| "left" | "right" | Which corner. | | offsetX | number | 24 | Distance from the docked side, px. | | offsetY | number | 24 | Distance from the bottom, px. | | label | string | — | Tooltip pill beside the collapsed launcher. | | defaultOpen | boolean | false | Expand on load. | | pulse | boolean | true | Attention ring. Respects prefers-reduced-motion. | | zIndex | number | 999999 | Stacking order. | | openLabel | string | "Open AI assistant" | Accessible label for the launcher. | | closeLabel | string | "Close" | Accessible label for the close button. |

Bubble instance API: expand(), collapse(), toggle(), isOpen, getWidget(), destroy().

getWidget() returns the inner widget. With prefetch: true it's available as soon as the bubble boots, including while collapsed. With prefetch: false the widget is created on first open and destroyed on close — so it's null between opens, and you must not cache it across a close.

Drive it from elsewhere on the page without holding a reference:

window.dispatchEvent(new CustomEvent('konai:bubble:open'));
window.dispatchEvent(new CustomEvent('konai:bubble:close'));

Escape also closes it. Under 480px wide the panel goes near-fullscreen.

Bubble mode applies a compact uiDetails preset (no header, status, transcript, fullscreen or captions — just the avatar and call/mute). Anything you pass in uiDetails merges over it.

Tools

Server-driven capabilities. You configure them on the avatar in Studio; the widget renders whatever the server pushes. Nothing here is client config.

| Tool | Trigger | Notes | |---|---|---| | Display | show_display | Images, videos, galleries and slideshows overlaid on the avatar. sequential: true plays a gallery as a timed slideshow; ttlMs caps how long a card stays up. | | 360° panorama | mediaType: 'panorama' | Equirectangular (2:1) images over HTTPS with CORS. Viewer lazy-loads on first use. | | Virtual tour | mediaType: 'tour' | Sandboxed iframe with a postMessage bridge. Sticky — only an explicit dismiss ends it. | | Google Map | mediaType: 'city' | 3D city + Street View. Requires googleMapsApiKey. | | Kiosk menu | kioskEnabled on the session | Domain-picker menu; the avatar greets in-domain. Two variants: hotel (default) and concierge (residential). Set server-side. | | CV / document upload | cv_upload_qr_show | QR card the guest scans to upload from their phone. |

Panorama and Maps are lazy-loaded — they add nothing to initial page weight if unused.

Camera vision

Lets the agent look through the guest's camera — "what am I holding?", "can you read this?". Turn on Camera Vision in the agent's Tools tab in Studio; the widget follows that setting, so there is nothing to configure.

The server tells the widget whether Camera Vision is on when the call connects (visionEnabled on the WebSocket ready message), so this works regardless of what your session endpoint returns. On the first call the camera therefore starts a moment after connecting rather than at the button press; later calls on the same page open it immediately. To override Studio:

init({ sessionEndpoint: '/api/widget-session', camera: false }); // never
init({ sessionEndpoint: '/api/widget-session', camera: true });  // always

When it's on, the widget asks for the camera when a call starts, shows a self-view tile bottom-left (captions move to its right), and adds a camera on/off button to the controls. A front/back switch appears on the tile when the device has more than one camera. A denied or missing camera is not an error — the call continues without it.

When the agent looks, the tile shows Looking… for about two seconds. If the guest has just finished speaking, the widget answers with the frame captured nearest that moment — what they were holding up while they asked — so there is no wait.

Privacy. Frames are never stored. The widget keeps at most the three most recent in memory, sampled every 500 ms and only while the page is visible, and drops them all when the call ends or the camera is turned off. Turning the camera off releases the device, so the OS camera indicator goes off too. Server-side, a frame exists only for the duration of one look.

Limits and safety (enforced by the server):

• 50 looks per call. Failed and speculative looks count against a separate ceiling of 100.
• Daily limits per guest and per organisation. A widget guest is never treated as the agent's owner, so the per-guest limit always applies.
• Printed names on cards and IDs are always blanked out. If an image is refused, the agent says calmly that it can't describe it rather than showing an error.
• Observations are not saved to agent memory unless the guest explicitly asks — and then only the object and what the guest said about it, never their appearance, documents or surroundings.

If the agent says it couldn't get a look, the frame was most likely rejected: the server drops frames that aren't real JPEG/PNG/WebP, exceed 1.5 MB, or exceed 4096 px on a side, and the look then times out. The widget sends JPEGs capped at 1024 px, so this should only happen with an unusual camera.

Debugging

The widget is quiet by default — it runs on other people's sites, so it only prints real errors and warnings that mean your setup needs fixing (a session payload with no expiry, a missing avatar.id, and similar). The full trace — session, connection, WebRTC timing, captions — is behind a debug switch:

• init({ debug: true })
• ?konpro_debug on the page URL
• localStorage.setItem("konpro:debug", "1") in the browser console, then reload

The last two need no change to the embed, so you can inspect a live site as it is deployed. Debug output includes what the guest said, so don't leave it on in production.

Content Security Policy & permissions

Baseline:

script-src   'self' https://cdn.konpro.ai;
style-src    'self' 'unsafe-inline';
connect-src  'self' wss://api.konpro.ai https://api.konpro.ai https://inference.konpro.ai;
img-src      'self' data: https:;
media-src    'self' blob: https://cdn.konpro.ai;

Feature-gated additions:

| Feature | Add | |---|---| | Panorama | script-src/connect-src: https://cdn.jsdelivr.net, plus your image hosts | | Virtual tour | frame-src https://studio.konpro.ai (or your tour host) | | Google Map | Google Maps origins in script-src, connect-src, img-src |

media-src needs the CDN origin for script-tag embeds. The connecting ringtone is an <audio> element loaded from wherever the bundle lives, and media-src governs that — connect-src does not apply. A wildcard such as https://*.cloudfront.net will not match cdn.konpro.ai, since that is a custom domain. Blocked, the symptom is a CSP media-src violation and a silent connect; the call itself is unaffected.

npm and bundler installs need nothing extra: the asset is emitted into your own build output, so 'self' already covers it. If you would rather not allow our origin, serve @konpro/widget/ringtone yourself and pass its URL as ringtone, or set ringtone: false.

If the avatar has an idle-loop video (usePlaceholderVideo, or a bubble videoUrl), add the host serving it as well — that is a CloudFront origin, not cdn.konpro.ai.

Microphone access is not CSP — it's Permissions Policy:

Permissions-Policy: microphone=(self), camera=(self), autoplay=(self), fullscreen=(self)

camera=(self) is only needed for agents with Camera Vision on. In an iframe you also need allow="microphone; camera; autoplay; fullscreen" — both the parent header and the iframe attribute must grant it.

The AudioWorklet. The CDN build loads it as a sibling of the bundle, so allowing cdn.konpro.ai covers it. On npm installs it depends on your bundler: Vite inlines it as a data: URI (under its 4 KB limit), webpack/Rollup usually emit a real asset served from 'self'. If either is blocked the widget falls back to a blob: worklet. If your CSP allows none of those, host @konpro/widget/dist/mic-capture.worklet.js yourself and set workletUrl.

Deployment notes

• HTTPS required for microphone access (localhost is exempt).
• PWA / standalone contexts are auto-detected; fullscreen switches to a CSS fallback where the native API is unreliable (Neat Frame, Android PWAs, iOS home-screen installs).
• Old Android WebViews — vendor-prefixed fullscreen and longhand inset fallbacks are included.
• Bundle size ~418 KB minified for the CDN build. Panorama (~800 KB) and Google Maps (~500 KB) load only on first use.

Troubleshooting

Mic never prompts. Serve over HTTPS. Check Permissions-Policy allows microphone — document.featurePolicy?.allowsFeature('microphone') returns false when it's blocked, and no prompt will ever appear. In an iframe, check allow="microphone".

AbortError: Unable to load a worklet's module / mic capture fails. Either your CSP blocks the worklet origin, or a self-hosted CDN is missing Access-Control-Allow-Origin (addModule() fetches in CORS mode, unlike a <script> tag — so the bundle loading proves nothing). Set workletUrl to a same-origin copy if your policy is strict.

Connection fails after idle. Your session endpoint is probably re-serving a cached session. Mint a fresh one per request, and return expiresAt/expiresIn so the widget can refresh before the 30-minute server TTL.

Avatar silent on iPad/iOS. iOS only lets an AudioContext start inside a user gesture. The widget creates its contexts inside the call-button click and retries on the next gesture if the window lapsed — but if you call startCall() programmatically (not from a real click), that guarantee is gone.

Text send does nothing. sendTextMessage() returns false when refused — which happens while the avatar is speaking if allowInterruption: false.

License

MIT