adeptly-live-widget

Embeddable AI voice guidance widget with screen sharing capabilities. Creates a floating Picture-in-Picture window that follows users across tabs and windows.

Installation

npm install adeptly-live-widget
# or
pnpm add adeptly-live-widget
# or
yarn add adeptly-live-widget

Required Dependencies

Make sure your project has these peer dependencies:

npm install react react-dom three

Environment Setup

1. WebSocket Server Required

This widget requires a running WebSocket server. You can either:

Option A: Use the Adeptly Live Server (Local Development)

# Clone the adeptly-live repo
git clone <adeptly-live-repo>
cd adeptly-live/server
npm install
npm run dev  # Runs on ws://localhost:8080

Option B: Use Production Server

wss://adeptly-live-production.up.railway.app

2. Environment Variables

Create a .env.local file in your project root:

# REQUIRED: WebSocket URL (must use ws:// or wss:// protocol)

# For local development:
NEXT_PUBLIC_WS_URL=ws://localhost:8080

# For production:
NEXT_PUBLIC_WS_URL=wss://your-server.railway.app

⚠️ IMPORTANT:

• Use ws:// for local development (HTTP)
• Use wss:// for production (HTTPS)
• Do NOT use http:// or https:// - WebSockets require ws:// or wss://

Usage

Basic Integration

'use client';  // Required for Next.js App Router

import { useState } from 'react';
import { AdeptlyWidget } from 'adeptly-live-widget';

export default function MyPage() {
  const [isActive, setIsActive] = useState(false);

  return (
    <>
      <button onClick={() => setIsActive(true)}>
        Launch Live Demo
      </button>
      
      <AdeptlyWidget 
        wsUrl={process.env.NEXT_PUBLIC_WS_URL || 'ws://localhost:8080'}
        autoStart={isActive}
        onSessionStart={() => console.log('Session started')}
        onSessionEnd={() => setIsActive(false)}
        onError={(error) => {
          console.error('Widget error:', error);
          setIsActive(false);
        }}
      />
    </>
  );
}

With Custom Assets

<AdeptlyWidget 
  wsUrl={process.env.NEXT_PUBLIC_WS_URL!}
  autoStart={isActive}
  logoPath="/custom-logo.png"
  exrPath="/custom-environment.exr"  // Optional 3D environment map
  onSessionEnd={() => setIsActive(false)}
/>

With PostHog Analytics

Track widget usage in your own PostHog instance:

import { AdeptlyWidget, EVENTS, PROPERTIES } from 'adeptly-live-widget';

<AdeptlyWidget 
  wsUrl={process.env.NEXT_PUBLIC_WS_URL!}
  autoStart={isActive}
  posthog={{
    apiKey: process.env.NEXT_PUBLIC_POSTHOG_KEY!,
    apiHost: 'https://us.i.posthog.com', // Optional, defaults to US
    source: 'my-app-name' // Optional, identifies your implementation
  }}
  onSessionEnd={() => setIsActive(false)}
/>

Events Tracked:

• session_started / session_ended / session_error
• widget_activated / widget_deactivated
• websocket_connected / websocket_disconnected / websocket_error
• screen_share_started / screen_share_ended
• pip_opened / pip_closed
• conversation_phase_changed
• media_permission_granted / media_permission_denied

Properties Included:

• session_id - Unique identifier for each session
• source - Your app name (from config)
• ws_url - WebSocket server URL
• timestamp - Event timestamp
• current_status - AI status (idle/listening/thinking/speaking)

All events are automatically enriched with session context.

Props

| Prop | Type | Required | Description | |------|------|----------|-------------| | wsUrl | string | ✅ Yes | WebSocket server URL (must start with ws:// or wss://) | | autoStart | boolean | No | Auto-trigger screen sharing when true | | position | 'bottom-right' \| 'bottom-left' | No | Widget position (currently not used with PiP) | | onSessionStart | () => void | No | Callback when session starts | | onSessionEnd | () => void | No | Callback when session ends | | onError | (error: string \| null) => void | No | Callback for errors (null to clear) | | logoPath | string | No | Custom logo path for PiP window | | exrPath | string | No | Custom EXR environment map for 3D visualization | | posthog | PostHogConfig | No | PostHog analytics config (optional) |

PostHogConfig Interface

interface PostHogConfig {
  apiKey: string;      // Your PostHog API key
  apiHost?: string;    // PostHog API host (defaults to 'https://us.i.posthog.com')
  source?: string;     // Source identifier (e.g., 'my-company-app')
}

How It Works

1. User clicks button → Triggers setIsActive(true)
2. Widget auto-starts → Requests microphone and screen sharing permissions
3. PiP window opens → Floating window with 3D animated orb
4. WebSocket connects → Streams audio (16kHz) and video (1fps JPEG)
5. AI responds → Voice guidance plays through PiP window
6. Window follows user → PiP stays visible across tabs/windows

Features

• 🎤 Real-time voice conversation - Bidirectional audio streaming
• 🖥️ Screen sharing - AI can see what you're doing
• 🎨 Animated 3D orb - Real-time audio visualization with Three.js
• 📊 Live status indicators - Ready/Listening/Thinking/Speaking
• 💬 Real-time transcription - Shows user and AI messages
• 🪟 Picture-in-Picture - Floating window that follows you everywhere

Browser Compatibility

Required

• Chrome/Edge - Document Picture-in-Picture API
• Modern browsers - getUserMedia, getDisplayMedia APIs

Check Browser Support

if ('documentPictureInPicture' in window) {
  // PiP supported ✅
} else {
  // Show fallback UI ❌
}

Troubleshooting

WebSocket Connection Fails

Problem: WebSocket connection to 'ws://localhost:8080/' failed

Solutions:

1. Check server is running: lsof -i:8080
2. Verify .env.local has correct URL with ws:// protocol
3. Restart your dev server to pick up env changes
4. Check browser console for CORS errors

Widget Doesn't Appear

Problem: Button clicks but nothing happens

Solutions:

1. Check browser console for errors
2. Verify browser supports Document PiP (Chrome/Edge only)
3. Ensure autoStart prop is being set to true
4. Check WebSocket connection is established first

Permission Denied Errors

Problem: Microphone/screen sharing permission denied

Solutions:

1. Must use HTTPS in production (or localhost for dev)
2. User must click button (can't auto-start without user gesture)
3. Check browser permission settings

Environment Variable Not Working

Problem: Widget uses default ws://localhost:8080 instead of your URL

Solutions:

1. File must be named .env.local (not .env)
2. Variable must start with NEXT_PUBLIC_ for client-side access
3. Restart dev server after creating/editing .env.local
4. Use process.env.NEXT_PUBLIC_WS_URL not process.env.WS_URL

Quick Start Checklist

• [ ] Install package: pnpm add @adeptly/live-widget
• [ ] Install dependencies: pnpm add three
• [ ] Create .env.local with NEXT_PUBLIC_WS_URL=ws://localhost:8080
• [ ] Start WebSocket server on port 8080
• [ ] Import and add <AdeptlyWidget> to your component
• [ ] Test button triggers autoStart={true}
• [ ] Grant microphone + screen sharing permissions
• [ ] Verify PiP window appears

Direct PiP Link Pattern

Want a shareable link that immediately launches the PiP experience? Create a dedicated route:

// app/pip/page.tsx
'use client';

import { useState, useEffect } from 'react';
import { AdeptlyWidget } from 'adeptly-live-widget';

export default function PiPPage() {
  const [isActive, setIsActive] = useState(false);

  useEffect(() => {
    // Auto-activate on first click anywhere
    const handleClick = () => {
      setIsActive(true);
      document.removeEventListener('click', handleClick);
    };
    document.addEventListener('click', handleClick);
    return () => document.removeEventListener('click', handleClick);
  }, []);

  if (isActive) {
    return (
      <>
        <div className="min-h-screen flex items-center justify-center">
          <h1>Launching PiP Experience...</h1>
        </div>
        <AdeptlyWidget
          wsUrl={process.env.NEXT_PUBLIC_WS_URL!}
          autoStart={true}
          onSessionEnd={() => setIsActive(false)}
        />
      </>
    );
  }

  return (
    <div onClick={() => setIsActive(true)} className="min-h-screen cursor-pointer">
      <h1>Click to Launch PiP</h1>
    </div>
  );
}

Now you can share: https://your-site.com/pip

Users click the link → Click anywhere → PiP launches immediately!

Use cases:

• Add to Canva presentations
• Embed in Google Slides
• Share via email or social media
• Include in documentation
• Generate QR codes

Example Projects

See integration examples:

• Direct PiP Link: /adeptly-lp/app/pip/page.tsx
• Enterprise LP: /adeptly-enterprise-lp/app/page.tsx
• Original App: /adeptly-live/client/app/page.tsx

Support

For issues:

1. Check browser console for [WebSocket], [MediaCapture], or [PiP] errors
2. Verify all environment variables are correct
3. Ensure WebSocket server is running and accessible
4. Test in Chrome/Edge (PiP requirement)

License

MIT