@coreexl/mentor

React UI for real-time AI mentors — chat, voice, and avatar video — powered by LiveKit Agents.

Drop in a <MentorPage>, point it at your token server, and you get a polished, fully themeable mentor interface. The package also ships a typed client + React hook for managing avatars (create / list / delete / live status) on a scxl-avtr avatar server.

npm install @coreexl/mentor
# or: pnpm add @coreexl/mentor

react and react-dom (^18.3 or ^19) are peer dependencies.

Contents

• Features
• How it works
• Styling
• Theming
• Backend requirements
• Quick start
• <MentorPage>
  • Props
  • Modes
  • Driving the connection yourself
• <AvatarPanel>
• Avatar management
  • Authenticating
  • useAvatars hook
  • Functional client
  • Types
• TTS providers
• Language support
• Overlay mode
• Tool calling
• Environment variables
• Exports
• Troubleshooting

Features

• One component, three modes — text chat, live voice, or a talking-head avatar.
• Avatar video via a scxl-avtr server (real-time lip-sync over LiveKit WebRTC).
• Camera vision & mic mute — publish the user's camera for the agent to see; mute/unmute the mic.
• Avatar management — a hook + client to create, list, delete, and watch avatars, with live status over WebSocket or polling.
• Two TTS engines — Google Chirp3 HD (50+ languages) or Chatterbox Turbo (voice cloning).
• Tool calling — forward OpenAI-format tools to the agent and handle calls in the browser.
• Fully themeable — scoped CSS variables; self-contained stylesheet with no Preflight reset.

How it works

Your React app  ──(accessKey, config)──▶  Your token server  ──dispatches──▶  LiveKit Agent
   <MentorPage>  ◀──────── token, url ────────┘                                    │
        │                                                                          │
        └────────────── LiveKit room (audio / video / data) ──────────────────────┘

<MentorPage> never holds your LLM/TTS/avatar credentials. It calls your token server (serverUrl) with the accessKey, gets a LiveKit token, joins the room, and the agent (server-side) runs the model, TTS, and avatar. The avatar-management API is a separate, optional surface that talks to the avatar server directly (see Avatar management).

Styling

The component's own scoped styles inject automatically on mount. To also get the layout (it uses Tailwind utilities under the hood), pick one option:

// A) Simplest — import the self-contained stylesheet. Ships utilities + theme tokens,
//    with NO Preflight, so it won't reset your app's base styles. Works with or
//    without Tailwind in your project.
import "@coreexl/mentor/mentor.css";

/* B) Already on Tailwind v4? Let your own build generate the classes instead: */
@source "../node_modules/@coreexl/mentor/dist/**/*.{js,mjs,cjs}";

Pick A or B, not both. Option A is the right default for non-Tailwind apps and the lowest-friction choice everywhere else.

Next.js

<MentorPage> is a client component. In the App Router, render it from a client component or load it dynamically:

"use client";
import dynamic from "next/dynamic";

const MentorPage = dynamic(
  () => import("@coreexl/mentor").then((m) => m.MentorPage),
  { ssr: false },
);

Theming

Everything is themeable via CSS variables scoped to .mentor-root — override any of them on that element (or a wrapper) and nothing leaks to the rest of your app:

| Variable | Default | Controls | |----------|---------|----------| | --mentor-primary | #775ff2 | Accent / gradient end | | --mentor-text | #1e1b2e | Default text color | | --mentor-bg | #ffffff | Root background | | --mentor-panel-bg | #ffffff | Panel / card background | | --mentor-panel-border | none | Outer panel border | | --mentor-panel-shadow | 0 4px 12px rgb(0 0 0 / .08) | Panel shadow | | --mentor-border | rgba(119,95,242,.15) | Inner borders / dividers | | --mentor-radius | 0.625rem | Corner radius | | --mentor-font | system sans | Font family |

.mentor-root { --mentor-primary: #e11d48; --mentor-radius: 0.75rem; }

The styles ship without Preflight (no CSS reset), so importing mentor.css won't restyle the rest of your page. Top-level components also accept className / style for per-instance overrides.

Backend requirements

You provide a token server at serverUrl. <MentorPage> POSTs to POST {serverUrl}/api/start with the session config and expects a LiveKit token back:

Request body (sent by the package):

{
  "access_key": "…",            // your accessKey prop
  "system_prompt": "…",
  "agent_name": "Coach Sarah",
  "video_mode": true,            // avatar mode
  "camera_mode": false,
  "avatar_id": "imogen",
  "avatar_service": "scxl-avtr",
  "language": "en-US",
  "gender": "female",
  "voice_name": "Aoede",
  "tts_provider": "google",
  "chatterbox_voice": null,
  "tools": [],
  "user_id": "…"
}

Response: { "token": "<livekit-jwt>", "url": "wss://<livekit-host>" }

Your server validates access_key, creates/configures the LiveKit room (e.g. stores this config as room metadata), dispatches the agent, and returns the token. The agent reads the metadata to pick the model, voice, and avatar.

Quick start

"use client";
import { MentorPage } from "@coreexl/mentor";
import "@coreexl/mentor/mentor.css";

export default function Tutor() {
  return (
    <MentorPage
      accessKey={process.env.NEXT_PUBLIC_ACCESS_KEY!}
      serverUrl="https://your-token-server.com"
      agentName="Coach Sarah"
      systemPrompt="You are a warm, encouraging tutor."
      language="en-US"
      gender="female"
    />
  );
}

<MentorPage>

The all-in-one interface: a chat panel with optional voice and avatar video.

<MentorPage> props

Required

| Prop | Type | Description | |------|------|-------------| | accessKey | string | Key your token server validates. |

Connection

| Prop | Type | Default | Description | |------|------|---------|-------------| | serverUrl | string | env NEXT_PUBLIC_COREEXL_MENTOR_SERVER | Token-server base URL. | | userId | string | auto | Stable user identity. | | debug | boolean | false | Verbose console logging. |

Agent & persona

| Prop | Type | Default | Description | |------|------|---------|-------------| | systemPrompt | string | — | Agent personality / instructions. | | agentName | string | "Coach Timothy" | Display name. | | agentTitle | string | "Concept Coach" | Subtitle. | | tools | Tool[] | — | OpenAI-format tool definitions forwarded to the agent. | | onToolCall | OnToolCallHandler | — | Invoked when the agent calls a tool. |

Voice (TTS / STT)

| Prop | Type | Default | Description | |------|------|---------|-------------| | language | string | "en-IN" | BCP-47 code for TTS + STT. | | gender | "female" \| "male" | "female" | Voice gender (auto-selects a voice). | | voiceName | string | — | Pin a specific Google Chirp3-HD voice (e.g. "Aoede"). | | ttsProvider | "google" \| "chatterbox" | "google" | TTS engine. | | chatterboxVoice | string | — | Named voice on the Chatterbox server. | | micEnabled | boolean | true | User microphone live; set false to mute. |

Avatar (video)

| Prop | Type | Default | Description | |------|------|---------|-------------| | mode | "chat" \| "avatar" | "chat" | UI mode. | | videoMode | boolean | false | Enable avatar video (use with mode="avatar"). | | avatarId | string | "imogen" | Avatar to drive (must exist on the avatar server). | | avatarService | string | "scxl-avtr" | Avatar backend. | | avatarSrc | string | — | Preview image shown (blurred) before connecting. | | cameraEnabled | boolean | false | Publish the user's camera for agent vision. | | showFps | boolean | false | Show a small FPS readout over the avatar. | | showTranscript | boolean | false | Live caption overlay on the avatar. |

Layout & lifecycle

| Prop | Type | Default | Description | |------|------|---------|-------------| | overlay | boolean | false | Render as a floating, minimizable widget. | | presentContent | ReactNode | — | Content for the Present tab. | | styles | MentorPageStyles | — | Per-slot className / style overrides. | | onConnect / onDisconnect | () => void | — | Connection lifecycle callbacks. | | onConnectionChange | (connected: boolean) => void | — | Fires on every connect ⇄ disconnect. | | onMessage | (m: MentorMessage) => void | — | Fires on each user message. | | controls | MutableRefObject<MentorControls \| null> | — | Populated with { connect, disconnect } — drive the session from your own UI. |

Modes

// Text chat
<MentorPage accessKey="..." mode="chat" />

// Talking-head avatar
<MentorPage accessKey="..." mode="avatar" videoMode avatarId="imogen" avatarSrc="/imogen.jpg" />

Driving the connection yourself

<MentorPage> owns connect/disconnect internally (a Connect button shows in the avatar view). To add your own controls — e.g. an End button in a custom dock — pass a controls ref and track connection state with onConnectionChange:

import { useRef, useState } from "react";
import { MentorPage, type MentorControls } from "@coreexl/mentor";

function Studio() {
  const controls = useRef<MentorControls | null>(null);
  const [connected, setConnected] = useState(false);

  return (
    <>
      <MentorPage
        accessKey="..." mode="avatar" videoMode avatarId="imogen"
        controls={controls}
        onConnectionChange={setConnected}
      />
      {connected
        ? <button onClick={() => controls.current?.disconnect()}>End</button>
        : <button onClick={() => controls.current?.connect()}>Start</button>}
    </>
  );
}

<AvatarPanel> — standalone avatar

Just the avatar video frame + connect button (no chat).

import { AvatarPanel } from "@coreexl/mentor";

<AvatarPanel
  accessKey={process.env.NEXT_PUBLIC_ACCESS_KEY!}
  serverUrl="https://your-token-server.com"
  avatarId="imogen"
  language="en-US"
  gender="female"
  width={340}
  height={512}
  borderRadius={20}
/>

| Prop | Type | Default | Description | |------|------|---------|-------------| | accessKey | string | required | Token-server key. | | serverUrl | string | env | Token-server URL. | | avatarId | string | "imogen" | Avatar to drive. | | avatarService | string | "scxl-avtr" | Avatar backend. | | avatarPreviewUrl | string | — | Image shown blurred before connecting. | | language / gender / voiceName | | | Same as <MentorPage>. | | ttsProvider / chatterboxVoice | | | Same as <MentorPage>. | | systemPrompt | string | — | Agent instructions. | | cameraMode | boolean | false | Show the camera toggle. | | width / height | string \| number | "100%" | Frame size. | | borderRadius | string \| number | 16 | Frame corner radius. | | resizable | boolean | false | Drag-to-resize handle. |

Avatar management

A typed client + React hook for the scxl-avtr avatar server. Use it to build your own avatar gallery / uploader: create avatars from a photo (with aspect + framing), list and delete them, and watch live generation status as the server renders each one — all without hand-rolling fetch calls or WebSocket plumbing.

This is separate from <MentorPage>: <MentorPage> drives an existing avatar (server-side, via your token server), while this API manages the avatars themselves and talks to the avatar server directly.

Authenticating

The avatar server is protected by a single shared API key. Every call here takes an AvatarApiConfig:

interface AvatarApiConfig {
  baseUrl: string;     // avatar server origin, or your proxy path
  apiKey?: string;     // the shared key (see below)
  fetchImpl?: typeof fetch;  // override for SSR / tests
}

There are two ways to supply the key — both are first-class:

1. Pass apiKey directly. It's attached to every request as the X-API-Key header (REST) and as ?key=… on the status WebSocket. Use this server-side, in internal/admin tools, or anywhere exposing the key to the client is acceptable.

const cfg = { baseUrl: "https://avatar.example.com", apiKey: "scxl_live_…" };
await listAvatars(cfg);

2. Use a server-side proxy (recommended for public web apps). Point baseUrl at your own route (e.g. /api/avatars) that injects the key before forwarding, so the browser never sees it. Then omit apiKey:

const cfg = { baseUrl: "/api/avatars" };   // your Next/Express route adds X-API-Key
await listAvatars(cfg);

Security: the API key grants full access to the avatar server. If your app runs in an untrusted browser context, prefer the proxy so the key isn't shipped in your bundle. The direct apiKey path exists for server components, scripts, and trusted internal tools.

The same apiKey / baseUrl flow into the useAvatars hook (its options extend AvatarApiConfig).

useAvatars hook

One hook for list + create + delete + live status.

import { useAvatars } from "@coreexl/mentor";

function AvatarManager() {
  const { avatars, status, loading, error, create, remove, refresh } = useAvatars({
    baseUrl: "https://avatar.example.com",
    apiKey: process.env.NEXT_PUBLIC_AVATAR_KEY,  // or omit + use a proxy baseUrl
    liveStatus: "ws",                            // "ws" (push) | "poll" | "off"
  });

  async function onUpload(file: File) {
    // create() resolves the new avatar_id; watch status[id].state to track progress
    const id = await create({ imageFile: file, aspect: "3:4", zoom: 1.2 });
    console.log("creating", id);
  }

  if (loading) return <p>Loading…</p>;
  if (error) return <p>{error}</p>;

  return (
    <ul>
      {avatars.map((a) => (
        <li key={a.avatar_id}>
          {a.avatar_id} — {status[a.avatar_id]?.state ?? "ready"}
          {" "}({Math.round((status[a.avatar_id]?.progress ?? 1) * 100)}%)
          <button onClick={() => remove(a.avatar_id)}>delete</button>
        </li>
      ))}
    </ul>
  );
}

Options — UseAvatarsOptions extends AvatarApiConfig (baseUrl, apiKey?, fetchImpl?) plus:

| Option | Type | Default | Description | |--------|------|---------|-------------| | liveStatus | "ws" \| "poll" \| "off" | "poll" | How generation status updates. ws = WebSocket push; poll = periodic GET. | | pollMs | number | 2000 | Poll interval when liveStatus="poll". | | wsBaseUrl | string | derived from baseUrl | Explicit ws(s):// base when liveStatus="ws" (e.g. when baseUrl is a relative proxy). |

Returns — UseAvatarsReturn:

| Field | Type | Description | |-------|------|-------------| | avatars | AvatarSummary[] | Current avatars. | | status | Record<string, AvatarStatus> | avatar_id → live status. | | loading | boolean | Initial load in flight. | | error | string \| null | Last error, if any. | | refresh | () => Promise<void> | Re-fetch the list. | | create | (input: CreateAvatarInput) => Promise<string> | Resolves the new avatar_id. | | remove | (avatarId: string) => Promise<void> | Delete an avatar. |

Functional client

The same operations as standalone functions (each takes the AvatarApiConfig first):

import {
  listAvatars, createAvatar, deleteAvatar,
  getAvatarStatus, subscribeAvatarStatus, fileToBase64,
} from "@coreexl/mentor";

const cfg = { baseUrl: "https://avatar.example.com", apiKey: process.env.AVATAR_KEY };

// List
const { avatars } = await listAvatars(cfg);

// Create (from a File, or raw base64 / a server-local path)
const { avatar_id } = await createAvatar(cfg, {
  imageBase64: await fileToBase64(file),
  aspect: "1:1",
});

// One-off status
const st = await getAvatarStatus(cfg, avatar_id);   // → AvatarStatus

// Live status (WebSocket); returns an unsubscribe fn
const stop = subscribeAvatarStatus(cfg, avatar_id, (s) => console.log(s.state, s.progress));
// …later: stop();

// Delete
await deleteAvatar(cfg, avatar_id);

| Function | Signature | Returns | |----------|-----------|---------| | listAvatars | (cfg) | { avatars: AvatarSummary[] } | | createAvatar | (cfg, input: CreateAvatarInput) | { status, avatar_id, aspect, width, height } | | getAvatarStatus | (cfg, avatarId) | AvatarStatus | | deleteAvatar | (cfg, avatarId) | { status, avatar_id } | | subscribeAvatarStatus | (cfg, avatarId, onStatus) | () => void (unsubscribe) | | fileToBase64 | (file: File) | Promise<string> (raw base64, no data: prefix) |

Avatar types

type AvatarState = "queued" | "generating_idle_loop" | "ready" | "failed" | "unknown";
type AvatarAspect = "1:1" | "3:4" | "9:16" | "4:3" | "16:9";

interface CreateAvatarInput {
  avatarId?: string;   // optional; derived from the image hash if omitted
  imageBase64?: string;
  imageFile?: File;    // convenience — converted to base64 for you (hook only)
  imagePath?: string;  // a path local to the avatar server
  aspect?: AvatarAspect;  // default "1:1"
  zoom?: number;       // >1 tighter on the face, <1 wider
  offsetX?: number;    // framing nudge: +right / -left
  offsetY?: number;    // framing nudge: +down  / -up
}

interface AvatarSummary {
  avatar_id: string;
  aspect?: string;
  width?: number | null;
  height?: number | null;
  has_idle_loop?: boolean;
  active?: boolean;
}

interface AvatarStatus {
  avatar_id: string;
  registered: boolean;
  aspect?: string;
  width?: number | null;
  height?: number | null;
  has_idle_loop: boolean;
  state: AvatarState;
  progress?: number | null;       // 0..1 during generation
  total_chunks?: number | null;
  elapsed_s?: number | null;
  streaming?: boolean;
  error?: string | null;
}

TTS providers

Google Chirp3 HD (default)

Neural voices in 50+ languages. Auto-selects from language + gender, or pin one with voiceName:

<MentorPage ttsProvider="google" language="en-US" gender="female" voiceName="Aoede" />

Chatterbox Turbo (voice cloning)

GPU TTS with voice cloning. Requires a running Chatterbox server configured on your backend. Supports inline paralinguistic tags: [laughs], [sighs], [excited], [chuckle], [gasp], [whispers].

<MentorPage ttsProvider="chatterbox" chatterboxVoice="jamaican_female" />

When chatterboxVoice contains "jamaican", the agent adopts a Jamaican voice style (authentic expressions, inline emotion tags, rhythmic punctuation).

Upload a custom voice once (saved by name on the Chatterbox server):

curl -X POST https://your-chatterbox-server/voices/upload \
  -F "name=my_voice" -F "[email protected]"

Then: <MentorPage ttsProvider="chatterbox" chatterboxVoice="my_voice" />

Language support

Pass any BCP-47 code via language; a voice is auto-picked per gender. Common codes: en-US, en-IN, en-GB, hi-IN, pa-IN, ur-IN, bn-IN, ta-IN, te-IN, mr-IN, gu-IN, kn-IN, ml-IN, fr-FR, de-DE, es-ES, pt-BR, ar-XA, zh-CN, ja-JP, ko-KR, it-IT, nl-NL, pl-PL, ru-RU, tr-TR, id-ID, th-TH, vi-VN, and more. Pin an exact voice with voiceName to override the auto-selection.

Overlay mode

<MentorPage accessKey="..." overlay agentName="Tutor" />

Renders as a floating, minimizable widget in the bottom-right corner.

Tool calling

Forward OpenAI-format tools to the agent and handle calls in the browser:

<MentorPage
  accessKey="..."
  tools={[{
    type: "function",
    function: {
      name: "show_hint",
      description: "Show a hint to the student",
      parameters: { type: "object", properties: { hint: { type: "string" } }, required: ["hint"] },
    },
  }]}
  onToolCall={async (toolCall) => {
    if (toolCall.name === "show_hint") {
      setHint(toolCall.arguments.hint as string);
      return { success: true };
    }
  }}
/>

Environment variables

| Variable | Description | |----------|-------------| | NEXT_PUBLIC_COREEXL_MENTOR_SERVER | Default token-server URL (used if serverUrl is omitted). | | NEXT_PUBLIC_ACCESS_KEY | Default access key (used if accessKey is omitted). |

Exports

| Group | Names | |-------|-------| | Components | MentorPage, AvatarPanel, VideoPanel, VoicePanel, ChatPanel, ChatInput, StreamdownMessage, UnauthorizedView, PresentPanel, PresentMode | | Hooks | useMentorState, useLiveKit, useConversation, useAccessVerification, useAvatars | | Avatar client | listAvatars, createAvatar, deleteAvatar, getAvatarStatus, subscribeAvatarStatus, fileToBase64 | | Providers / utils | SWRProvider, createAuthHeaders, logger, configureLogger | | Types | MentorPageProps, MentorControls, Agent, MentorMessage, Tool, AvatarStatus, AvatarSummary, AvatarState, AvatarAspect, CreateAvatarInput, AvatarApiConfig, UseAvatarsOptions, UseAvatarsReturn |

Troubleshooting

Unstyled / broken layout — import @coreexl/mentor/mentor.css (option A) or add the @source directive (option B). Don't do both.

Avatar greets twice / echoes — your token server must start the agent with room audio output disabled in avatar mode (the avatar republishes the agent's audio; otherwise it's heard twice). This is a backend/agent config, not a package issue.

Avatar status never updates (useAvatars with liveStatus: "ws") — if baseUrl is a relative proxy path, a WebSocket can't be opened against it. Pass an absolute wsBaseUrl (e.g. wss://avatar.example.com), or use liveStatus: "poll".

401/403 from the avatar API — the apiKey is missing or wrong, or your proxy isn't injecting X-API-Key.

License: MIT