@microsoft/voice-widget-react

Headless React binding for the Voice Agent Widget SDK — a standalone useVoiceAgent() hook, an optional <VoiceAgentProvider>, and granular hooks, all over the @microsoft/voice-widget core. No UI; depends only on the headless core (never @microsoft/voice-widget-ui). react is a peer dependency (>=18).

Want the ready-made UI in React? Don't use this package -- drop the <voice-agent> custom element straight into your JSX (it's a web component). This package is for teams building their own UI in React.

Standalone hook (primary API)

import { useVoiceAgent } from "@microsoft/voice-widget-react";
import "@microsoft/voice-widget-provider-voicelive"; // side-effect: registers the "voicelive" provider

function SupportWidget() {
  const { status, start, stop, sendText } = useVoiceAgent({
    provider: "voicelive",
    config: { targetType: "agent", agentName: "support-bot" },
    authEndpoint: "/api/voice/session",
    onTranscript: (m) => appendToMyTranscript(m), // accumulation is your UI's job
  });
  // Render your own UI from the returned state/methods. The full return is
  // { status, mode, muted, error, start, stop, setMuted, sendText, getOutputLevel } —
  // getOutputLevel() is RAF-poll only (non-reactive), for a level meter.
  return <button onClick={() => sendText("Hello")}>Send a typed turn</button>;
}

One useVoiceAgent(opts) call owns one session. opts are frozen at mount (change provider/config by remounting via a React key); event callbacks always call the latest closure.

This package bundles no provider. Any registered one works — swap the import and the provider name. See Writing a provider.

Shared session across components

Wrap a subtree in <VoiceAgentProvider> and read it with the granular hooks -- each re-renders only on its slice.

import {
  VoiceAgentProvider,
  useVoiceAgentStatus,
  useVoiceAgentControls,
} from "@microsoft/voice-widget-react";
import "@microsoft/voice-widget-provider-voicelive";

function App() {
  return (
    <VoiceAgentProvider provider="voicelive" config={{ targetType: "agent", agentName: "support-bot" }} authEndpoint="/api/voice/session">
      <Header />
      <Panel />
    </VoiceAgentProvider>
  );
}

const Header = () => <span>{useVoiceAgentStatus()}</span>;
function Panel() {
  const { start, stop, sendText } = useVoiceAgentControls(); // stable -- never re-renders
  return <><button onClick={start}>Call</button><button onClick={stop}>End</button><button onClick={() => sendText("Hello")}>Send</button></>;
}

One owner per session: use a standalone useVoiceAgent(opts) OR a <VoiceAgentProvider>, never both for the same session. Two useVoiceAgent(opts) calls = two sessions (two mic prompts).

Client tools

Client tools let the agent invoke client-side functionality — open the cart, read the cart total, go to checkout. The tool's schema (name, description, parameters) is declared server-side in your broker or agent definition; what you write here is the matching handler, looked up by name. Names are case-sensitive and must match the schema exactly — a mismatch is the usual reason a tool never fires. If a handler returns a value it is passed back to the agent as the tool result. See the client tools guide for the schema and policy model.

Pass your handlers as an object of functions on useVoiceAgent (or on <VoiceAgentProvider>):

const { start } = useVoiceAgent({
  provider: "voicelive",
  config: { targetType: "agent", agentName: "support-bot" },
  authEndpoint: "/api/voice/session",
  clientTools: {
    openCart: () => store.openCart(),
    getCartTotal: () => ({ total: store.cartTotal() }),
  },
});

This is the path to reach for: every tool sits in one place, registered before the session connects, so the agent can call it on the very first turn. Note that opts are frozen at mount, so these handlers keep the closure they were created with — fine for a tool that goes through a store, router, or API, and the reason a tool that must read a component's current state belongs in that component instead.

For a more React-idiomatic way to register a tool whose handler needs component state, see useVoiceAgentClientTool below.

Client tools run untrusted input. A handler's arguments are filled in by the model and can be prompt-injected — validate them before any sensitive action (e.g. allow only http:/https: URLs before navigating; never eval or inject a model-supplied string as HTML/JS). Its return value is sent to the model and may be spoken aloud, so return only end-user-safe values — no PII or internal error detail. See Security - handler inputs and outputs.

useVoiceAgentClientTool

A hook for dynamically registering client tools from React components. Tools are automatically unregistered when the component unmounts.

This is useful when a tool's handler needs component state or props that aren't available where the session is configured. The handler is re-read on every commit, so you don't need to worry about stale state — a call always runs the latest committed closure.

import { useVoiceAgentClientTool } from "@microsoft/voice-widget-react";

function MapPanel() {
  const [location, setLocation] = useState({ lat: 0, lng: 0 });

  // Registered inside MapPanel and kept live — getLocation always returns the CURRENT location.
  useVoiceAgentClientTool("getLocation", () => location);

  return <button onClick={() => setLocation({ lat: 48.8, lng: 2.3 })}>Move to Paris</button>;
}

• Requires a <VoiceAgentProvider> ancestor — it registers into that shared session.
• Unregisters on unmount / name change, and is StrictMode-safe. Two mounted components claiming the same name throw.
• The tool exists only while the component is mounted. If the agent calls it while unmounted, that surfaces as an unhandled tool call — so keep such tools on components that live for the whole conversation, or declare them in clientTools.

It registers into the same registry the clientTools map seeds, so the two compose freely: declare what you can statically, and reach for the hook for the handlers that need to live with a component. useVoiceAgent(opts) also returns registerClientTool(name, handler) (it returns an off()) for the rare case with no render to hang a hook on — an async callback, or after a lazy import resolves.

Registering mid-session (the hook or registerClientTool, after start()) reaches the running session only if the provider declares features.dynamicClientTools. The Voice Live adapter does; otherwise the registration applies to the next connect and the core logs a warning.

API

• useVoiceAgent(opts): UseVoiceAgentResult -- standalone; owns a session.
• <VoiceAgentProvider {...opts}> -- optional; owns one shared session.
• useVoiceAgentStatus() / useVoiceAgentMode() / useVoiceAgentMuted() / useVoiceAgentError() -- one slice each (require a provider).
• useVoiceAgentControls() -- { start, stop, setMuted, sendText, getOutputLevel }, stable (requires a provider).
• useVoiceAgentClientTool(name, handler) -- a hook for dynamically registering client tools from React components; use it when a handler needs a component's live state (requires a <VoiceAgentProvider>). See Client tools above.
• useVoiceAgent(opts) also returns registerClientTool -- imperative registration for non-hook code paths. See Client tools above.

opts is VoiceAgentOptions (from @microsoft/voice-widget): provider, config, and one of authEndpoint / getSession / negotiate, plus optional clientTools, widgetId, fetchCredentials, telemetryConsole, and the event callbacks onStatusChange / onModeChange / onError / onTranscript / onConnect / onDisconnect / onMuted / onUnhandledClientToolCall / onRawEvent / onTelemetryEvent.

Notes

• Invalid config fails fast at render. A malformed inline config is validated when the session is created (during the hook's first render), so it throws synchronously rather than surfacing via the reactive error field (which is for connection-time failures). Validate config before mount, or wrap the component in an error boundary.
• SSR. The hook renders the idle snapshot on the server (SSR-safe). Full server rendering also requires the provider package (e.g. @microsoft/voice-widget-provider-voicelive) to be import-safe and registered in the server bundle, since the core is constructed during the server render pass.
• Imperative callbacks may fire during teardown. On unmount the hook stops the session, which can invoke your onStatusChange one final time with "idle". This is harmless (the component is gone and no re-render happens), but avoid side effects in these callbacks that assume the component is still mounted.
• onRawEvent is a debugging hatch, not part of the contract. It fires for every raw provider event — the Voice Live adapter fires it per voice-live-events data-channel message, so dozens to hundreds per turn. Keep the handler cheap (no synchronous work, no per-event network calls, and don't setState on every event), and don't drive product behavior from the payload: it is untyped and its shape is whatever the provider sends. Use onTranscript / onModeChange / onStatusChange / onError for that. See Debugging: raw provider events.
• onTelemetryEvent is the structured telemetry stream. One typed VoiceAgentTelemetryEvent per lifecycle/transport event, correlatable across the widget and your broker. Like onRawEvent it is read live from a ref, so replacing it between renders takes effect — but unlike onRawEvent it is a stable, low-frequency contract and is fault-isolated (a throw is reported once, then suppressed). telemetryConsole is a plain boolean and, like other session options, is frozen at mount — change it via a React key remount. See docs/telemetry.md.