@microsoft/voice-widget-core-client

The provider-neutral contract for the Voice Agent Widget SDK: the VoiceAgentProvider interface, event/config types, and the provider registry. No UI, no provider-specific code.

What's here

• Types — VoiceAgentProvider, SessionRequest, SessionGrant (discriminated union: webrtc/direct vs websocket/relay), ProviderConfig, ConnectionStatus + AgentMode (orthogonal axes), VoiceAgentProviderEvents, ConnectOptions, ClientTool(s), capabilities.
• Client-tool types — ClientToolRegistry / createClientToolRegistry (core-owned live registry; unique names, identity-guarded unregister), ClientToolPolicy (followUpResponse?), UnhandledClientToolCall, SdpNegotiationResult (carries clientToolPolicy from the broker).
• Broker lifecycle hooks — ConnectOptions.negotiate creates the media session, sendSessionEvent forwards control events, and endSession closes the broker-owned control session during teardown.
• Registry — registerProvider / createProvider / hasProvider / listProviders. Providers self-register so the widget can resolve provider="…" to an adapter without the core importing any provider package.

Audio primitives

Provider-neutral browser audio toolkit under src/audio/ (re-exported from the package root):

• MicCapture — getUserMedia + an AudioWorklet capture processor; emits mono 16-bit PCM frames at a target rate (resampled from the context rate) plus a smoothed input level; setMuted.
• AudioPlayback — streaming PCM playback via an AudioWorklet queue; enqueue (16-bit or Float32), clear for barge-in/interruption, setMuted, output level, onDrained.
• createStreamLevelMeter — AnalyserNode-based output level for a MediaStream (used by the WebRTC path, where agent audio is a remote stream rather than a PCM queue).
• Helpers — floatToPcm16, pcm16ToFloat, resampleLinear, rms.

Any provider implementation lives in a separate provider package (e.g. @microsoft/voice-widget-provider-voicelive).

clientTools vs dynamicClientTools capability flags

Providers that support client-side tools advertise one of two feature flags:

• features.clientTools — the adapter takes a snapshot of handlers at connect() time; handlers registered after connect() are not seen by the running session.
• features.dynamicClientTools — the adapter does a live lookup of the handler from the registry at each call, so handlers can be added or removed during a session via VoiceAgentCore.registerClientTool. Most useful when handler availability changes after connect (lazy-loaded handlers or provider-scoped hooks).

Both flags describe handler resolution in the browser — tool schemas are declared server-side (model mode: session.update; agent mode: the Foundry agent definition). Delivery of the agent's function call to the browser is supported for model and agent targets.

Usage

import { registerProvider, createProvider } from "@microsoft/voice-widget-core-client";
import { AcmeProvider } from "./acme-provider.js";

// A provider package registers a factory on import. Each widget gets its own instance.
registerProvider("acme", () => new AcmeProvider());

// Session owners request a fresh adapter by name:
const provider = createProvider("acme");