@supportsally/voice-realtime
v0.1.0-alpha.2
Published
Sally live speech-to-speech agent: OpenAI Realtime events, Grok first (D-28)
Readme
@supportsally/voice-realtime
Live speech-to-speech agent: OpenAI Realtime WebSocket events, Grok first (Milestone AA / D-28). Conversation + tool selection only — @supportsally/core still executes.
Alpha (0.1.0-alpha.2) — APIs may change. @supportsally/react already depends on this package. Import it directly only after pnpm add @supportsally/voice-realtime (pnpm hides transitive imports). Browser must not hold provider API keys; mint ephemeral tokens on the host.
Mint tokens on the host. For hosted Studio, POST tokenUrl with Authorization: Bearer <embed key> and pass stopUrl so billed minutes settle. Any minted holdId is settled on stop, on failed setup (0 ms), and on pagehide (keepalive). BYOK same-origin proxies may omit apiKey / stopUrl. Idle 90s and max 10 min/tap pause Talk without abandoning the guide.
createRealtimeVoiceAgent({
tokenUrl: "https://studio.example/api/v1/runtime/voice-session",
stopUrl: "https://studio.example/api/v1/runtime/voice-stop",
apiKey: "sally_embed_…",
});Dictation (alpha): createGrokDictation streams the mic to Grok speech-to-text (wss://api.x.ai/v1/stt) for the note on an attached PDF. Words go to the widget input; no replies, no tools. Mint with the same token shape (hosted: /api/v1/runtime/dictation-session).
createGrokDictation({
tokenUrl: "https://studio.example/api/v1/runtime/dictation-session",
stopUrl: "https://studio.example/api/v1/runtime/voice-stop",
apiKey: "sally_embed_…",
});Do not enable provider search/MCP tools. Pin grok-voice-3.0 in production. Do not use grok-voice-latest. Leave turn_detection.idle_timeout_ms unset. Gemini Live and GPT-Live stay out.
See monorepo root README and SECURITY.md.
