react-native-deepgram
v3.0.1
Published
React Native SDK for Deepgram's AI-powered speech-to-text, real-time transcription, and text intelligence APIs. Supports live audio streaming, file transcription, sentiment analysis, and topic detection for iOS and Android.
Maintainers
Readme
react-native-deepgram
The complete client for Deepgram's voice AI platform on React Native and Expo — real-time transcription, conversational voice agents, text-to-speech, and text intelligence in a single dependency, with hardware echo cancellation and background-audio support wired up for you.
yarn add react-native-deepgramAt a glance
| | |
| --- | --- |
| 🗣️ Voice Agent | Full-duplex agents over Deepgram's WebSocket. Hardware AEC on iOS (VPIO) and Android (VOICE_COMMUNICATION + MODE_IN_COMMUNICATION). Function calling, prompt updates, latency telemetry. |
| 🎙️ Speech-to-Text | Live PCM streaming over WebSocket on STT v1 or Flux v2 (flux-general-en). File transcription with summarisation, topics, intents, entities. Optionally record the mic to a WAV file while you stream. |
| 🔊 Text-to-Speech | One-shot HTTP synthesis or low-latency WebSocket streaming for Aura and Flux TTS voices, with Flush / Clear / Close controls and Flux barge-in (interrupt()). |
| 🧠 Text Intelligence | Summaries, topics, intents, sentiment over text or URLs. |
| 🛠️ Management API | Typed wrapper for projects, models, keys, usage, balances, members, invitations, scopes, purchases, and temporary tokens. |
| ⚙️ Expo plugin | One-line install — handles permissions, background-audio modes, and Android foreground service automatically. |
| 🧰 Resilient by default | Audio interruptions, route changes, headphone unplugging, mediaserverd resets, audio-focus loss, and queue overflow are all handled — plus opt-in WebSocket auto-reconnect with exponential backoff. |
| 🆕 Modern iOS APIs | AVAudioApplication.requestRecordPermission and AllowBluetoothHFP on iOS 17+, with safe fallbacks. |
Table of contents
- Quick start (5 minutes)
- Requirements
- Installation
- Migrating from 2.x
- Core concepts
- Audio session, AEC & background
- Voice Agent
- Speech-to-Text
- Text-to-Speech
- Text Intelligence
- Management API
- Expo config plugin reference
- Common recipes
- Troubleshooting
- Example app
- Roadmap
- Contributing
- License
Quick start (5 minutes)
1. Install
yarn add react-native-deepgram2. Configure the Expo plugin
// app.config.js (or app.json under "plugins")
module.exports = {
expo: {
plugins: [
[
'react-native-deepgram',
{
microphonePermission:
'Allow $(PRODUCT_NAME) to access your microphone.',
backgroundAudio: true, // default — keeps audio alive in the background; set false to skip the Android foreground service + iOS background mode
},
],
],
},
};npx expo prebuild
npx expo run:ios # or expo run:androidBare React Native? Skip the plugin, run
cd ios && pod install. The package autolinks viareact-native.config.js.
3. Configure the SDK once at startup
// App.tsx
import { configure } from 'react-native-deepgram';
configure({ apiKey: process.env.EXPO_PUBLIC_DEEPGRAM_API_KEY! });Never ship raw API keys. Use
EXPO_PUBLIC_*env vars, secrets storage, or a backend proxy that mints scoped keys.
4. Stream microphone audio to Deepgram (live STT)
import { useDeepgramSpeechToText } from 'react-native-deepgram';
import { Button, Text, View } from 'react-native';
export function LiveCaption() {
const {
startListening,
stopListening,
state, // { status: 'idle' | 'listening' | 'error' | ... }
transcript, // accumulated final transcript
interimTranscript, // current partial
} = useDeepgramSpeechToText({
trackState: true,
trackTranscript: true,
live: { punctuate: true, interimResults: true },
});
return (
<View>
<Text>{transcript} {interimTranscript}</Text>
<Button title="Start" onPress={() => startListening()} />
<Button title="Stop" onPress={stopListening} />
{state?.error && <Text style={{ color: 'red' }}>{String(state.error)}</Text>}
</View>
);
}5. Talk to a Voice Agent (full-duplex with AEC)
import { useDeepgramVoiceAgent } from 'react-native-deepgram';
const agent = useDeepgramVoiceAgent({
trackState: true,
trackConversation: true,
defaultSettings: {
audio: {
input: { encoding: 'linear16', sample_rate: 16_000 }, // mic capture rate
output: { encoding: 'linear16', sample_rate: 24_000, container: 'none' },
},
agent: {
greeting: 'Hi! What can I help you with?',
listen: { provider: { type: 'deepgram', model: 'nova-3', language: 'en' } },
think: { provider: { type: 'open_ai', model: 'gpt-4o' }, prompt: 'You are a helpful assistant.' },
speak: { provider: { type: 'deepgram', version: 'v2', model: 'flux-kit-en' } },
},
},
});
await agent.connect(); // requests mic, opens WS, starts streaming
agent.injectUserMessage('Hi'); // optional: prompt the agent with text
agent.disconnect(); // tears everything downThat's a working, low-latency conversational agent with hardware echo cancellation. Open the example app to try the full surface area.
Requirements
| | Minimum | | --- | --- | | Node | 18 | | React | 19.0 | | React Native | 0.80 (New Architecture) | | iOS | 15.1 (React Native's minimum) | | Android | API 24 (Android 7.0) | | Xcode | 16.1 (React Native 0.80's minimum) |
The native module is a TurboModule and requires the New Architecture (the default since React Native 0.76). The legacy architecture is no longer supported as of 3.0.
Installation
yarn add react-native-deepgram
# or
npm install react-native-deepgramBare React Native
cd ios && pod installThe package is autolinked through react-native.config.js — no manual MainApplication edits required.
Add android.permission.RECORD_AUDIO to your AndroidManifest.xml and NSMicrophoneUsageDescription to your Info.plist if you use any microphone feature. If you also want audio to keep running in the background, see Background audio for the opt-in foreground-service entries.
Expo (managed or bare)
Add the plugin to your config:
// app.config.js
module.exports = {
expo: {
plugins: [
[
'react-native-deepgram',
{
microphonePermission:
'Allow $(PRODUCT_NAME) to access your microphone.',
// backgroundAudio: true, // default — see plugin docs below
},
],
],
},
};Then prebuild and run:
npx expo prebuild
npx expo run:ios # or expo run:androidAlready running an existing development build? Re-run
expo prebuild(andpod installon iOS) so the new permissions, background modes, and Android foreground service are picked up.
Migrating from 2.x
3.0 drops the legacy architecture and the APIs that were deprecated in 2.x. After upgrading, rebuild the native app (npx expo prebuild --clean or pod install) — a JS-only reload is not enough.
Platform
- React Native 0.80+, React 19 and the New Architecture are required.
- The package is ESM-only; deep imports from
react-native-deepgram/lib/commonjsno longer exist. - If you listened to native device events directly (
DeepgramAudioPCM/AudioChunk,DeepgramAudioLevel,DeepgramRouteChange,DeepgramInterruption) or calledNativeModules.Deepgram/MicPermission, use the typedDeepgramexport instead:Deepgram.onAudioPCM,onAudioLevel,onAudioRouteChange,onAudioInterruptionandDeepgram.requestRecordPermission().addAudioRouteChangeListenerandaddInterruptionListenerare unchanged. Deepgram.stopPlayer()now returns a promise.
Speech-to-Text
| 2.x | 3.0 |
| --- | --- |
| diarize: true | diarizeModel: 'latest' |
Text-to-Speech
| 2.x | 3.0 |
| --- | --- |
| options: { model, encoding, sampleRate, speed, mipOptOut } | options: { http: { … }, stream: { … } } (set per transport) |
| options: { bitRate, container, tag, callback, callbackMethod } | options: { http: { … } } |
| options.format / http.format | http.container |
| sendMessage({ type: 'Text', text }) | sendMessage({ type: 'Speak', text }) or sendText(text) |
| DeepgramTextToSpeechEncoding | DeepgramTextToSpeechHttpEncoding / DeepgramTextToSpeechStreamEncoding |
synthesize() only plays raw linear16 output (the default) and respects autoPlayAudio; compressed formats are returned without playback.
Voice Agent
| 2.x | 3.0 |
| --- | --- |
| agent.language | agent.listen.provider.language (Deepgram voices take the language from the model name) |
| audio.input.sample_rate above 16000 | Capped at 16000, the microphone capture rate |
| injectAgentMessage(message, 'queue') | Unchanged, or injectAgentMessage(message, { behavior, dynamic }) |
See the changelog for the full list, including the new Flux, Configure and Voice Agent APIs.
Core concepts
A few ideas you only have to learn once.
1. Configure once, use anywhere
import { configure } from 'react-native-deepgram';
configure({ apiKey: process.env.EXPO_PUBLIC_DEEPGRAM_API_KEY! });configure is global — call it at the entry point (e.g. App.tsx). Every hook reads from this config. The Management API additionally needs a key with management scopes.
Never ship raw API keys. Use
EXPO_PUBLIC_*env vars, secrets storage, or a backend proxy that mints scoped keys.
Custom endpoints (regional, Dedicated & self-hosted)
Point the SDK at a non-default Deepgram deployment by passing base URLs to configure. Every override is optional and falls back to the public Deepgram endpoints.
configure({
apiKey: process.env.EXPO_PUBLIC_DEEPGRAM_API_KEY!,
baseUrl: 'https://api.beta.deepgram.com/v1', // REST (must include the version segment)
baseWss: 'wss://api.beta.deepgram.com/v1', // live STT + streaming TTS sockets
agentUrl: 'wss://agent.deepgram.com/v1/agent/converse', // Voice Agent socket
});| Option | Applies to | Default |
| ------ | ---------- | ------- |
| baseUrl | REST calls (file STT, TTS, Text Intelligence, Management) | https://api.deepgram.com/v1 |
| baseWss | Live STT + streaming TTS sockets | wss://api.deepgram.com/v1 |
| agentUrl | Voice Agent socket | wss://agent.deepgram.com/v1/agent/converse |
baseUrl/baseWss must include the version segment (/v1). Flux v2 endpoints are derived automatically by swapping the trailing /v1 for /v2. See Deepgram's custom endpoints reference.
Ephemeral / scoped tokens (keep your API key off-device)
Instead of shipping a long-lived apiKey, you can hand configure a getToken provider that returns a short-lived Deepgram token minted by your backend (which proxies Deepgram's /v1/auth/grant). The SDK caches the token, refreshes it before it expires, and de-duplicates concurrent refreshes — so a burst of requests never triggers multiple grants.
configure({
// No raw apiKey on the device. `getToken` takes precedence when provided.
getToken: async () => {
const res = await fetch('https://your-backend.example.com/deepgram-token');
const { access_token, expires_in } = await res.json();
return { token: access_token, expiresInSeconds: expires_in };
},
});| Field | Type | Description |
| ----- | ---- | ----------- |
| token | string | The short-lived token. Sent as Authorization: Bearer <token>. |
| expiresInSeconds | number (optional) | TTL hint used to refresh ahead of expiry. Defaults to 30. |
- When both
getTokenandapiKeyare set,getTokenwins; the API key is the fallback. - The token only needs to be valid at connection time — live STT / Voice Agent sockets keep running after it expires.
- Exception: the Management API (
useDeepgramManagement) cannot authenticate with temporary tokens, so it always uses the configuredapiKey.
2. Hooks return reactive state — opt in
Every hook supports track* flags that turn on a reactive return value. Without them, the hook stays event-driven (callbacks only) so it never causes unnecessary re-renders.
const { transcript, state } = useDeepgramSpeechToText({
trackTranscript: true, // exposes `transcript` + `interimTranscript`
trackState: true, // exposes `state.status`, `state.error`
});| Hook | Tracking flags |
| --- | --- |
| useDeepgramVoiceAgent | trackState, trackConversation, trackAgentStatus |
| useDeepgramSpeechToText | trackState, trackTranscript |
| useDeepgramTextToSpeech | trackState |
| useDeepgramTextIntelligence | trackState |
3. Audio session is automatic
You don't manage AVAudioSession, AudioManager, or focus. The native module activates the right session category, switches to VoiceChat / MODE_IN_COMMUNICATION when echo cancellation is enabled, requests/abandons audio focus, starts/stops a foreground service on Android, and rebuilds after mediaserverd resets on iOS.
For tuning details, see Audio session, AEC & background.
4. Hooks at a glance
| Hook | Purpose | Endpoint |
| --- | --- | --- |
| useDeepgramVoiceAgent | Conversational agents (full-duplex) | wss://agent.deepgram.com/v1/agent/converse |
| useDeepgramSpeechToText | Live STT + file transcription | wss://api.deepgram.com/v1/listen (v1) / /v2/listen (Flux) |
| useDeepgramTextToSpeech | TTS via REST or streaming | https://api.deepgram.com/v1/speak & wss://api.deepgram.com/v1/speak |
| useDeepgramTextIntelligence | Summaries, topics, intents, sentiment | https://api.deepgram.com/v1/read |
| useDeepgramManagement | Projects / keys / usage REST API | https://api.deepgram.com/v1 |
5. Typed errors
Every error the hooks surface — mic-permission denials, native start/stop failures, fetch and WebSocket errors — is normalized to a DeepgramError with a typed code, so you can branch on a stable enum instead of string-matching messages. The original failure is preserved on cause.
import { DeepgramError, useDeepgramSpeechToText } from 'react-native-deepgram';
useDeepgramSpeechToText({
onError: (err) => {
if (err instanceof DeepgramError && err.code === 'permission_denied') {
promptForMicAccess();
}
},
});The code is one of the native reject codes (or 'unknown' when it can't be mapped):
permission_denied · init_failed · start_error · stop_error · audio_start_error · audio_stop_error · stop_player_error · invalid_data · playback_error
Audio session, AEC & background
This section covers what the native module does for you so you can debug or tune behaviour with confidence.
Hardware echo cancellation (full-duplex)
When you build a Voice Agent (or any duplex flow where Deepgram is speaking and the mic is open), set enableVoiceProcessing: true:
import { Deepgram } from 'react-native-deepgram';
await Deepgram.startRecording({ enableVoiceProcessing: true });useDeepgramVoiceAgent opts into this automatically. What this gives you:
| Platform | Behaviour |
| --- | --- |
| iOS | Capture is routed through AVAudioEngine.inputNode with setVoiceProcessingEnabled:YES on both input and output, and the audio session uses AVAudioSessionModeVoiceChat. This engages Apple's Voice-Processing I/O Audio Unit — the only way to get hardware AEC on iOS. |
| Android | Capture uses MediaRecorder.AudioSource.VOICE_COMMUNICATION and the system is set to MODE_IN_COMMUNICATION so the platform's hardware AEC engages with the active playback signal as its reference. The previous mode is restored on stopRecording. |
For pure speech-to-text usage you generally don't want this — leave the option off (the default). STT-only paths use VOICE_RECOGNITION on Android and a pure AudioQueue capture on iOS so Deepgram receives the rawest possible signal.
Background audio
Background audio is opt-in: the library itself ships with no foreground-service permissions, so apps that only use foreground audio (or the pure-REST Text Intelligence / Management APIs) never have to justify foreground-service types in the Google Play Console.
Leave backgroundAudio: true on the Expo plugin (it's the default) to keep playback and capture alive when the user leaves the app:
- iOS —
UIBackgroundModes: ["audio"]is added to yourInfo.plist. - Android — the
FOREGROUND_SERVICE,FOREGROUND_SERVICE_MICROPHONE, andFOREGROUND_SERVICE_MEDIA_PLAYBACKpermissions are added to your manifest, and the bundledDeepgramAudioServiceis givenforegroundServiceType="microphone|mediaPlayback". The service is started/stopped automatically whenever recording or playback is active.
On bare React Native, add the equivalent entries to your AndroidManifest.xml yourself:
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
<application>
<!-- Merges the foreground-service type onto the library's service -->
<service
android:name="com.deepgram.DeepgramAudioService"
android:foregroundServiceType="microphone|mediaPlayback" />
</application>(If your app is playback-only — TTS but no mic in the background — you can omit FOREGROUND_SERVICE_MICROPHONE and use android:foregroundServiceType="mediaPlayback"; the service degrades gracefully.)
Things to know:
Play Console declaration. Apps targeting Android 14+ that declare foreground-service types must describe each type's use (and link a demo video) under App content → Foreground service permissions in the Play Console. For this package:
TYPE_MICROPHONE→ “Background Audio Access” (live transcription / voice agent continues while backgrounded),TYPE_MEDIA_PLAYBACK→ “Media Playback” (TTS/agent audio continues while backgrounded). If you can't justify them, setbackgroundAudio: false(or omit the manifest entries) — then no declaration is needed.App Store privacy. The pod ships a
PrivacyInfo.xcprivacyprivacy manifest declaring audio-data collection (App Functionality, not linked to identity, no tracking) and its required-reason API usage — Xcode aggregates it into your app's privacy report automatically. Remember to also declare your own use of the captured audio (e.g. Deepgram processing/retention) in App Store Connect.Customizing the notification. By default the Android keep-alive notification uses your app's name and launcher icon, and tapping it opens the app. Brand it via the Expo plugin's
androidNotificationoption, or on bare React Native with<meta-data>entries in your<application>element:<application> <meta-data android:name="com.deepgram.notification.TITLE" android:value="Transcribing…" /> <meta-data android:name="com.deepgram.notification.TEXT" android:value="Live captions are running" /> <meta-data android:name="com.deepgram.notification.CHANNEL_NAME" android:value="Live audio" /> <meta-data android:name="com.deepgram.notification.ICON" android:value="ic_stat_audio" /> </application>Each entry is optional. Values can also reference string resources for localization (
android:resource="@string/..."instead ofandroid:value);ICONnames a drawable/mipmap resource and safely falls back to the launcher icon if missing.Graceful degradation. If the permissions or the service's
foregroundServiceType(it must includemediaPlayback) aren't declared, everything still works while the app is in the foreground; the native module simply skips the keep-alive service instead of crashing. Other libraries declaringFOREGROUND_SERVICEdon't opt you in.
Built-in resilience
The native modules handle the awkward edge cases for you:
- Audio interruptions — phone calls, Siri, system alerts pause + resume cleanly on iOS.
- Route changes — unplugging headphones or disconnecting Bluetooth pauses playback (instead of suddenly blasting through the speaker), matching Apple's
AVAudioSessionRouteChangeReasonOldDeviceUnavailableguidance and Android'sACTION_AUDIO_BECOMING_NOISYbroadcast. - Audio focus loss (Android) — transient losses pause playback; permanent losses tear down recording + playback so you don't fight other apps for the mic.
- mediaserverd resets (iOS) — if iOS restarts its audio service mid-session, the audio engine is rebuilt from scratch on the next call.
- Bluetooth on iOS 17+ — uses
AllowBluetoothHFPwhen available, with a deprecation-safe fallback for older toolchains. - iOS 17+ permission API —
Deepgram.requestRecordPermission()callsAVAudioApplication.requestRecordPermissionWithCompletionHandlerwhen available, falling back toAVAudioSessionon older OS versions. - Bounded playback queue (Android) — if JS feeds audio faster than the device can play it, the oldest chunks are dropped at ~1.5 MB so you never OOM mid-call.
Interruption events
All of the pausing/resuming above happens automatically — but your UI can observe it. Subscribe to interruption events to show “paused — on a call” states or to restart a session after Android tears it down on permanent focus loss:
import { addInterruptionListener } from 'react-native-deepgram';
const sub = addInterruptionListener((e) => {
if (e.type === 'began') {
// capture/playback paused by the system
// e.reason: 'phoneCall' | 'focusLoss' | 'routeChange' | 'unknown'
} else if (e.type === 'ended') {
// interruption over; e.shouldResume === true means audio already resumed
} else {
// e.type === 'stopped' — Android permanent focus loss tore the session down;
// call startListening()/connect() again if you want to continue
}
});
// later: sub.remove();The same events are available as an onInterruption callback on useDeepgramSpeechToText, useDeepgramVoiceAgent, and useDeepgramTextToSpeech. While a live STT or agent session is active the hooks also react on your behalf: they keep the Deepgram socket warm with KeepAlive frames for as long as the interruption holds the microphone (otherwise Deepgram closes the idle socket after ~10 s with NET-0001), resume normal streaming when it ends, and end the session gracefully (onEnd/onClose, no error) when Android tears it down on permanent focus loss.
Audio output routing
Steer playback between the loudspeaker, the earpiece and a connected Bluetooth headset, and observe route changes made outside your app (Control Center, plugging in headphones, …):
import {
setAudioRoute,
getAudioRoute,
addAudioRouteChangeListener,
} from 'react-native-deepgram';
await setAudioRoute('speaker'); // 'speaker' | 'earpiece' | 'bluetooth' | 'auto'
const active = await getAudioRoute(); // 'speaker' | 'earpiece' | 'bluetooth' | 'wired'
const sub = addAudioRouteChangeListener((route) => {
console.log('now playing through', route);
});
// later: sub.remove();Things to know:
- Best-effort semantics. The OS always has the final say: a wired headset wins over everything, and
bluetoothonly engages once a headset is actually connected (the request is remembered and applied when one appears). Invalid route names reject withinvalid_data; OS-level failures reject withplayback_error. - Call any time. You can set a route before or during a session. Set it before
connect()/startListening()and it's applied when the audio session spins up. - Bluetooth is opt-in during full-duplex. In echo-cancelled (
enableVoiceProcessing) sessions the module doesn't auto-route to Bluetooth — HFP's loopback defeats hardware AEC — so the agent stays on the built-in mic/speaker until you explicitly request'bluetooth'. - External switches are adopted, not fought. If the user reroutes audio from Control Center or the system UI, the module accepts the new route (and tells you via the listener) instead of forcing your last request back.
- Android permissions —
MODIFY_AUDIO_SETTINGSandBLUETOOTH(API ≤ 30) are merged in from the library manifest automatically.
Voice Agent (useDeepgramVoiceAgent)
useDeepgramVoiceAgent connects to wss://agent.deepgram.com/v1/agent/converse, captures microphone audio, and optionally auto-plays the agent's streamed responses. It wraps the full Voice Agent messaging surface so you can react to conversation updates, function calls, warnings, and raw PCM audio.
🔊 Hardware AEC enabled by default. The hook calls
startRecording({ enableVoiceProcessing: true }), so iOS routes capture through Apple's VPIO Audio Unit and Android switches toVOICE_COMMUNICATION+MODE_IN_COMMUNICATION. The result: the agent never hears itself speak, even on speakerphone. (Note: VPIO is not available in the iOS Simulator — test on a physical device.)
Quick start
const {
connect,
disconnect,
state, // { connectionState, error, warning }
agentStatus, // { thinking, latency }
conversation, // Array<{ role, content }>
injectUserMessage,
sendFunctionCallResponse,
updatePrompt,
} = useDeepgramVoiceAgent({
trackState: true, // Enable reactive state tracking
trackConversation: true, // Enable conversation history tracking
trackAgentStatus: true, // Enable agent status tracking
autoPlayAudio: true, // Automatically play agent audio
defaultSettings: {
audio: {
input: { encoding: 'linear16', sample_rate: 16_000 }, // mic capture rate
output: { encoding: 'linear16', sample_rate: 24_000, container: 'none' },
},
agent: {
greeting: 'Hello! How can I help you today?',
listen: {
provider: {
type: 'deepgram',
model: 'nova-3',
language: 'en',
smart_format: true,
},
},
think: {
provider: { type: 'open_ai', model: 'gpt-4o', temperature: 0.7 },
prompt: 'You are a helpful voice concierge.',
},
speak: {
// Flux TTS (v2); use version 'v1' with an aura-* voice for Aura.
provider: { type: 'deepgram', version: 'v2', model: 'flux-kit-en' },
},
},
tags: ['demo'],
},
onConversationText: (msg) => {
console.log(`${msg.role}: ${msg.content}`);
},
onAgentThinking: (msg) => console.log('thinking:', msg.content),
onAgentAudioDone: () => console.log('Agent finished speaking'),
onServerError: (err) => console.error('Agent error', err.description),
});
const begin = async () => {
try {
await connect();
} catch (err) {
console.error('Failed to start agent', err);
}
};
const askQuestion = () => {
injectUserMessage("What's the weather like?");
};
const provideTooling = () => {
sendFunctionCallResponse({
id: 'func_12345',
name: 'get_weather',
content: JSON.stringify({ temperature: 72, condition: 'sunny' }),
client_side: true,
});
};
const rePrompt = () => {
updatePrompt('You are now a helpful travel assistant.');
};
return (
<>
<Text>Status: {state.connectionState}</Text>
<Button title="Start agent" onPress={begin} />
<Button title="Ask" onPress={askQuestion} />
<Button title="Send tool output" onPress={provideTooling} />
<Button title="Update prompt" onPress={rePrompt} />
<Button title="Stop" onPress={disconnect} />
</>
);💬 The hook requests mic permissions, streams PCM to Deepgram, and surfaces the agent's replies as text so nothing plays back into the microphone.
API reference (Voice Agent)
Signature
const {
// Connection
connect, disconnect, isConnected,
// Mic control
mute, unmute, isMuted,
// Messaging
sendMessage, sendSettings, sendMedia, sendKeepAlive,
injectUserMessage, injectAgentMessage, forceEndTurn, updatePrompt,
updateListen, updateThink, updateSpeak, updateThinkLive,
// Function calls
sendFunctionCallResponse,
// Reactive state (opt-in via track* flags)
state, conversation, agentStatus, clearConversation,
} = useDeepgramVoiceAgent(props);Hook props
Configuration
| Prop | Type | Default | Description |
| ---- | ---- | ------- | ----------- |
| endpoint | string | wss://agent.deepgram.com/v1/agent/converse | WebSocket endpoint used for the agent conversation. |
| defaultSettings | DeepgramVoiceAgentSettings | – | Base Settings payload sent on connect; merge per-call overrides via connect(override). |
| autoStartMicrophone | boolean | true | Automatically requests mic access and starts streaming PCM. |
| autoPlayAudio | boolean | true | Plays received audio using the native player. |
| bargeIn | boolean | false | Flush the agent's queued playback when the server reports UserStartedSpeaking, so the agent audibly stops when the user talks over it. Only flushes while agent audio is actually playing and the mic is not muted; the player stays usable for the next agent turn. Requires autoPlayAudio. Barge-in depends on hardware echo cancellation — test on a physical device (simulators have no VPIO, so the agent's own audio can trigger the VAD). |
| downsampleFactor | number | heuristic | Manually override the downsample ratio applied to captured audio. |
| reconnect | DeepgramReconnectOptions | { enabled: false } | Auto-reconnect config for the agent socket. The stored Settings payload is re-sent on every successful reconnect. |
Reactive tracking flags (opt-in — extra renders only when set)
| Prop | Default | Enables |
| ---- | ------- | ------- |
| trackState | false | state return — connection status, error, warning. |
| trackConversation | false | conversation return — running history of { role, content }. |
| trackAgentStatus | false | agentStatus return — thinking flag and latency metrics. |
| trackStats | false | stats return — session telemetry, updated at most once per second. |
Callbacks
| Callback | Signature | Fired when |
| -------- | --------- | ---------- |
| onBeforeConnect | () => void | connect is called, before mic prompt or socket open. |
| onConnect | () => void | Socket opens and the initial settings payload is delivered. |
| onClose | (event?: any) => void | Socket closes (manual disconnect or remote). |
| onError | (error: unknown) => void | Unexpected client-side error (mic, playback, socket send). |
| onReconnecting | (attempt: number) => void | A reconnect attempt begins (1-based attempt number). Requires reconnect.enabled. |
| onReconnected | () => void | The socket reconnects and the stored settings are re-sent. |
| onBargeIn | () => void | A bargeIn flush actually happened (agent audio was cut short by the user speaking). |
| onServerError | (message: DeepgramVoiceAgentErrorMessage) => void | API reports a structured error (description + code). |
| onWarning | (message: DeepgramVoiceAgentWarningMessage) => void | Non-fatal warning (e.g. degraded audio quality). |
| Callback | Signature | Fired when |
| -------- | --------- | ---------- |
| onWelcome | (message: DeepgramVoiceAgentWelcomeMessage) => void | Agent returns the initial Welcome envelope. |
| onSettingsApplied | (message: DeepgramVoiceAgentSettingsAppliedMessage) => void | Settings are acknowledged by the agent. |
| onConversationText | (message: DeepgramVoiceAgentConversationTextMessage) => void | Transcript update (role + content) arrives. |
| onUserStartedSpeaking | (message: DeepgramVoiceAgentUserStartedSpeakingMessage) => void | Server-side VAD detects the user speaking. |
| onAgentThinking | (message: DeepgramVoiceAgentAgentThinkingMessage) => void | Agent reports internal reasoning state. |
| onAgentStartedSpeaking | (message: DeepgramVoiceAgentAgentStartedSpeakingMessage) => void | Response playback session begins (latency metrics included). |
| onAgentAudioDone | (message: DeepgramVoiceAgentAgentAudioDoneMessage) => void | Agent finishes emitting audio for a turn. |
| onAudio | (audioData: ArrayBuffer) => void | Raw binary audio chunk received from the socket. |
| onAudioConfig | (message: DeepgramVoiceAgentAudioConfigMessage) => void | The server announces the output audio format (the native player is restarted at that rate). |
| onPromptUpdated | (message: DeepgramVoiceAgentPromptUpdatedMessage) => void | Active prompt is updated (e.g. after updatePrompt). |
| onSpeakUpdated | (message: DeepgramVoiceAgentSpeakUpdatedMessage) => void | Active speak configuration changes (server-driven). |
| onListenUpdated | (message: DeepgramVoiceAgentListenUpdatedMessage) => void | Listen configuration is updated (e.g. after updateListen). |
| onThinkUpdated | (message: DeepgramVoiceAgentThinkUpdatedMessage) => void | Think configuration is updated (e.g. after updateThink). |
| onThinkLiveUpdated | (message: DeepgramVoiceAgentThinkLiveUpdatedMessage) => void | think_live behaviours are updated (after updateThinkLive). |
| onLatencyReport | (message: DeepgramVoiceAgentLatencyReportMessage) => void | Per-turn latency breakdown (stt_latency, ttt_*_latency, tts_latency, total_latency). |
| onFrustrationDetected | (message: DeepgramVoiceAgentFrustrationDetectedMessage) => void | think_live detected a frustrated user. |
| onCustomFromThinkProvider | (message: DeepgramVoiceAgentCustomFromThinkProviderMessage) => void | A custom think provider forwarded a payload. |
| onHistory | (message: DeepgramVoiceAgentHistoryMessage) => void | Conversation history (text turn or function calls) is replayed by the server. |
| onInjectionRefused | (message: DeepgramVoiceAgentInjectionRefusedMessage) => void | Inject request rejected (typically while the agent is speaking). |
| onMessage | (message: DeepgramVoiceAgentServerMessage) => void | Catch-all for every JSON message from the API. |
| Callback | Signature | Fired when |
| -------- | --------- | ---------- |
| onFunctionCallRequest | (message: DeepgramVoiceAgentFunctionCallRequestMessage) => void | Agent asks the client to execute a tool marked client_side: true. Respond with sendFunctionCallResponse. |
| onFunctionCallResponse | (message: DeepgramVoiceAgentReceiveFunctionCallResponseMessage) => void | Server shares the outcome of a non-client-side function call. |
| onFunctionCallCancelled | (message: DeepgramVoiceAgentFunctionCallCancelledMessage) => void | Pending calls were cancelled (e.g. the user kept talking); drop their results. |
Returned methods
Connection
| Method | Signature | Description |
| ------ | --------- | ----------- |
| connect | (settings?: DeepgramVoiceAgentSettings) => Promise<void> | Opens the socket, optionally merges settings, starts mic streaming. |
| disconnect | () => void | Tears down the socket, stops recording, removes listeners. |
| isConnected | () => boolean | Returns true when the socket is open. |
| mute | () => void | Stops forwarding mic audio while keeping the socket alive with periodic KeepAlive frames. |
| unmute | () => void | Resumes forwarding microphone audio. |
| getStats | () => SessionStats | Snapshot of the session telemetry (always available, never re-renders). |
Messaging
| Method | Signature | Description |
| ------ | --------- | ----------- |
| sendMessage | (message: DeepgramVoiceAgentClientMessage) => boolean | Send a pre-built client envelope (custom message types). |
| sendSettings | (settings: DeepgramVoiceAgentSettings) => boolean | Update settings mid-session. |
| sendMedia | (chunk: ArrayBuffer \| Uint8Array \| number[]) => boolean | Stream additional PCM (e.g. pre-recorded buffer). |
| sendKeepAlive | () => boolean | Emit a KeepAlive ping. |
| injectUserMessage | (content: string) => boolean | Inject a user-side text turn. |
| injectAgentMessage | (message: string, options?: DeepgramVoiceAgentInjectBehavior \| { behavior?; dynamic? }) => boolean | Make the agent speak. behavior: 'default' (refused mid-turn), 'queue' or 'interrupt'; dynamic: true lets the LLM phrase the reply from message. |
| forceEndTurn | () => boolean | End the user's turn now (push-to-talk; pairs with Flux eot_threshold: 1.0). |
| updatePrompt | (prompt: string) => boolean | Replace the active system prompt. |
| updateListen | (listen: DeepgramVoiceAgentListenConfig) => boolean | Swap the speech-recognition (listen) provider mid-session. |
| updateThink | (think: DeepgramVoiceAgentThinkConfig) => boolean | Swap the LLM (think) provider mid-session. |
| updateSpeak | (speak: DeepgramVoiceAgentSpeakConfig \| DeepgramVoiceAgentSpeakConfig[]) => boolean | Swap the text-to-speech (speak) provider mid-session. |
| updateThinkLive | (thinkLive: Record<string, unknown>) => boolean | Update the think_live behaviours mid-session. |
Function calls
| Method | Signature | Description |
| ------ | --------- | ----------- |
| sendFunctionCallResponse | (response: Omit<DeepgramVoiceAgentFunctionCallResponseMessage, 'type'>) => boolean | Return tool results for a client-side function call. |
Reactive state
Each value is undefined unless its corresponding track* flag is true.
| Return | Type | Requires |
| ------ | ---- | -------- |
| state | { connectionState: 'idle' \| 'connecting' \| 'connected' \| 'disconnected'; error: string \| null; warning: string \| null } | trackState: true |
| isMuted | boolean | trackState: true |
| conversation | Array<{ role: string; content: string }> | trackConversation: true |
| agentStatus | { thinking: string \| null; latency: { total?: number; tts?: number; ttt?: number } \| null } | trackAgentStatus: true |
| clearConversation | () => void | trackConversation: true |
| stats | SessionStats (bytesSent, bytesReceived, framesDropped, reconnects, connectedAtMs, firstResultMs) | trackStats: true |
Settings payload (DeepgramVoiceAgentSettings)
| Field | Type | Purpose |
| ----- | ---- | ------- |
| tags | string[] | Labels applied to the session for analytics/routing. |
| flags.history | boolean | Enable prior history playback to the agent. |
| audio.input | DeepgramVoiceAgentAudioConfig | Configure encoding/sample rate for microphone audio. |
| audio.output | DeepgramVoiceAgentAudioConfig | Choose output encoding/sample rate/bitrate for agent speech. |
| agent | DeepgramVoiceAgentAgentConfig \| string | Inline configuration, or the UUID of a reusable agent configuration stored in your Deepgram project. |
| agent.context.messages | DeepgramVoiceAgentContextMessage[] | Seed the conversation with prior turns or system notes. |
| agent.listen.provider | DeepgramVoiceAgentListenProvider | Speech recognition: version 'v1' (Nova: language, keyterms, smart_format) or 'v2' (Flux: eot_threshold, eager_eot_threshold, eot_timeout_ms, language_hints, keyterms). |
| agent.think.provider | DeepgramVoiceAgentThinkProvider | LLM selection (type, model, temperature, reasoning_mode, version, etc.). |
| agent.think.functions | DeepgramVoiceAgentFunctionConfig[] | Tooling exposed to the agent (name, parameters, optional endpoint, defer_until_eot). |
| agent.think.prompt | string | System prompt presented to the thinking provider. |
| agent.think_live | Record<string, unknown> | Conversational behaviours (backchannels, presence checks, frustration detection). |
| agent.speak | DeepgramVoiceAgentSpeakConfig \| DeepgramVoiceAgentSpeakConfig[] | Text-to-speech provider (Deepgram version 'v2' + flux-* model with speed / expressivity, or 'v1' + aura-*); an array lists fallbacks. Defaults to flux-kit-en. |
| agent.greeting | string | Optional greeting played once settings are applied. |
| mip_opt_out | boolean | Opt the session out of the Model Improvement Program. |
Speech-to-Text (useDeepgramSpeechToText)
The speech hook streams microphone audio over WebSocket and also handles prerecorded transcription. It defaults to STT v1 and auto-boots into Flux v2 when apiVersion: 'v2' is supplied (model defaults to flux-general-en).
💭 Flux v2 envelope handled correctly. The hook decodes Deepgram's
TurnInfoevents and treatsevent === 'EndOfTurn'as the final flag, so you don't have to reason about the new envelope yourself.Connected,ConfigureSuccess/ConfigureFailure, andErrormessages are surfaced via callbacks.
Live streaming quick start
const {
startListening,
stopListening,
state, // { status, error }
transcript, // "Hello world..."
} = useDeepgramSpeechToText({
trackState: true,
trackTranscript: true,
onTranscript: console.log,
live: {
apiVersion: 'v2',
model: 'flux-general-en',
eotThreshold: 0.55,
},
});
<Text>Transcript: {transcript}</Text>
<Button
title="Start"
onPress={() => startListening({ keyterm: ['Deepgram'] })}
/>
<Button title="Stop" onPress={stopListening} />💡 When you opt into
apiVersion: 'v2'the hook automatically selectsflux-general-enif you do not provide a model.
File transcription quick start
import * as DocumentPicker from 'expo-document-picker';
const { transcribeFile } = useDeepgramSpeechToText({
onTranscribeSuccess: (text) => console.log(text),
prerecorded: {
punctuate: true,
summarize: 'v2',
},
});
const pickFile = async () => {
const res = await DocumentPicker.getDocumentAsync({ type: 'audio/*' });
if (res.canceled) return;
const { uri, name, mimeType } = res.assets[0]!;
await transcribeFile({ uri, name, type: mimeType }, { topics: true, intents: true });
};API reference (Speech-to-Text)
Signature
const {
// Live streaming
startListening, stopListening, pause, resume,
// File transcription
transcribeFile,
// Reactive state (opt-in)
state, transcript, interimTranscript, isPaused, audioLevel, recordingUri,
} = useDeepgramSpeechToText(props);Hook props
Configuration
| Prop | Type | Description |
| ---- | ---- | ----------- |
| live | DeepgramLiveListenOptions | Default options merged into every live stream. |
| prerecorded | DeepgramPrerecordedOptions | Default options merged into every file transcription. |
| reconnect | DeepgramReconnectOptions | Auto-reconnect config for the live socket. Disabled unless reconnect.enabled is true. |
| metering | { enabled?: boolean; intervalMs?: number } | Microphone audio-level (VU meter) events. Disabled by default; set enabled: true to emit a normalized RMS level (~10 Hz, tune with intervalMs). |
| recordToFile | { enabled?: boolean; path?: string; format?: 'wav' } | Persist the captured mic audio to a WAV file while it streams. Disabled by default. Omit path to let the native module pick an app-specific location. |
Live streaming callbacks
| Callback | Signature | Fired when |
| -------- | --------- | ---------- |
| onBeforeStart | () => void | Before requesting mic permissions or starting a stream. |
| onStart | () => void | The WebSocket opens. |
| onTranscript | (transcript: string, event?: DeepgramTranscriptEvent) => void | Every transcript update (partial and final). |
| onTurnInfo | (message: DeepgramFluxTurnInfo) => void | Every Flux (v2) TurnInfo event, including StartOfTurn, EagerEndOfTurn and TurnResumed. |
| onMessage | (message: DeepgramListenServerMessage) => void | Every JSON message from the live socket (Results, UtteranceEnd, ConfigureSuccess, ConfigureFailure, Warning, ...). |
| onError | (error: unknown) => void | A streaming error occurs. |
| onAudioLevel | (level: number) => void | A new mic audio level (0..1 normalized RMS) is available. Requires metering.enabled. |
| onRecordingComplete | (uri: string) => void | A recordToFile session finished; receives the file:// URI of the saved WAV. Requires recordToFile.enabled. |
| onReconnecting | (attempt: number) => void | A reconnect attempt begins (1-based attempt number). Requires reconnect.enabled. |
| onReconnected | () => void | The live socket successfully reconnects. |
| onEnd | () => void | The socket closes (manual stop, or after reconnect attempts are exhausted). |
File transcription callbacks
| Callback | Signature | Fired when |
| -------- | --------- | ---------- |
| onBeforeTranscribe | () => void | Before posting a prerecorded request. |
| onTranscribeSuccess | (transcript: string) => void | The final transcript arrives. |
| onTranscribeError | (error: unknown) => void | The prerecorded request fails. |
Reactive tracking flags (opt-in)
| Prop | Default | Enables |
| ---- | ------- | ------- |
| trackState | false | state return — status + last error. |
| trackTranscript | false | transcript and interimTranscript returns. |
| trackStats | false | stats return — session telemetry, updated at most once per second. |
Returned methods
| Method | Signature | Description |
| ------ | --------- | ----------- |
| startListening | (options?: DeepgramLiveListenOptions) => Promise<void> | Requests mic access, starts recording, streams audio to Deepgram. |
| stopListening | () => void | Stops recording and closes the active WebSocket. |
| pause | () => void | Pauses mic streaming while keeping the socket alive via KeepAlive. On v1 a Finalize is sent first to flush buffered audio. |
| resume | () => void | Resumes mic streaming after a pause. |
| configure | (options: DeepgramLiveConfigureOptions) => boolean | Updates keyterms and numerals (Nova-3 and Flux) or Flux turn thresholds and languageHints mid-stream. The server replies with ConfigureSuccess / ConfigureFailure via onMessage. |
| forceEndTurn | () => boolean | Flux only: ends the current turn now (e.g. push-to-talk release). |
| getStats | () => SessionStats | Snapshot of the session telemetry (always available, never re-renders). |
| transcribeFile | (file: DeepgramPrerecordedSource, options?: DeepgramPrerecordedOptions) => Promise<void> | Uploads a file/URI/URL and resolves via the success/error callbacks. |
Reactive state
| Return | Type | Requires |
| ------ | ---- | -------- |
| state | { status: 'idle' \| 'loading' \| 'listening' \| 'transcribing' \| 'error'; error: Error \| null } | trackState: true |
| isPaused | boolean | trackState: true |
| audioLevel | number (0..1 normalized RMS) | trackState: true + metering.enabled |
| recordingUri | string (last saved file:// WAV) | trackState: true + recordToFile.enabled |
| transcript | string | trackTranscript: true |
| interimTranscript | string | trackTranscript: true |
| stats | SessionStats (bytesSent, bytesReceived, framesDropped, reconnects, connectedAtMs, firstResultMs) | trackStats: true |
Live transcription options (DeepgramLiveListenOptions)
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| apiVersion | 'v1' \| 'v2' | 'v1' | Selects the realtime API generation ('v2' unlocks Flux streaming). |
| model | DeepgramLiveListenModel | 'nova-3' (v1) / 'flux-general-en' (v2) | Streaming model. Flux also supports 'flux-general-multi' for multilingual turns. |
| version | string | – | Specific model version. |
| language | string | Auto | BCP-47 hint (e.g. en-US), or 'multi' for Nova-3 multilingual. |
| languageHint | string \| string[] | – | Bias language detection for Nova-3 language: 'multi' and flux-general-multi. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| encoding | DeepgramLiveListenEncoding | 'linear16' | Codec supplied to Deepgram. |
| sampleRate | number | 16000 | PCM sample rate. |
| channels | number | – | Channel count. |
| multichannel | boolean | false | Transcribe each channel independently. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| punctuate | boolean | false | Auto-insert punctuation/capitalisation. |
| smartFormat | boolean | false | Apply Deepgram smart formatting. |
| numerals | boolean | false | Convert spoken numbers to digits. |
| measurements | boolean | false | Convert measurements to abbreviations. |
| dictation | boolean | false | Dictation features. |
| fillerWords | boolean | false | Include "um"/"uh". |
| profanityFilter | boolean | false | Remove profanity. |
| detectEntities | boolean | false | Extract entities (names, places, etc.) into the results. |
| redact | DeepgramLiveListenRedaction \| DeepgramLiveListenRedaction[] | – | Remove PCI/PII. |
| replace | string \| string[] | – | Replace specific terms. |
| search | string \| string[] | – | Return timestamps for search terms. |
| keywords | string \| string[] | – | Boost or suppress keywords on legacy/non-Nova-3 models. |
| keyterm | string \| string[] | – | Bias Nova-3 and Flux with key terms. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| endpointing | number \| boolean | – | Endpoint detection (false to disable). |
| interimResults | boolean | false | Emit interim (non-final) transcripts. |
| utteranceEndMs | number | – | Delay before emitting utterance-end event. |
| vadEvents | boolean | false | Emit voice activity detection events. |
| diarizeModel | DeepgramLiveListenDiarizeModel | – | Diarisation model ('latest' / 'v1'). Separates speakers. |
Flux also accepts keyterm, numerals, profanityFilter, redact ('numbers' / 'aggressive_numbers'), mipOptOut and tag. Other v1 options are dropped from the URL.
| Option | Type | Purpose |
| ------ | ---- | ------- |
| eotThreshold | number | Confidence required to finalise a turn (0.5–1.0; 1.0 disables automatic end-of-turn). |
| eagerEotThreshold | number | Confidence required to emit an eager turn. |
| eotTimeoutMs | number | Silence timeout before closing a turn. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| callback | string | – | Webhook URL invoked when the stream finishes. |
| callbackMethod | 'POST' \| 'GET' \| 'PUT' \| 'DELETE' | 'POST' | HTTP verb for callback. |
| extra | Record<string, string \| number \| boolean> | – | Custom metadata returned with the response. |
| tag | string | – | Label the request for reporting. |
| mipOptOut | boolean | false | Opt out of the Model Improvement Program. |
Prerecorded transcription options (DeepgramPrerecordedOptions)
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| model | DeepgramPrerecordedModel | API default | Transcription model. |
| version | DeepgramPrerecordedVersion | 'latest' | Specific model version. |
| language | string | Auto | BCP-47 language hint, or 'multi' for Nova-3 multilingual. |
| languageHint | string \| string[] | – | Bias detection with Nova-3 language: 'multi'. |
| detectLanguage | boolean \| string \| string[] | false | Auto-detect language or restrict detection. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| encoding | DeepgramPrerecordedEncoding | – | Codec of the uploaded audio. |
| multichannel | boolean | false | Transcribe each channel independently. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| punctuate | boolean | false | Auto-insert punctuation. |
| smartFormat | boolean | false | Smart formatting. |
| numerals | boolean | false | Convert spoken numbers to digits. |
| measurements | boolean | false | Convert measurements to abbreviations. |
| paragraphs | boolean | false | Split transcript into paragraphs. |
| dictation | boolean | false | Dictation formatting. |
| fillerWords | boolean | false | Include filler words. |
| profanityFilter | boolean | false | Remove profanity. |
| redact | DeepgramPrerecordedRedaction \| DeepgramPrerecordedRedaction[] | – | Remove sensitive content. |
| replace | string \| string[] | – | Replace specific terms. |
| search | string \| string[] | – | Timestamps for search terms. |
| keywords | string \| string[] | – | Boost or suppress keywords on legacy/non-Nova-3 models. |
| keyterm | string \| string[] | – | Bias Nova-3 and Flux with key terms. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| diarizeModel | DeepgramPrerecordedDiarizeModel | – | Diarisation model ('latest' / 'v1' / 'v2'). Separates speakers. |
| utterances | boolean | false | Utterance-level timestamps. |
| uttSplit | number | – | Pause duration (s) used to split utterances. |
| detectEntities | boolean | false | Extract entities (names, places, etc.). |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| summarize | DeepgramPrerecordedSummarize | false | AI summary (true, 'v1', 'v2'). |
| topics | boolean | false | Detect topics. |
| customTopic | string \| string[] | – | Additional topics to monitor. |
| customTopicMode | DeepgramPrerecordedCustomMode | 'extended' | 'extended' (additive) or 'strict' (exact). |
| intents | boolean | false | Detect intents. |
| customIntent | string \| string[] | – | Custom intents. |
| customIntentMode | DeepgramPrerecordedCustomMode | 'extended' | 'extended' or 'strict'. |
| sentiment | boolean | false | Sentiment analysis. |
| Option | Type | Default | Purpose |
| ------ | ---- | ------- | ------- |
| callback | string | – | Webhook URL. |
| callbackMethod | DeepgramPrerecordedCallbackMethod | 'POST' | HTTP verb for callback. |
| extra | DeepgramPrerecordedExtra | – | Metadata returned with the response. |
| tag | string \| string[] | – | Label the request. |
| mipOptOut | boolean | false | Opt out of the Model Improvement Program. |
Text-to-Speech (useDeepgramTextToSpeech)
Generate audio via a single HTTP call or stream interactive responses over WebSocket. The hook exposes granular configuration for both request paths.
Aura vs Flux TTS
The hook picks the endpoint from the model name:
| | Aura (aura-*) | Flux TTS (flux-*) |
| - | - | - |
| Endpoint | /v1/speak | /v2/speak |
| Turn model | Text buffer; Flush emits audio | Flush ends the turn; per-turn SpeechStarted / SpeechMetadata |
| Barge-in | clearStream() sends Clear | interrupt() stops playback, cancels the turn and reports text_spoken / text_remaining |
| Mid-stream changes | – | configureStream({ speed }) |
| Extra options | – | expressivity (-2..2, beta), speed 0.5–1.5 |
Both stream raw linear16 / mulaw / alaw and accept Token or Bearer auth. Aura allows up to 20 Flush messages per 60 seconds. Flux closes a session after 60 s without client messages; React Native's WebSocket cannot send ping frames, so start a new stream after long pauses.
HTTP synthesis quick start
const { synthesize } = useDeepgramTextToSpeech({
options: {
http: {
model: 'aura-2-asteria-en',
encoding: 'mp3',
bitRate: 48000,
container: 'none',
},
},
onSynthesizeSuccess: (buffer) => {
console.log('Received bytes', buffer.byteLength);
},
});
await synthesize('Hello from Deepgram!');Streaming quick start
const {
startStreaming,
sendText,
flushStream,
clearStream,
closeStreamGracefully,
stopStreaming,
state, // { status, error }
} = useDeepgramTextToSpeech({
trackState: true,
autoPlayAudio: true, // Automatically play received audio
options: {
stream: {
model: 'aura-2-asteria-en',
encoding: 'linear16',
sampleRate: 24000,
autoFlush: false,
},
},
onAudioChunk: (chunk) => console.log('Audio chunk', chunk.byteLength),
onStreamMetadata: (meta) => console.log(meta.model_name),
});
await startStreaming('Booting stream…');
sendText('Queue another sentence', { sequenceId: 1 });
flushStream();
closeStreamGracefully();Flux TTS streaming with barge-in
const { startStreaming, sendText, flushStream, interrupt } =
useDeepgramTextToSpeech({
options: {
stream: { model: 'flux-haley-en', expressivity: 1, autoFlush: false },
},
onSpeechMetadata: (turn) => console.log(turn.billable_character_count),
onSpeechInterrupted: ({ text_spoken }) => trimLlmContext(text_spoken),
});
await startStreaming('Sure, ');
for await (const token of llmTokens) sendText(token); // sent verbatim
flushStream(); // ends the turn
// User talks over the agent:
interrupt({ playbackOffsetMs: playedMs });API reference
Signature
const {
// HTTP synthesis
synthesize, synthesizeToBytes,
// Streaming
startStreaming, sendText, sendMessage,
flushStream, clearStream, interrupt, configureStream,
closeStreamGracefully, stopStreaming,
// Reactive state (opt-in)
state,
} = useDeepgramTextToSpeech(props);Hook props
Configuration
| Prop | Type | Default | Description |
| ---- | ---- | ------- | ----------- |
| options | UseDeepgramTextToSpeechOptions | – | Defaults merged into HTTP and streaming requests. |
| autoPlayAudio | boolean | true | Auto-play received audio with the native player. |
| trackState | boolean | false | Enable state return. |
HTTP synthesis callbacks
| Callback | Signature | Fired when |
| -------- | --------- | ---------- |
| onBeforeSynthesize | () => void | Before dispatching the HTTP request. |
| onSynthesizeSuccess | (audio: ArrayBuffer) => void | The HTTP request succeeds. |
| onSynthesizeError | (error: unknown) => void | The HTTP request fails. |
Streaming callbacks
| Callback | Signature | Fired when |
| -------- | --------- | ---------- |
| onBeforeStream | () => void | Before opening the WebSocket. |
| onStreamStart | () => void | Socket is open and ready. |
| onAudioChunk | (chunk: ArrayBuffer) => void | Each PCM chunk arrives. |
| onStreamMetadata | (metadata: DeepgramTextToSpeechStreamMetadataMessage) => void | Aura stream metadata arrives. |
| onStreamFlushed | (event: DeepgramTextToSpeechStreamFlushedMessage) => void | Deepgram confirms a flush. |
| onStreamCleared | (event: DeepgramTextToSpeechStreamClearedMessage) => void | Deepgram confirms an Aura clear. |
| onStreamWarning | (warning: DeepgramTextToSpeechStreamWarningMessage) => void | Deepgram warns about the stream. |
| onSpeechStarted | (event: DeepgramTextToSpeechSpeechStartedMessage) => void | Flux: a turn's audio starts. |
| onSpeechMetadata | (event: DeepgramTextToSpeechSpeechMetadataMessage) => void | Flux: a turn finished (billing and timing). |
| onSpeechInterrupted | (event: DeepgramTextToSpeechSpeechInterruptedMessage) => void | Flux: answer to interrupt(). |
| onStreamMessage | (message: DeepgramTextToSpeechStreamResponseMessage) => void | Every JSON message (e.g. Flux Connected, ConfigureSuccess, SessionMetadata). |
| onStreamError | (error: unknown) => void | The WebSocket errors. |
| onStreamEnd | () => void | The stream closes (gracefully or otherwise). |
Returned methods
HTTP
| Method | Signature | Description |
| ------ | --------- | ----------- |
| synthesize | (text: string) => Promise<ArrayBuffer> | Single-shot REST call; resolves with the full audio buffer and, for raw linear16 output with autoPlayAudio, plays it. |
| synthesizeToBytes | (text: string, options?: DeepgramTextToSpeechHttpOptions) => Promise<{ data: ArrayBuffer; mimeType: string }> | Like synthesize but does not play the audio — returns the raw bytes + MIME type for saving/caching. Results are cached in-memory (LRU) keyed by text + format, so repeated identical prompts skip the network. Pass options to override the format per call. |
Streaming
| Method | Signature | Description |
| ------ | --------- | ----------- |
| startStreaming | (text: string) => Promise<void> | Open the WebSocket and queue the first message. |
| sendText | (text: string, options?: { flush?: boolean; sequenceId?: number }) => boolean | Queue additional text; can suppress auto-flush or set an Aura sequence id. Flux text is sent verbatim, Aura text is trimmed. |
| sendMessage | (message: DeepgramTextToSpeechStreamInputMessage) => boolean | Send a raw control message (Speak, Flush, Clear, Close, Interrupt, Configure). |
| flushStream | () => boolean | Aura: emit all buffered audio now. Flux: end the current turn. |
| clearStream | () => boolean | Aura: clear buffered text/audio. Flux: same as interrupt(). |
| interrupt | (options?: { playbackOffsetMs?: number }) => boolean | Flux: stop playback, cancel the turn and drop in-flight audio. |
| configureStream | (options: { speed?: number }) => boolean | Flux: change the speed mid-stream. |
| closeStreamGracefully | () => boolean | Drain outstanding audio, then close. |
| stopStreaming | () => void | Force-close and release resources. |
Reactive state
| Return | Type | Requires |
| ------ | ---- | -------- |
| state | DeepgramTextToSpeechState | trackState: true |
Configuration (UseDeepgramTextToSpeechOptions)
UseDeepgramTextToSpeechOptions has one section per transport plus shared query parameters.
| Option | Type | Applies to | Purpose |
| ------------- | ------------------------------------------------ | ---------------- | ----------------------------------------------------------------------------- |
| queryParams | Record<string, string \| number \| boolean> | Both | Shared query string parameters appended to all requests. |
| http | DeepgramTextToSpeechHttpOptions | HTTP | HTTP synthesis configuration. |
| stream | DeepgramTextToSpeechStreamOptions | Streaming | Streaming configuration. |
| Option | Type | Purpose |
| -------------- | -------------------------------------------------- | ----------------------------------------------------------------------------- |
| model | DeepgramTextToSpeechModel | Aura or Flux voice (flux-* uses /v2/speak).
