nitro-speech

⚠️ Work in Progress

This library is under active development. (Last version is stable)

Speech recognition for React Native, powered by Nitro Modules.

Table of Contents

• Installation
• Permissions
• Features
• Usage
  • Recommended: useRecognizer Hook
  • With React Navigation (important)
  • Alternative: Static Recognizer
• API Reference
• Requirements
• Troubleshooting

Installation

npm install @gmessier/nitro-speech react-native-nitro-modules
# or
yarn add @gmessier/nitro-speech react-native-nitro-modules
# or 
bun add @gmessier/nitro-speech react-native-nitro-modules

Expo

This library works with Expo. You need to run prebuild to generate native code:

npx expo prebuild

Note: Make sure New Arch is enabled in your Expo configuration before running prebuild.

iOS

cd ios && pod install

Android

No additional setup required.

Permissions

Android

The library declares the required permission in its AndroidManifest.xml (merged automatically):

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.VIBRATE" />

iOS

Add the following keys to your app's Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app needs microphone access for speech recognition</string>
<key>NSSpeechRecognitionUsageDescription</key>
<string>This app needs speech recognition to convert speech to text</string>

Both permissions are required for speech recognition to work on iOS.

Features

| Feature | Description | iOS | Android | |---------|-------------|-----|---------| | Real-time transcription | Get partial results as the user speaks, enabling live UI updates | ✅ | ✅ | | Auto-stop on silence | Automatically stops recognition after configurable inactivity period (default: 8s) | ✅ | ✅ | | Auto-finish progress | Progress callbacks showing countdown until auto-stop | ✅ | ❌ (TODO) | | Locale support | Configure speech recognizer for different languages | ✅ | ✅ | | Background handling | Auto-stop when app loses focus/goes to background | ✅ | Not Safe (TODO) | | Contextual strings | Domain-specific vocabulary for improved accuracy | ✅ | ✅ | | Repeating word filter | Removes consecutive duplicate words from artifacts | ✅ | ✅ | | Permission handling | Dedicated onPermissionDenied callback | ✅ | ✅ | | Haptic feedback | Optional haptics on recording start/stop | ✅ | ✅ | | Automatic punctuation | Adds punctuation to transcription (iOS 16+) | ✅ | Auto | | Language model selection | Choose between web search vs free-form models | Auto | ✅ | | Offensive word masking | Control whether offensive words are masked | Auto | ✅ | | Formatting quality | Prefer quality vs speed in formatting | Auto | ✅ |

Usage

Recommended: useRecognizer Hook

import { useRecognizer } from '@gmessier/nitro-speech';

function MyComponent() {
  const { 
    startListening, 
    stopListening, 
    addAutoFinishTime, 
    updateAutoFinishTime 
  } = useRecognizer({
    onReadyForSpeech: () => {
      console.log('Listening...');
    },
    onResult: (textBatches) => {
      console.log('Result:', textBatches.join('\n'));
    },
    onRecordingStopped: () => {
      console.log('Stopped');
    },
    onAutoFinishProgress: (timeLeftMs) => {
      console.log('Auto-stop in:', timeLeftMs, 'ms');
    },
    onError: (error) => {
      console.log('Error:', error);
    },
    onPermissionDenied: () => {
      console.log('Permission denied');
    },
  });

  return (
    <View>
      <TouchableOpacity onPress={() => startListening({ 
        locale: 'en-US',
        autoFinishRecognitionMs: 8000,
        contextualStrings: ['custom', 'words'],
        // Haptics (both platforms)
        startHapticFeedbackStyle: 'medium',
        stopHapticFeedbackStyle: 'light',
        // iOS specific
        iosAddPunctuation: true,
        // Android specific
        androidMaskOffensiveWords: false,
        androidFormattingPreferQuality: false,
        androidUseWebSearchModel: false,
      })}>
        <Text>Start Listening</Text>
      </TouchableOpacity>
      <TouchableOpacity onPress={stopListening}>
        <Text>Stop Listening</Text>
      </TouchableOpacity>
      <TouchableOpacity onPress={() => addAutoFinishTime(5000)}>
        <Text>Add 5s to Timer</Text>
      </TouchableOpacity>
      <TouchableOpacity onPress={() => updateAutoFinishTime(10000)}>
        <Text>Update Timer to 10s</Text>
      </TouchableOpacity>
    </View>
  );
}

With React Navigation (important)

React Navigation doesn’t unmount screens when you navigate — the screen can stay mounted in the background and come back without remounting. See: Navigation lifecycle (React Navigation).

Because of that, prefer tying recognition cleanup to focus state, not just component unmount. A simple approach is useIsFocused() and passing it into useRecognizer’s destroyDeps so recognition stops when the screen blurs. See: useIsFocused (React Navigation).

const isFocused = useIsFocused();
const { 
  // ...
} = useRecognizer(
  {
    // ...
  },
  [isFocused]
);

Alternative: Static Recognizer (Not Safe)

import { Recognizer } from '@gmessier/nitro-speech';

// Set up callbacks
Recognizer.onReadyForSpeech = () => {
  console.log('Listening...');
};

Recognizer.onResult = (textBatches) => {
  console.log('Result:', textBatches.join('\n'));
};

Recognizer.onRecordingStopped = () => {
  console.log('Stopped');
};

Recognizer.onAutoFinishProgress = (timeLeftMs) => {
  console.log('Auto-stop in:', timeLeftMs, 'ms');
};

Recognizer.onError = (error) => {
  console.log('Error:', error);
};

Recognizer.onPermissionDenied = () => {
  console.log('Permission denied');
};

// Start listening
Recognizer.startListening({
  locale: 'en-US',
});

// Stop listening
Recognizer.stopListening();

// Manually add time to auto finish timer
Recognizer.addAutoFinishTime(5000); // Add 5 seconds
Recognizer.addAutoFinishTime(); // Reset to original time

// Update auto finish time
Recognizer.updateAutoFinishTime(10000); // Set to 10 seconds
Recognizer.updateAutoFinishTime(10000, true); // Set to 10 seconds and refresh progress

⚠️ About dispose()

The Recognizer.dispose() method is NOT SAFE and should rarely be used. Hybrid Objects in Nitro are typically managed by the JS garbage collector automatically. Only call dispose() in performance-critical scenarios where you need to eagerly destroy objects.

See: Nitro dispose() documentation

API Reference

useRecognizer(callbacks, destroyDeps?)

A React hook that provides lifecycle-aware access to the speech recognizer.

Parameters

• callbacks (object):
  • onReadyForSpeech?: () => void - Called when speech recognition starts
  • onResult?: (textBatches: string[]) => void - Called every time when partial result is ready (array of text batches)
  • onRecordingStopped?: () => void - Called when recording stops
  • onAutoFinishProgress?: (timeLeftMs: number) => void - Called each second during auto-finish countdown
  • onError?: (message: string) => void - Called when an error occurs
  • onPermissionDenied?: () => void - Called if microphone permission is denied
• destroyDeps (array, optional) - Additional dependencies for the cleanup effect. When any of these change (or the component unmounts), recognition is stopped.

Returns

• startListening(params: SpeechToTextParams) - Start speech recognition with the given parameters
• stopListening() - Stop speech recognition
• addAutoFinishTime(additionalTimeMs?: number) - Add time to the auto-finish timer (or reset to original if no parameter)
• updateAutoFinishTime(newTimeMs: number, withRefresh?: boolean) - Update the auto-finish timer

SpeechToTextParams

Configuration object for speech recognition.

Common Parameters

• locale?: string - Language locale (default: "en-US")
• autoFinishRecognitionMs?: number - Auto-stop timeout in milliseconds (default: 8000)
• contextualStrings?: string[] - Array of domain-specific words for better recognition
• disableRepeatingFilter?: boolean - Disable filter that removes consecutive duplicate words (default: false)
• startHapticFeedbackStyle?: 'light' | 'medium' | 'heavy' - Haptic feedback style when microphone starts recording (default: null / disabled)
• stopHapticFeedbackStyle?: 'light' | 'medium' | 'heavy' - Haptic feedback style when microphone stops recording (default: null / disabled)

iOS-Specific Parameters

• iosAddPunctuation?: boolean - Add punctuation to results (iOS 16+, default: true)

Android-Specific Parameters

• androidMaskOffensiveWords?: boolean - Mask offensive words (Android 13+, default: false)
• androidFormattingPreferQuality?: boolean - Prefer quality over latency (Android 13+, default: false)
• androidUseWebSearchModel?: boolean - Use web search language model instead of free-form (default: false)
• androidDisableBatchHandling?: boolean - Disable default batch handling (may add many empty batches, default: false)

Requirements

• React Native >= 0.76
• New Arch Only
• react-native-nitro-modules

Troubleshooting

Android Gradle sync issues

If you're having issues with Android Gradle sync, try running the prebuild for the core Nitro library:

cd android && ./gradlew :react-native-nitro-modules:preBuild

License

MIT

TODO

• [ ] (Android) Timer till the auto finish is called
• [ ] (Android) Cleanup when app loses the focus