react-native-vision-camera-mlkit

A Nitro-based React Native Vision Camera v5 plugin that exposes high-performance Google ML Kit frame output features. v2 implements text recognition (OCR) and barcode scanning for live frames and static images. Other ML Kit vision features are post-v2 roadmap items.

The example app is intentionally heavy and demo-focused. For integration details, follow the documentation below.

Requirements

• React Native 0.86+
• iOS 15.1+ and Android SDK 24+
• react-native-vision-camera >=5 (Nitro-based v5, not v4)
• react-native-nitro-modules

Install Vision Camera and its Nitro dependency:

npm i react-native-vision-camera react-native-nitro-modules
cd ios && pod install

react-native-vision-camera has its own peer dependencies (currently react-native-nitro-image); follow the prompts from your package manager or the Vision Camera docs to install those too.

If you're using the live-frame hooks (useTextRecognition, useBarcodeScanning), you also need Vision Camera's frame processor dependencies, since those hooks run inside a useFrameOutput worklet:

npm i react-native-vision-camera-worklets react-native-worklets

// babel.config.js
module.exports = {
  plugins: [['react-native-worklets/plugin']],
};

The static-image APIs (processImageTextRecognition, processImageBarcodeScanning) don't use frame outputs, Worklets, or a camera. They do use the package's configured Nitro recognizers, so VisionCamera v5 and Nitro Modules remain installation requirements for the native package.

For Expo, follow the Vision Camera guide for camera permissions/setup: visioncamera.margelo.com/docs. This library's own Expo config plugin (selective ML Kit dependencies) is documented below.

Installation

npm install react-native-vision-camera-mlkit
# or
yarn add react-native-vision-camera-mlkit

cd ios && pod install

ML Kit Models Installation (Selective)

v2 implements two feature families: text recognition and barcode scanning. Both are enabled by default, including all five OCR language models. Disable OCR languages you do not need to reduce binary size. Configuration keys for post-v2 roadmap features remain reserved and disabled; enabling one does not provide a public JS/Nitro API in v2.

Both textRecognition and barcodeScanning are fully optional at compile time on both platforms: disabling one removes its native SDK from the build entirely instead of only gating it at runtime, so consumers who only need barcode scanning (or only text recognition) do not pay for the other's dependencies or binary size.

Android (Gradle)

In your app's android/build.gradle (root project), add:

ext["react-native-vision-camera-mlkit"] = [
  mlkit: [
    textRecognition: true,
    textRecognitionChinese: false,
    textRecognitionDevanagari: false,
    textRecognitionJapanese: false,
    textRecognitionKorean: false,
    barcodeScanning: true,
  ]
]

iOS (Podfile)

In your ios/Podfile, add a configuration hash before target:

$VisionCameraMLKit = {
  'textRecognition' => true,
  'textRecognitionChinese' => false,
  'textRecognitionDevanagari' => false,
  'textRecognitionJapanese' => false,
  'textRecognitionKorean' => false,
  'barcodeScanning' => true,
}

Expo (config plugin)

If you're using an Expo prebuild workflow (expo prebuild, EAS Build, or a custom dev client), a config plugin mirrors the same selective ML Kit flags into the generated android/build.gradle and ios/Podfile automatically, so you don't have to hand-edit native config. Add it to your app.json/ app.config.js plugins array with the same keys as above:

{
  "expo": {
    "plugins": [
      [
        "react-native-vision-camera-mlkit",
        {
          "textRecognition": true,
          "barcodeScanning": true
        }
      ]
    ]
  }
}

Any key you omit keeps the library's own default (see the Android/iOS sections above). Passing an unrecognized key throws at prebuild time instead of silently being ignored. Requires @expo/config-plugins (already a dependency of expo itself, so Expo projects have it for free). See docs/expo-plugin.md for platform-specific caveats (Android-only keys, Kotlin DSL support, idempotency).

Static-image processing accepts local files up to 25 MB encoded size. iOS accepts sources up to 25 megapixels and 10,000 pixels on either dimension, then downsamples them to at most 4 megapixels and 4,096 pixels per dimension during ImageIO decode. Android currently accepts decoded dimensions up to 4 megapixels and 4,096 pixels per dimension. Larger inputs are rejected before full decode to protect application disk, memory, and CPU resources.

Usage

API Docs

• Text Recognition API
• Barcode Scanning API
• Selective ML Kit Installation
• Expo Config Plugin
• Migrating from v1 to v2

Text Recognition (Frame Output)

import {
  useFrameOutput,
  CommonResolutions,
  Camera,
} from 'react-native-vision-camera';
import { useTextRecognition } from 'react-native-vision-camera-mlkit';

const { textRecognition } = useTextRecognition({
  language: 'LATIN',
  scaleFactor: 1,
  invertColors: false,
});

const frameOutput = useFrameOutput({
  targetResolution: CommonResolutions.VGA_16_9,
  pixelFormat: 'yuv',
  onFrame(frame) {
    'worklet';

    try {
      const result = textRecognition(frame, {
        outputOrientation: 'portrait',
      });
      console.log(result.text);
    } finally {
      frame.dispose();
    }
  },
});

<Camera
  device={device}
  isActive={true}
  outputs={[frameOutput]}
  constraints={[{ resolutionBias: frameOutput }]}
/>;

TextRecognitionOptions:

• language?: 'LATIN' | 'CHINESE' | 'DEVANAGARI' | 'JAPANESE' | 'KOREAN'
• scaleFactor?: number (0.9-1.0)
• invertColors?: boolean
• roi?: RegionOfInterest (crop processing to a rectangle of the frame; see Region of Interest)
• frameProcessInterval?: number (deprecated compatibility field; v2 does not use it to throttle processing)

TextRecognitionArguments:

• outputOrientation?: 'portrait' | 'portrait-upside-down' | 'landscape-left' | 'landscape-right' (iOS only)

Image Processing (Static Images)

Use processImageTextRecognition to analyze a local image without the camera (for example, an image picked from the gallery). Each call creates a Nitro text recognizer configured with the supplied options, then processes the image asynchronously.

import { processImageTextRecognition } from 'react-native-vision-camera-mlkit';

const result = await processImageTextRecognition(imageUri, {
  language: 'LATIN',
  orientation: 'portrait',
  invertColors: false,
});

console.log(result.blocks);

TextRecognitionImageOptions:

• language?: 'LATIN' | 'CHINESE' | 'DEVANAGARI' | 'JAPANESE' | 'KOREAN'
• orientation?: 'portrait' | 'portrait-upside-down' | 'landscape-left' | 'landscape-right' (overrides EXIF orientation; omit it to use image metadata)
• invertColors?: boolean
• scaleFactor?: number (0.9-1.0)
• roi?: RegionOfInterest (crop processing to a rectangle of the image; see Region of Interest)

Static images must be local and readable. On iOS, the target must be a regular, non-symlink file supplied as an absolute path or file:// URI. Android accepts an absolute path, file:// URI, or content:// URI (copied to a temporary cache file for processing). Remote URLs and platform-library schemes such as ph:// are not supported.

Barcode Scanning (Frame Output)

import {
  useFrameOutput,
  CommonResolutions,
  Camera,
} from 'react-native-vision-camera';
import { useBarcodeScanning } from 'react-native-vision-camera-mlkit';

const { barcodeScanning } = useBarcodeScanning({
  formats: ['QR_CODE', 'CODE_128'],
  enableAllPotentialBarcodes: true,
  scaleFactor: 1,
  invertColors: false,
});

const frameOutput = useFrameOutput({
  targetResolution: CommonResolutions.VGA_16_9,
  pixelFormat: 'yuv',
  onFrame(frame) {
    'worklet';

    try {
      const result = barcodeScanning(frame, {
        outputOrientation: 'portrait',
      });

      for (const barcode of result.barcodes) {
        console.log(
          barcode.formatName,
          barcode.valueTypeName,
          barcode.rawValue
        );
        if (barcode.value?.type === 'TYPE_URL') {
          console.log(barcode.value.data.url);
        }
      }
    } finally {
      frame.dispose();
    }
  },
});

<Camera
  device={device}
  isActive={true}
  outputs={[frameOutput]}
  constraints={[{ resolutionBias: frameOutput }]}
/>;

BarcodeScanningOptions:

• formats?: ('UNKNOWN' | 'ALL_FORMATS' | 'CODE_128' | 'CODE_39' | 'CODE_93' | 'CODABAR' | 'DATA_MATRIX' | 'EAN_13' | 'EAN_8' | 'ITF' | 'QR_CODE' | 'UPC_A' | 'UPC_E' | 'PDF417' | 'AZTEC')[]
• enableAllPotentialBarcodes?: boolean (Android only)
• scaleFactor?: number (0.9-1.0)
• invertColors?: boolean
• roi?: RegionOfInterest (crop processing to a rectangle of the frame; see Region of Interest)
• frameProcessInterval?: number (deprecated compatibility field; v2 does not use it to throttle processing)

Supported formats values:

| Value | Symbology | | ------------- | ----------- | | ALL_FORMATS | All types | | QR_CODE | QR Code | | AZTEC | Aztec | | PDF417 | PDF417 | | DATA_MATRIX | Data Matrix | | CODE_128 | Code 128 | | CODE_39 | Code 39 | | CODE_93 | Code 93 | | CODABAR | Codabar | | EAN_13 | EAN-13 | | EAN_8 | EAN-8 | | ITF | ITF | | UPC_A | UPC-A | | UPC_E | UPC-E | | UNKNOWN | Unknown |

BarcodeScanningArguments:

• outputOrientation?: 'portrait' | 'portrait-upside-down' | 'landscape-left' | 'landscape-right' (iOS only)

Barcode Scanning (Static Images)

Use processImageBarcodeScanning to analyze a local image without the camera. Each call creates a Nitro barcode scanner configured with the supplied options, then processes the image asynchronously.

import { processImageBarcodeScanning } from 'react-native-vision-camera-mlkit';

const result = await processImageBarcodeScanning(imageUri, {
  formats: ['QR_CODE', 'PDF417'],
  enableAllPotentialBarcodes: true,
  orientation: 'portrait',
  invertColors: false,
  scaleFactor: 1,
});

for (const barcode of result.barcodes) {
  console.log(barcode.formatName, barcode.valueTypeName, barcode.displayValue);
}

BarcodeScanningImageOptions:

• formats?: ('UNKNOWN' | 'ALL_FORMATS' | 'CODE_128' | 'CODE_39' | 'CODE_93' | 'CODABAR' | 'DATA_MATRIX' | 'EAN_13' | 'EAN_8' | 'ITF' | 'QR_CODE' | 'UPC_A' | 'UPC_E' | 'PDF417' | 'AZTEC')[]
• enableAllPotentialBarcodes?: boolean (Android only)
• orientation?: 'portrait' | 'portrait-upside-down' | 'landscape-left' | 'landscape-right' (overrides EXIF orientation; omit it to use image metadata)
• invertColors?: boolean
• scaleFactor?: number (0.9-1.0)
• roi?: RegionOfInterest (crop processing to a rectangle of the image; see Region of Interest)

BarcodeScanningResult includes:

• barcodes[] with format, formatName, valueType, valueTypeName, rawValue, displayValue, rawBytes, bounds, corners, isPotential
• value?: { type: TYPE_*, data: ... } for parsed payloads (WiFi, URL, SMS, Contact, Calendar Event, Driver License, etc.)

Feature Utilities

The package also exposes helpers from the plugin factory:

import {
  getFeatureErrorMessage,
  isFeatureAvailable,
  assertFeatureAvailable,
  getAvailableFeatures,
} from 'react-native-vision-camera-mlkit';

• getAvailableFeatures(): MLKitFeature[]
• isFeatureAvailable(feature: MLKitFeature): boolean
• assertFeatureAvailable(feature: MLKitFeature): void
• getFeatureErrorMessage(feature: MLKitFeature): string

Error Handling

Recognizer creation throws a setup error when OCR or barcode scanning is not enabled in Gradle/Podfile. Static Nitro calls reject invalid schemes, missing or unreadable files, and images the platform decoder cannot read. The package also exports these compatibility error-message constants:

• IMAGE_NOT_FOUND_ERROR
• INVALID_URI_ERROR
• IMAGE_PROCESSING_FAILED_ERROR
• UNSUPPORTED_IMAGE_FORMAT_ERROR

These constants are retained for source compatibility and user-facing fallback messages. Nitro rejection messages include platform-specific file details and are not guaranteed to equal these strings; do not branch on error.message.

Use the feature helpers to provide user-friendly configuration hints:

import {
  assertFeatureAvailable,
  MLKIT_FEATURE_KEYS,
} from 'react-native-vision-camera-mlkit';

assertFeatureAvailable(MLKIT_FEATURE_KEYS.TEXT_RECOGNITION);

Region of Interest

Every feature accepts an optional roi to crop processing to a rectangle of the frame/image before running ML Kit, reducing CPU/GPU work. Result coordinates (bounding boxes, corners) are always mapped back to full source coordinates, so overlays don't need to account for the crop.

type RegionOfInterest = {
  x: number;
  y: number;
  width: number;
  height: number;
  /** @default 'normalized' */
  unit?: 'normalized' | 'pixel';
};

By default x/y/width/height are normalized to the 0..1 range relative to the source frame/image (so x + width and y + height must each be <= 1); set unit: 'pixel' to use absolute pixel values instead.

const { textRecognition } = useTextRecognition({
  roi: { x: 0.25, y: 0.25, width: 0.5, height: 0.5 },
});

Performance

• Follow the Vision Camera performance guide
• Always call frame.dispose() exactly once for every frame delivered to onFrame, including skipped frames. A try/finally around the whole callback is the safest pattern; unreleased v5 frames can stall the camera buffer pool.
• frameProcessInterval is retained only for source compatibility and has no throttling effect in the Nitro recognizers. Throttle in onFrame by deciding whether to call the recognizer while still disposing every frame.
• useAsyncRunner() can offload genuinely independent heavy work to a separate thread, but avoid wrapping a single native plugin call in it - passing a plugin's worklet-callable method through nested worklet closures into a different Worklet Runtime is fragile and can throw at runtime.

iOS Orientation Notes (Text Recognition)

iOS camera sensors are fixed in landscape orientation. The frame buffer stays landscape-shaped even when the UI rotates, so ML Kit needs an explicit orientation hint to rotate text correctly. On iOS, pass outputOrientation to textRecognition(frame, { outputOrientation }) so ML Kit can map the buffer to upright text. Android handles rotation automatically.

Troubleshooting

⚠️ iOS Simulator (Apple Silicon) – Heads-up

On Apple Silicon Macs, building for the iOS Simulator (arm64) may fail after installing this package.

This is a known limitation of Google ML Kit, which does not currently ship an arm64-simulator slice for some iOS frameworks. The library works correctly on physical iOS devices and on the iOS Simulator when running under Rosetta.

Xcode 26.4 Build Failures (fmt / consteval)

If your iOS build fails in Pods with errors referencing fmt/include/fmt/base.h or FMT_USE_CONSTEVAL after updating to Xcode 26.4, see docs/xcode-26.4-fmt-consteval-workaround.md for the root cause and a post_install Podfile patch. This is a React Native / Xcode toolchain issue, not specific to this library.

Google ML Kit Vision Features Roadmap

| Feature | Release status | Intended platform | | --------------------------------- | ----------------- | ----------------- | | Text recognition v2 | Available in v2 | Android and iOS | | Barcode scanning | Available in v2 | Android and iOS | | Face detection | Post-v2 roadmap | Android and iOS | | Face mesh detection | Post-v2 roadmap | Android | | Pose detection | Post-v2 roadmap | Android and iOS | | Selfie segmentation | Post-v2 roadmap | Android and iOS | | Subject segmentation | Post-v2 roadmap | Android | | Document scanner | Post-v2 roadmap | Android | | Image labeling | Post-v2 roadmap | Android and iOS | | Object detection and tracking | Post-v2 roadmap | Android and iOS | | Digital ink recognition | Post-v2 roadmap | Android and iOS |

Roadmap entries are not implemented public APIs and are not commitments to a particular release.

Sponsor on GitHub

If this project helps you, please consider sponsoring its development

react-native-vision-camera-mlkit is provided as is and maintained in my free time.

If you’re integrating this library into a production app, consider funding the project.