react-native-fast-litert

A high-performance LiteRT library for React Native, built with Nitro Modules.

• ⚡ Powered by Nitro Modules
• 💨 Direct ArrayBuffer input and output
• 🔧 Uses the LiteRT 2.1.6 C++ CompiledModel runtime
• 🔄 Supports swapping out TensorFlow Models at runtime
• 🖥️ Prefers GPU acceleration through Metal on iOS and OpenCL/OpenGL on Android
• 📸 Easy VisionCamera integration

Migrating from v2

If you are upgrading from v2, see the Migration Guide for breaking changes and upgrade steps.

Installation

1. Add the npm packages:

   yarn add react-native-fast-litert react-native-nitro-modules

2. In metro.config.js, add tflite as a supported asset extension:

   module.exports = {
     // ...
     resolver: {
       assetExts: ['tflite', // ...
       // ...

   This allows you to drop .tflite files into your app and swap them out at runtime without rebuilding. 🔥
3. GPU acceleration is bundled and preferred by default. See GPU acceleration for Android vendor-library configuration.
4. Run your app (yarn android / npx pod-install && yarn ios)

LiteRT 2.1.6 requires iOS 15 or newer. Android supports armeabi-v7a, arm64-v8a, and x86_64; LiteRT 2.1.6 does not publish an Android x86 runtime.

Usage

1. Find a TensorFlow Lite (.tflite) model. There are thousands of public models on tfhub.dev.

2. Drag your model into your app's asset folder (e.g. src/assets/my-model.tflite)

3. Load the Model:

   // Option A: Standalone Function
   const model = await loadTensorflowModel(require('assets/my-model.tflite'))
   
   // Option B: Hook in a Function Component
   const plugin = useTensorflowModel(require('assets/my-model.tflite'))

4. Call the Model:

   const inputData: ArrayBuffer = ...
   const outputData = await model.run([inputData])
   console.log(outputData)

Loading Models

Models can be loaded from the React Native bundle via require(..), or any URI/URL (http://.. or file://..):

// Asset from React Native Bundle
loadTensorflowModel(require('assets/my-model.tflite'))
// File on the local filesystem
loadTensorflowModel({ url: 'file:///var/mobile/.../my-model.tflite' })
// Remote URL
loadTensorflowModel(
  { url: 'https://tfhub.dev/google/lite-model/object_detection_v1.tflite' }
)

Loading a Model is asynchronous since buffers need to be allocated. Make sure to handle errors when loading.

Input and Output data

TensorFlow uses tensors as input and output. Since TensorFlow Lite is optimized for fixed-size byte buffers, you are responsible for interpreting the raw data yourself.

Input and output values are passed as ArrayBuffer. To inspect tensor shapes, open your model in Netron.

For example, the object_detection_mobile_object_localizer_v1_1_default_1.tflite model on tfhub.dev has 1 input tensor and 4 output tensors:

In the description on tfhub.dev we can find the description of all tensors:

From that we know we need a 192 x 192 input image with 3 bytes per pixel (RGB).

Usage (VisionCamera)

If you're using this model with a VisionCamera Frame Processor, you need to convert the Frame to the model's expected input size. Use vision-camera-resizer to do the conversion:

import {
  Camera,
  useAsyncRunner,
  useFrameOutput,
} from 'react-native-vision-camera'
import { useResizer } from 'react-native-vision-camera-resizer'
import { useTensorflowModel } from 'react-native-fast-litert'

const objectDetection = useTensorflowModel(require('object_detection.tflite'))
const model =
  objectDetection.state === 'loaded' ? objectDetection.model : undefined

// 1. Create a resizer that converts Frames to 192x192x3 (RGB, uint8)
const { resizer } = useResizer({
  width: 192,
  height: 192,
  channelOrder: 'rgb',
  dataType: 'uint8',
})

// 2. Keep camera delivery responsive when inference is slower than capture.
const inferenceRunner = useAsyncRunner()

const frameOutput = useFrameOutput({
  pixelFormat: 'yuv',
  onFrame(frame) {
    'worklet'
    if (model == null || resizer == null) {
      frame.dispose()
      return
    }

    const accepted = inferenceRunner.runAsync(() => {
      'worklet'
      try {
        // 3. Resize on the GPU and retain the pooled GPUFrame until inference
        // has consumed its pixel buffer.
        const resized = resizer.resize(frame)
        try {
          const outputs = model.runSync([resized.getPixelBuffer()])

          // 4. Interpret outputs according to the model metadata.
          const detection_boxes = new Float32Array(outputs[0]!)
          const detection_classes = new Float32Array(outputs[1]!)
          const detection_scores = new Float32Array(outputs[2]!)
          const num_detections = new Float32Array(outputs[3]!)
          console.log(`Detected ${num_detections[0]} objects!`)
        } finally {
          resized.dispose()
        }
      } finally {
        frame.dispose()
      }
    })

    // The runner is busy. Drop this frame instead of blocking camera delivery.
    if (!accepted) {
      frame.dispose()
    }
  },
})

return <Camera device="back" isActive={true} outputs={[frameOutput]} {...otherProps} />

[!NOTE] Unlike v4, VisionCamera v5 no longer requires boxing the model with NitroModules.box(). Since v5 is built on Nitro Modules and uses react-native-worklets, worklets can access HybridObjects like the TFLite model directly.

The resizer configuration must exactly match model.inputs[0]. A model whose input is [1, 416, 416, 3] and float32 needs a 416 × 416, interleaved, float32 resizer output. A filename containing float16 often describes compressed weights, not the input tensor type; inspect model.inputs instead of deriving the data type from the filename.

GPU acceleration

loadTensorflowModel(...) and useTensorflowModel(...) default to ['gpu']. LiteRT compiles supported operations for Metal on iOS or OpenCL/OpenGL on Android, while unsupported operations remain on CPU. If GPU compilation is not safe or fails, model creation automatically retries on CPU.

// GPU preferred, CPU fallback
const model = await loadTensorflowModel(require('assets/my-model.tflite'))

// Explicitly require CPU
const cpuModel = await loadTensorflowModel(
  require('assets/my-model.tflite'),
  []
)

The platform-specific metal and android-gpu names remain aliases for gpu. The old core-ml and nnapi delegates are not part of LiteRT 2.x's CompiledModel API.

LiteRT's GPU accelerator supports only the compatible parts of a graph. Dynamic post-processing tensors can force a CPU fallback, while fully float16 activation graphs can fail on both backends. For mobile GPU models, prefer float32 input/output tensors with float16-compressed weights. Always configure the resizer from model.inputs; do not infer the tensor type from the filename.

On Android 12 and newer, GPU access can additionally require vendor native libraries to be declared in the app manifest.

Expo

Expo

Use the config plugin in your expo config (app.json, app.config.json or app.config.js) with enableAndroidGpuLibraries:

{
  "name": "my app",
  "plugins": [
    [
      "react-native-fast-litert",
      {
        "enableAndroidGpuLibraries": true
      }
    ]
  ]
}

By default, when enabled, libOpenCL.so will be included in your AndroidManifest.xml. You can also include more libraries by passing an array:

{
  "name": "my app",
  "plugins": [
    [
      "react-native-fast-litert",
      {
        "enableAndroidGpuLibraries": ["libOpenCL-pixel.so", "libGLES_mali.so"]
      }
    ]
  ]
}

[!NOTE] For Expo, remember to run prebuild if the library is not yet included in your AndroidManifest.xml.

Bare React Native

Add any needed entries to your AndroidManifest.xml:

<uses-native-library android:name="libOpenCL.so" android:required="false" />
<uses-native-library android:name="libOpenCL-pixel.so" android:required="false" />
<uses-native-library android:name="libGLES_mali.so" android:required="false" />
<uses-native-library android:name="libPVROCL.so" android:required="false" />

[!NOTE] Android does not officially support OpenCL, but most GPU vendors do.

The implementation follows the official LiteRT C++ CompiledModel API and bundles Google's prebuilt GPU accelerators.

Community Discord

Join the Margelo Community Discord to chat about React Native performance libraries.

Adopting at scale

This library is provided as is, I work on it in my free time.

This project is based on the original react-native-fast-tflite work by Marc Rousavy and contributors.

Contributing

1. Clone the repo
2. Make sure you have installed Xcode CLI tools such as gcc, cmake and python/python3. See the TensorFlow documentation on what you need exactly.
3. Run yarn bootstrap and select y on all iOS and Android related questions.
4. Open the example app and start developing
   • iOS: example/ios/TfliteExample.xcworkspace
   • Android: example/android

See the contributing guide to learn how to contribute to the repository and the development workflow.

License

MIT