@rn-ai/litert

Nitro AI SDK 7 V4 provider for the official LiteRT-LM 0.14 SDK. Supports .litertlm text generation, streaming, tools/JSON, local file image/audio inputs, speculative decoding, and CPU/GPU/NPU preference with CPU fallback.

The official iOS CLiteRTLM.xcframework is fetched only from the URL and SHA-256 configured in native-artifacts.json. Android pins com.google.ai.edge.litertlm:litertlm-android:0.14.0 and builds with JDK 21.

Download and run a model

The native framework and the model are separate downloads. Install the native dependency while building the app, then use runtime download for model weights on both iOS and Android. This example uses a public small FunctionGemma 270M LiteRT-LM artifact:

import { generateText } from 'ai'
import { createOnDeviceRuntime } from '@rn-ai/core'
import { createLiteRTProvider, defineLiteRTModel } from '@rn-ai/litert'

const runtime = createOnDeviceRuntime()
const litert = createLiteRTProvider({
  runtime,
  models: {
    functions: defineLiteRTModel({
      source: {
        kind: 'remote',
        url: 'https://huggingface.co/litert-community/functiongemma-mobile-actions_q8_ekv1024.litertlm/resolve/82d0f654a6270c518d16c600edce3136221b3347/mobile-actions_q8_ekv1024.litertlm?download=true',
        sizeBytes: 284_426_240,
        sha256: '92109695f911d1872fa8ae07c1e3ff0ed70f2c3d1690d410ec6db8587c2ab409',
        fileName: 'functiongemma-mobile-actions-q8-ekv1024.litertlm',
      },
      contextTokens: 1024,
    }),
  },
})

const model = litert.languageModel('functions')
await model.control.download({
  onProgress: ({ fraction }) => console.log(`${Math.round(fraction * 100)}%`),
})
await model.control.prepare()
console.log((await generateText({ model, prompt: 'Say hello in one sentence.' })).text)

The runtime stores and reuses the verified .litertlm file in the app-private model cache. The user does not need to choose a local path. Use AbortController to cancel, model.control.remove() to delete the cache entry, and immutable Hugging Face revisions plus exact Git LFS metadata when adding other model choices. Check each model's license and device-memory requirements.