ai-sdk-litert-lm
v0.1.2
Published
LiteRT-LM provider for the Vercel AI SDK: run .litertlm models on iOS, Android and in the browser with generateText and streamText.
Maintainers
Readme
ai-sdk-litert-lm
Run LLMs on-device from the Vercel AI SDK: generateText and streamText on iPhones, on Android phones and in the browser, with no server. It is a provider for LiteRT-LM, Google's runtime for on-device LLMs.
import { streamText } from 'ai'
import { litertlm } from 'ai-sdk-litert-lm'
const result = streamText({
model: litertlm('hf://litert-community/gemma-4-E2B-it-litert-lm'),
prompt: 'What can you do without a network connection?',
})
for await (const text of result.textStream) console.log(text)The first call downloads the model from Hugging Face, 2 to 3 GB, and keeps it for later calls. On an iPhone, react-native-litert-lm asks for an entitlement for a model this size, and that needs a paid Apple Developer account (Install). Tools and structured output work too. In the browser, a reasoning model's thoughts reach your code. On a phone, only thoughts the model writes at the start of its text, in <think> or <thinking> tags, do.
On a phone the provider runs on react-native-litert-lm. In a browser it runs on Google's @litert-lm/core with WebGPU. Your bundler picks the right one, so one Expo app can target iPhone, Android and the web with the same AI code.
Install
Expo (iOS, Android, web)
npx expo install ai-sdk-litert-lm ai react-native-litert-lm \
react-native-nitro-modules @ungap/structured-clone \
@stardazed/streams-text-encoding
# For the web target:
npx expo install @litert-lm/core react-dom react-native-webKeep the React Native version your Expo release expects. With 0.88.0 instead of Expo's 0.88.1, my iOS build crashed on launch.
Add the plugin to app.json:
{
"expo": {
"plugins": [
[
"react-native-litert-lm",
{ "ios": { "extendedVirtualAddressing": true, "increasedMemoryLimit": true } }
]
]
}
}The two iOS settings let an app load models of about 2 GB or more, such as Gemma 4 E2B. They need a paid Apple Developer account. For a smaller model, "plugins": ["react-native-litert-lm"] is enough.
The AI SDK needs a few web APIs that React Native doesn't have. Copy example/polyfills.js into your app and import it first:
import './polyfills'Then build a native app. Expo Go can't load native modules:
npx expo prebuild
npx expo run:android --variant releaseFor iOS, build the Xcode project that npx expo prebuild generates.
Bare React Native
Install ai-sdk-litert-lm, ai, react-native-litert-lm and react-native-nitro-modules, then follow the polyfill guide for bare apps, which also adds web streams. For a web build, use the Browser only install below.
Browser only
npm install ai-sdk-litert-lm ai @litert-lm/coreTry the example
example/ is one Expo app for all three targets. It has a small chat and a button that runs this package's checks with a real model on your device. On 2026-11-05 and 2026-11-06 I ran them on a Pixel 11 Pro, on an iPhone 18 Pro and in Chrome 156, and they all passed.
git clone https://github.com/john-rocky/ai-sdk-litert-lm
cd ai-sdk-litert-lm
npm install
npm pack # the example installs this package from the tarball
cd example
npm install
npx expo start --web # Browser
npx expo run:android --variant release # AndroidFor iOS, run npx expo prebuild -p ios --clean, set a bundle id your team can sign in the Xcode project it generates, and build that project, which is how I built it. The test results appear on the screen.
Documentation
- Choose a model: model ids, which file each platform gets, and where to find models
- Usage: preloading, tools, structured output, reasoning, long conversations, per-call options and errors
- Test results
Development
npm install
npm test
npm run typecheckLicense
MIT. Independent project, not affiliated with Google or Vercel.
