@capgo/capacitor-llm

v8.1.3

Published

a day ago

Adds support for LLM locally runned for Capacitor

0High
0Medium
0Low

riderx

capacitor plugin native llm local gemma gemma4 gemma3 gemma3-270m gemma2-2b litert litert-lm apple-intelligence gpt

@capgo/capacitor-llm

On-device LLM support for Capacitor.

Current platform strategy:

iOS: Apple Intelligence by default, plus LiteRT-LM .litertlm custom models in SwiftPM integrations when the path ends in .litertlm or modelType: 'litertlm' is passed
Android: Gemini Nano system model on supported devices where it is already available, LiteRT-LM for .litertlm bundles, and a compatibility fallback for legacy MediaPipe .task models
Web: Gemma 4 web models through @mediapipe/tasks-genai

Documentation

The most complete plugin docs are available at capgo.app/docs/plugins/llm.

Compatibility

| Plugin version | Capacitor compatibility | Maintained | | -------------- | ----------------------- | ---------- | | v8.. | v8.. | ✅ | | v7.. | v7.. | On demand | | v6.. | v6.. | ❌ | | v5.. | v5.. | ❌ |

Note: The plugin major version follows the Capacitor major version. Use the version that matches your Capacitor installation.

Installation

npm install @capgo/capacitor-llm
npx cap sync

If you use the web implementation, also install the MediaPipe peer dependency:

npm install @mediapipe/tasks-genai

Model Setup

iOS

Recommended path:

Use Apple Intelligence with path: 'Apple Intelligence'
Requires iOS 26.0+

Custom iOS LiteRT-LM path:

Available only when the plugin is integrated into the iOS app through Swift Package Manager
Uses the official LiteRT-LM Swift API and prebuilt iOS xcframework for .litertlm models
Selected only when the path ends in .litertlm or modelType: 'litertlm' is passed
CocoaPods builds keep Apple Intelligence and the legacy MediaPipe .task compatibility path
Other custom iOS model types keep the legacy MediaPipe compatibility path for backward compatibility

Example:

import { CapgoLLM } from '@capgo/capacitor-llm';

await CapgoLLM.setModel({ path: 'Apple Intelligence' });
const chat = await CapgoLLM.createChat();

Android

Recommended path:

Use Gemini Nano with path: 'Gemini Nano' when the Android device already has it available through AICore
Use LiteRT-LM .litertlm bundles
Gemma 4 E2B and E4B are good default examples
Models are available from the public litert-community Hugging Face repos

Gemini Nano system model example:

await CapgoLLM.setModel({ path: 'Gemini Nano' });
const chat = await CapgoLLM.createChat();

Quickstart with a downloaded Gemma 4 model:

import { CapgoLLM } from '@capgo/capacitor-llm';

const result = await CapgoLLM.downloadModel({
  url: 'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it.litertlm?download=true',
  filename: 'gemma-4-E2B-it.litertlm',
});

await CapgoLLM.setModel({
  path: result.path,
  modelType: 'litertlm',
  maxTokens: 4096,
  topk: 40,
  temperature: 0.8,
});

const chat = await CapgoLLM.createChat();

Bundled asset example:

await CapgoLLM.setModel({
  path: '/android_asset/gemma-4-E2B-it.litertlm',
  modelType: 'litertlm',
  maxTokens: 4096,
});

Legacy compatibility:

Existing Android .task models still load through the compatibility path
New integrations should prefer .litertlm

Web

The web implementation uses @mediapipe/tasks-genai with web-ready model artifacts.

Gemma 4 web models are published next to the mobile LiteRT-LM bundles and use *-web.task.

Example:

await CapgoLLM.setModel({
  path: 'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it-web.task?download=true',
  modelType: 'task',
  maxTokens: 4096,
});

Usage

import { CapgoLLM } from '@capgo/capacitor-llm';

const { readiness } = await CapgoLLM.getReadiness();
console.log('LLM readiness:', readiness);

const { id } = await CapgoLLM.createChat();

await CapgoLLM.addListener('textFromAi', (event) => {
  console.log('chunk', event.text);
});

await CapgoLLM.addListener('aiFinished', ({ chatId }) => {
  console.log('finished', chatId);
});

await CapgoLLM.sendMessage({
  chatId: id,
  message: 'Explain why local inference is useful on mobile.',
});

Notes

Android now prefers LiteRT-LM and Gemma 4 style .litertlm bundles.
iOS LiteRT-LM custom-model support now uses the official LiteRT-LM Swift API and prebuilt iOS binaries, is available only in SwiftPM integrations of this plugin, and is selected only for explicit .litertlm models.
CocoaPods builds on iOS should use Apple Intelligence or the legacy MediaPipe .task compatibility path.
Web uses Gemma 4 *-web.task artifacts through @mediapipe/tasks-genai.
Apple Intelligence remains the preferred default on iOS where available.

LLM Plugin interface for interacting with on-device language models

createChat()

createChat() => Promise<{ id: string; instructions?: string; }>

Creates a new chat session

Returns: Promise<{ id: string; instructions?: string; }>

sendMessage(...)

sendMessage(options: { chatId: string; message: string; }) => Promise<void>

Sends a message to the AI in a specific chat session

| Param | Type | Description | | ------------- | ------------------------------------------------- | --------------------------------- | | options | { chatId: string; message: string; } | - The chat id and message to send |

getReadiness()

getReadiness() => Promise<{ readiness: string; }>

Gets the readiness status of the LLM

Returns: Promise<{ readiness: string; }>

setModel(...)

setModel(options: ModelOptions) => Promise<void>

Sets the model configuration

iOS: Use "Apple Intelligence" as path for the system model. Custom LiteRT-LM .litertlm models are supported on iOS only when this plugin is integrated through Swift Package Manager, and are selected only when modelType: 'litertlm' is passed or the path ends in .litertlm.
Android: Use "Gemini Nano" for the AICore system model on supported devices where it is already available. Prefer LiteRT-LM .litertlm bundles for custom models; legacy MediaPipe .task models are still supported
Web: Provide a web-ready model asset for @mediapipe/tasks-genai such as Gemma 4 *-web.task

| Param | Type | Description | | ------------- | ----------------------------------------------------- | ------------------------- | | options | ModelOptions | - The model configuration |

downloadModel(...)

downloadModel(options: DownloadModelOptions) => Promise<DownloadModelResult>

Downloads a model from a URL and saves it to the appropriate location

iOS: Downloads to the app's documents directory
Android: Downloads to the app's files directory

| Param | Type | Description | | ------------- | --------------------------------------------------------------------- | ---------------------------- | | options | DownloadModelOptions | - The download configuration |

Returns: Promise<DownloadModelResult>

addListener('textFromAi', ...)

addListener(eventName: 'textFromAi', listenerFunc: (event: TextFromAiEvent) => void) => Promise<{ remove: () => Promise<void>; }>

Adds a listener for text received from AI

| Param | Type | Description | | ------------------ | ------------------------------------------------------------------------------- | ----------------------------------- | | eventName | 'textFromAi' | - Event name 'textFromAi' | | listenerFunc | (event: TextFromAiEvent) => void | - Callback function for text events |