react-native-qnn-llm

Qualcomm lib Genie binding for React Native

Installation

npm install react-native-qnn-llm

Android Setup

Download and setup env

1. Download SDK from here
2. Setup env source path/to/QNN/SDK/bin/envsetup.sh

QNN libs

Add QNN libs dependency into android/app/build.gradle

dependencies {
+   implementation "com.qualcomm.qti:qnn-runtime:latest.integration@aar"

Or set useQnnSdkLib=true in android/gradle.properties

Notes: If you use dependency, the version should match QNN SDK version

Proguard

+ -keep class com.qnnllm.** { *; }

Usage

import { Context, SentenceCode } from 'react-native-qnn-llm';

const context = await Context.create(/* Genie config object */);
// Or load bundled
// const context = await Context.load({ bundle_path: 'path/to/bundle', unpack_dir: 'path/to/store/unpacked', n_thread?: Number })

await context.query('Hello, world!', (result, sentenceCode) => {
  console.log(result);
});

await context.save_session('path/to/session-directory');

await context.restore_session('path/to/session-directory');

await context.set_stop_words(['stop_word1', 'stop_word2']);

await context.apply_sampler_config({
  /* Genie sampler config */
});

await context.release();

Bundled File

To easier to deploy model, we announced packed file struct.

• Constant entry config path.
• Auto resolve file path.
• Patch config on load.

You can quickly pack your model files use pack.py.

Usage: pack.py path/to/config.json

Contributing

See the contributing guide to learn how to contribute to the repository and the development workflow.

License

MIT

Made with create-react-native-library