ruNNtime is a framework for running AI models on the user's own GPU, implemented using TypeGPU. It composes GPU kernels into the layers a model is made of, attention, convolutions, norms, matmuls and builds common architectures out of them. The kernels are plain TypeGPU code compiled to WGSL at runtime, so there is no WASM binary to download, and the same code runs anywhere WebGPU does: the browser, React Native and Electron.

It ships as one package, runntime, with two entry points:

• runntime/zoo - ready-to-run model implementations for commonly used tasks: speech, vision, and natural language processing. You call a single function, and get a running model.
• runntime/core - this package is not yet exposed to the public. It is a set of pre-built kernels for building neural networks on the Web, mirroring PyTorch semantics.

Table of Contents

• Key Features
• Quickstart
  • 1. Installation
  • 2. Set up the engine
  • 3. Run a model
• Documentation
• Migrating from transformers.js
• Created by Software Mansion

Key Features

• One function per task - createObjectDetector() hands you a loaded model. Call detectObjects(image), get boxes back. The zoo api is meant to be straight-forward and just work.
• No WASM to download - the kernels are TypeGPU code compiled to WGSL at runtime, so there is no WASM binary shipped alongside your app.
• Same code everywhere WebGPU runs - the browser, React Native (through react-native-webgpu) and Electron.
• Weights come from the Hub - every model points at a software-mansion repo. Turn on the OPFS cache and the second load skips the download.
• Drop-in for transformers.js - keep your pipeline() code, swap the backend underneath.

Quickstart

1. Installation

npm install runntime
# or
pnpm add runntime
# or
yarn add runntime

ruNNtime is built on TypeGPU, so you need it installed. The kernels are already compiled to shaders in the published package, so no bundler plugin is required. Follow the TypeGPU installation guide.

[!IMPORTANT] ruNNtime needs WebGPU:

• Web - Chrome, Edge, Safari 26 and newer, recent Firefox. See caniuse.
• Electron - ships Chromium, so the web setup works as is.
• React Native - through react-native-webgpu. Call its installWebGPU(), request the device yourself and wrap it with tgpu.initFromDevice({ device }).

Object detection, instance segmentation, pose, depth estimation and image classification run in half precision and need the shader-f16 feature. Text embedding, speech to text and the privacy filter can run on devices without it.

2. Set up the engine

Once per page, before you create any model:

import { initRunntime } from 'runntime/zoo';
import tgpu from 'typegpu';

initRunntime(await tgpu.init({ device: { optionalFeatures: ['subgroups', 'shader-f16'] } }));

Both features are optional, so the call succeeds on devices without them. Many kernels are much faster with subgroups. If your app already uses TypeGPU, pass the root you have.

3. Run a model

import { createObjectDetector, imageBufferFromImageData, models } from 'runntime/zoo';

// Load the model once. The weights download from the Hugging Face Hub.
const detector = await createObjectDetector(models.objectDetection.YOLO26.DEFAULT);

// Get the pixels, here from a canvas the picture is drawn on.
const ctx = canvas.getContext('2d');
const image = imageBufferFromImageData(ctx.getImageData(0, 0, canvas.width, canvas.height));

// Find the objects: one entry per object, best first.
const objects = await detector.detectObjects(image);
for (const { label, confidence, box } of objects) {
  console.log(label, confidence, box);
}
// bus 0.93 { format: 'xyxy', xmin: 6, ymin: 228, xmax: 807, ymax: 749 }
// person 0.92 { format: 'xyxy', xmin: 47, ymin: 399, xmax: 237, ymax: 902 }

// Free the GPU memory when done.
detector.dispose();

A video or a camera works the same way: draw each frame on the canvas and call detectObjects again. The detector is loaded once.

Models

Every task has its own factory function and its own methods. Pass it an entry from models, or spread one and change a field.

| Task | Factory | Models | | --------------------- | ------------------------- | ------------------------------------- | | Text embedding | createTextEmbedder | all-MiniLM-L6-v2 | | Privacy filter | createPrivacyFilter | OpenAI privacy-filter (int8) | | Speech to text | createSpeechToText | Moonshine tiny / base, streaming tiny | | Object detection | createObjectDetector | YOLO26 n / m | | Instance segmentation | createInstanceSegmenter | YOLO26-seg n / m | | Pose & keypoints | createKeypointDetector | YOLO26-pose n / m | | Depth estimation | createDepthEstimator | DepthART s / b | | Image classification | createImageClassifier | MobileNetV4 small |

Coming from transformers.js

Already on transformers.js and don't want to move your whole codebase? Keep your pipeline() code and swap the engine underneath. By default transformers.js runs on ONNX Runtime, which pays for its generality in speed. One initRunntimeBackend() call replaces that backend with ruNNtime. It only changes anything for the models ruNNtime supports, everything else keeps loading on ONNX Runtime.

import { initRunntimeBackend } from 'runntime/zoo/transformers';
import { pipeline } from '@huggingface/transformers';

await initRunntimeBackend({ fallbackToOnnx: true });

const pipe = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2');

Documentation

Full docs live at docs.swmansion.com/runntime - a guide per task, and some cool interactive examples you can run in the browser on your own GPU.

Created by Software Mansion

Since 2012, Software Mansion has been building mobile and web apps, contributing to open-source software, and dealing with all kinds of React Native and graphics challenges. We are Core React Native Contributors and the authors of TypeGPU. We can help you build your next AI product – Hire us.