VolvoxAI

A zero-dependency deep-learning runtime for browsers, Node.js, and native Windows, Linux, macOS, and Android targets.

VolvoxAI runs compact graph packages without embedding a general-purpose ML framework. It supports WebNN, WebGPU, WASM SIMD, JavaScript CPU, native CPU, Vulkan, OpenGL, optional CUDA, Metal, and NNAPI integrations.

The repository is also a from-scratch textbook:

• Idea, Build, and Deep tracks
• Architecture
• Runtime design

Highlights

• One explicit inference lifecycle: Runtime → Model → CompiledModel → ExecutionContext → ExecutionResult.
• Stable named outputs on every backend. Host reads return caller-owned arrays; WebGPU results may also expose result-owned device buffers.
• Independent execution and decode contexts with immutable compiled model and weight revisions.
• Required or preferred backend policy with independent operator-fallback control and machine-readable reports.
• A single context-aware provider contract for built-in and external devices.
• A full profile with a retained Trainer for CPU, WebGPU, or strict WASM training.
• Inspectable model packages using graph.json and safetensors.
• Strict inference/training composition boundaries in JavaScript, WASM, and native builds.

Install

npm install volvoxai

For repository development:

npm install
npm run typecheck
npm run build:all

Release artifacts

The fixed browser release files for package version 0.3.0 are:

dist/0.3.0/volvoxai.js
dist/0.3.0/volvoxai.min.js
dist/0.3.0/volvoxai.full.js
dist/0.3.0/volvoxai.full.min.js
dist/0.3.0/volvoxai.wasm.js
dist/0.3.0/volvoxai.wasm.min.js
dist/0.3.0/volvoxai.wasm
dist/0.3.0/volvoxai.full.wasm

The standard JavaScript entry is inference-only and resolves the forward-only WASM sidecar. The full entry adds training and resolves volvoxai.full.wasm. The WASM-only JavaScript entry contains strict WASM inference and training but no CPU, WebNN, WebGPU, WGSL, or Node filesystem implementation.

Build all browser artifacts reproducibly with:

make build_web

Model packages

An inference package contains:

graph.json
model.safetensors

Every graph root, including named subgraphs, must carry the exact case-sensitive discriminator:

{
  "format": "volvox-graph/v1"
}

The loader rejects a missing or different discriminator before allocating weights or backend resources. Every node input must resolve to a declared graph input, a named weight, or an earlier node output.

Inference

import { VolvoxAI } from 'volvoxai';

const runtime = await VolvoxAI.createRuntime({
  backends: ['webnn', 'webgpu', 'wasm', 'cpu'],
  onDiagnostic(event) {
    console.debug(event.kind, event.report ?? event);
  },
});

const model = await runtime.loadModel(
  './models/my-model/model.safetensors',
);
const compiled = await model.compile({
  backend: {
    mode: 'prefer',
    order: ['webgpu', 'wasm', 'cpu'],
    operatorFallback: 'allow',
  },
});
const context = await compiled.createContext();

const result = await context.execute({
  images: new Float32Array(1 * 224 * 224 * 3),
});
const scores = await result.output('scores').read();

await result.close();
await context.close();
await compiled.close();
await model.close();
await runtime.close();

Runtime loading resolves graph.json beside the first safetensors URL. Pass graphUrl in the loader options when the graph is stored elsewhere; its basename must be graph.json or a named *.graph.json document.

Compilation pins an immutable topology and weight revision. Create multiple contexts from one compiled model for independent request or decode state. Each context serializes its own accepted operations, while different contexts may progress concurrently.

ExecutionResult owns a stable snapshot of every declared graph output. A result remains usable after later executions and after its context closes. Each read() returns a fresh typed array. A device result may expose deviceBuffer; that buffer remains owned by the result and must not be destroyed by the caller.

Use a strict policy when execution must stay on one provider:

const compiled = await model.compile({
  backend: {
    mode: 'require',
    backend: 'webgpu',
    operatorFallback: 'forbid',
  },
});

Backend selection finishes during compilation. Execution failure is reported and is never retried on another provider.

Training

Training is available only from the full and WASM-only profiles. Trainer owns gradients, optimizer slots, accumulation, and a private working revision. trainStep() mutates only that private revision. commit() atomically publishes it as a new Model weight revision; already compiled models and contexts remain pinned to their original revision.

import {
  ModelBuilder,
  VolvoxAI,
} from 'volvoxai/full';

const builder = new ModelBuilder();
const x = builder.input('x', [1, 4]);
const weight = builder.weight('projection', [4, 8], 'float32', {
  initializer: { type: 'xavierUniform', seed: 17 },
});
const logits = builder.addOp(
  'MatMul',
  { input: x, weight },
  { out: { name: 'logits', shape: [1, 8] } },
  {},
  { id: 'projection', wLayout: 'din' },
).out;
builder.outputs(logits);
const graph = builder.build();

const runtime = await VolvoxAI.createRuntime({ backends: ['cpu'] });
const model = runtime.createModel(graph);
const trainer = await VolvoxAI.createTrainer(model, {
  backend: 'cpu',
});

const step = await trainer.trainStep({
  inputs: { x: new Float32Array([1, 2, 3, 4]) },
  logitsTensor: 'logits',
  targets: new Int32Array([3]),
  trainableTensors: ['projection'],
  updateMode: 'adamw',
  optimizer: { learningRate: 1e-3, maxGradNorm: 1 },
});
await trainer.commit();

await trainer.close();
await model.close();
await runtime.close();

The same Trainer contract accepts backend: 'webgpu' or backend: 'wasm'. WASM training is strict and rejects an unsupported graph before mutating weights. There is no implicit publication: call commit() before compiling inference against the update, or rollback() to restore the last committed baseline. The full profile also exports training builders, checkpoints, gradient accumulation controls, LoRA helpers, and PTQ authoring tools. See model construction and training and the operation matrix.

WASM-only browser extensions

For a Manifest V3 extension, package one WASM-only JavaScript variant, the full sidecar, and the model:

vendor/volvoxai.wasm.min.js
vendor/volvoxai.full.wasm
model/graph.json
model/model.safetensors

import { VolvoxAI } from './vendor/volvoxai.wasm.min.js';

const runtime = await VolvoxAI.createRuntime({
  wasmUrl: chrome.runtime.getURL('vendor/volvoxai.full.wasm'),
});
const model = await runtime.loadModel(
  chrome.runtime.getURL('model/model.safetensors'),
  { graphUrl: chrome.runtime.getURL('model/graph.json') },
);

Extension pages need wasm-unsafe-eval in their content security policy. The WASM-only release has no dynamic import and contains no alternate backend. See Browser and Node runtime.

Native use

make build_native

./native/volvoxai --help
./native/volvoxai-full --help

The inference executable provides model-agnostic tensor execution. The full executable additionally provides training:

./native/volvoxai run models/tinystories_1m \
  --input tokens=models/tinystories_1m/tokens.i32 \
  --input positions=models/tinystories_1m/positions.i32 \
  --output logits=out.f32

Raw files use a storage suffix matching their declared dtype: .f32, .i32, .i8, or .u8. Outputs contain the complete declared tensor; applications select task-specific rows or slices. Model-specific tokenization, image decoding, generation, and postprocessing live under examples/.

Native releases use embedded shaders. For shader development, VOLVOXAI_SHADER_DIR may point to generated spv/, glsl/, gles/, and metal/ directories; VolvoxAI logs once when that external override is actually used.

Example models

Weights are not committed. Recreate the example packages from public sources:

make models_deps
make models_efficientdet
make models_tinystories
make validate_model_packages

See Models and exporters.

Repository layout

ts/core/             graph/data objects and runtime ownership
ts/ops/              operators, validation, and normalization
ts/backends/         backend providers and device resources
ts/training/         Trainer, autograd, optimizers, checkpoints, and PTQ
examples/            model-specific applications and integrations
shaders/             authoritative WGSL source
native/include/      public opaque inference/provider and full Trainer/PTQ C APIs
native/src/runtime/  runtime/model/context/result implementation
native/src/kernels/  portable and optimized CPU/WASM kernels
native/src/backends/ native device integrations
native/src/training/ full-profile training implementation
runtime/             optional in-process Synurang FFI plugin

Documentation

• Quickstart
• Browser and Node runtime
• Native runtime
• Backend SDK
• Model format
• Graph exporter and optimizer design
• Typed PTQ
• Model construction and training
• Operation support matrix
• Testing and validation
• Models and exporters
• Textbook

License

MIT. See LICENSE.