@runanywhere/web-llamacpp

Llama.cpp WASM backend for the RunAnywhere Web SDK — LLM, GGUF embeddings, VLM, LoRA, tools, and structured output in the browser.

Installation

npm install @runanywhere/[email protected] @runanywhere/[email protected]

See the Web SDK README for bundler configuration and cross-origin isolation headers.

Usage

import { RunAnywhere, SDKEnvironment } from '@runanywhere/web';
import { LlamaCPP } from '@runanywhere/web-llamacpp';

await RunAnywhere.initialize({
  environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
});
await LlamaCPP.register({ acceleration: 'auto' });
await RunAnywhere.completeServicesInitialization();

const stream = await RunAnywhere.generateStream({
  prompt: 'Write a haiku about local AI.',
  maxTokens: 64,
});
for await (const token of stream.stream) {
  renderToken(token);
}

See the Web SDK README for model lifecycle and Vite setup.

Support

• Web SDK documentation
• Discord
• [email protected]

License

RunAnywhere License. See LICENSE.