webllm

⚠️ Experimental Release This is an experimental release of the WebLLM protocol. The API is subject to change and may not be stable for production use.

Universal npm bundle for WebLLM - Standard WebLLM protocol for browser integration.

Learn more at webllm.org

Why WebLLM?

Add AI to your web app without the backend complexity and costs. WebLLM enables Bring Your Own AI (BYOAI) - users configure their AI provider once, and your app just works.

Key Benefits for Developers

• 💰 Zero Infrastructure Costs - No servers, no API bills. Users bring their own AI (local models or their API keys)
• 🚀 SOTA Models - Access Claude, GPT-5, Llama, and more - not limited to one weak model
• 🔒 Privacy Control - Local processing option means data never leaves the user's device
• ⚡ Simple Integration - 3 lines of code to add AI features. No backend required
• 🔓 No Vendor Lock-in - Open standard, works with existing ecosystems like Vercel AI SDK
• 🎯 User Choice - Users control their AI provider, costs, and privacy settings

Use Cases

• SaaS dashboards with AI features (no backend costs)
• Browser extensions with built-in AI
• Content websites with smart reader tools
• Any web app that needs AI without server infrastructure

Installation

npm install webllm

Quick Start

Basic Text Generation

import { webLlmReady, generateText } from 'webllm';

// Wait for extension to be ready
await webLlmReady();

// Generate text - WebLLM routes to user's best provider
const result = await generateText({
  prompt: 'Explain my microwave in one sentence',
});

console.log(result.text);
console.log('Tokens used:', result.usage.totalTokens);

With Installation Prompt

import { promptInstall, generateText } from 'webllm';

try {
  // Show installation modal if extension not available
  await promptInstall();

  // Extension is now ready
  const result = await generateText({
    prompt: 'Write a haiku about coding',
    task: 'creative',
  });

  console.log(result.text);
} catch (error) {
  console.error('WebLLM not available:', error.message);
}

Chat Conversations

import { generateText } from 'webllm';

const result = await generateText({
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is quantum computing?' },
  ],
  task: 'qa',
  temperature: 0.7,
});

console.log(result.text);

Task-Specific Routing

import { generateText } from 'webllm';

// Translation - prioritize accuracy
const translation = await generateText({
  task: 'translation',
  hints: {
    quality: 'high',
    capabilities: { multilingual: true },
  },
  prompt: 'Translate to Spanish: Hello, how are you?',
});

// Coding - prefer capable models
const code = await generateText({
  task: 'coding',
  hints: {
    quality: 'best',
    capabilities: { codeGeneration: true },
  },
  prompt: 'Write a React hook for fetching data',
});

// Quick answer - prefer speed
const quick = await generateText({
  task: 'qa',
  hints: { speed: 'fastest' },
  prompt: 'What is 2+2?',
});

API Overview

Core Functions

// Text generation
generateText(options); // Generate text synchronously
streamText(options); // Stream text in real-time
generateObject(options); // Generate structured JSON output
streamObject(options); // Stream structured output

// Installation & availability
promptInstall(); // Show installation modal
webLlmReady(timeout); // Wait for extension to be ready
isAvailable(); // Check if extension is installed
getBrowserInfo(); // Get browser compatibility info

Type Exports

import type {
  Message,
  Task,
  ModelHints,
  Usage,
  LLMRequest,
  LLMResponse,
  GenerateTextOptions,
  GenerateTextResult,
  StreamTextOptions,
  StreamTextResult
} from 'webllm';

Browser Usage (CDN)

Using unpkg

<script src="https://unpkg.com/webllm"></script>
<script>
  // Available as global WebLLM object
  const { generateText, promptInstall } = WebLLM;

  async function generate() {
    await promptInstall();
    const result = await generateText({
      prompt: 'Hello, AI!',
    });
    console.log(result.text);
  }
</script>

Using jsdelivr

<script src="https://cdn.jsdelivr.net/npm/webllm"></script>
<script>
  WebLLM.generateText({ prompt: 'Hello!' }).then((result) => console.log(result.text));
</script>

Features

• ✅ Universal module support (ESM, CJS, IIFE)
• ✅ TypeScript support with full type definitions
• ✅ Tree-shakeable ES modules
• ✅ CDN-ready (unpkg, jsdelivr)
• ✅ Source maps included

Documentation

• Full Documentation - Complete guides and API reference
• GitHub Repository - Source code and issues

Contributing

Interested in contributing? See DEVELOPERS.md for development setup and build instructions.

License

MIT