llguidance

Browser and Node.js WASM bindings for guidance-ai/llguidance.

This package loads a bundled WebAssembly build of llguidance and exposes a small JavaScript runtime for constrained decoding with json_object and json_schema response formats. It also supports regex constraints.

Install

pnpm add llguidance

Basic Usage

import { loadBundledLLGuidance } from "llguidance";

const runtime = await loadBundledLLGuidance();

// Use the same tokenizer that your LLM uses for inference.
const tokenizer = modelTokenizer;

const interpreter = runtime.createInterpreter({
  // This can be a Hugging Face tokenizer object, a Transformers.js tokenizer,
  // or a tokenizer returned by runtime.createTokenizer().
  tokenizer,
  response_format: {
    type: "json_schema",
    json_schema: {
      type: "object",
      properties: {
        answer: { type: "string" },
      },
      required: ["answer"],
      additionalProperties: false,
    },
  },
});

const { mask, vocabSize } = interpreter.computeMask();

// Apply `mask` to your model logits, sample an allowed token, then commit it.
const tokenId = 0;
const result = interpreter.commitToken(tokenId);

Browser Bundlers And WASM URLs

The browser build uses a bundler-resolved WASM URL internally, so Vite/Rollup-style apps can usually call loadBundledLLGuidance() directly.

If your bundler needs explicit asset wiring, pass the WASM source yourself:

import { loadBundledLLGuidance } from "llguidance";
import wasmUrl from "llguidance/wasm/llguidance_wasm_bg.wasm?url";

const runtime = await loadBundledLLGuidance({ wasm: wasmUrl });

wasm may be a string, URL, Request, Response, BufferSource, or WebAssembly.Module accepted by wasm-bindgen's initializer. wasmUrl is also accepted as an alias for compatibility.

You can also load the wasm-bindgen JS factory yourself, including from a blob URL, CDN URL, service worker cache, or other application-managed source:

import { loadBundledLLGuidance } from "llguidance";

const [factorySource, wasmBytes] = await Promise.all([
  caches.open("models").then(async (cache) => (await cache.match("/llguidance_wasm.js")).text()),
  caches.open("models").then(async (cache) => (await cache.match("/llguidance_wasm_bg.wasm")).arrayBuffer()),
]);

const wasmFactoryUrl = URL.createObjectURL(
  new Blob([factorySource], { type: "text/javascript" }),
);

const runtime = await loadBundledLLGuidance({
  wasmFactoryUrl,
  wasm: wasmBytes,
});

If you already have the factory module namespace, pass it directly:

import { loadBundledLLGuidance } from "llguidance";
import * as wasmFactory from "llguidance/wasm/llguidance_wasm.js";

const runtime = await loadBundledLLGuidance({
  wasmFactory,
  wasm: wasmBytesOrUrl,
});

The tokenizer must be the same tokenizer used by the LLM during inference. llguidance constrains model token ids, so its vocabulary must match the model vocabulary exactly.

Tokenizers

If you already have a compatible tokenizer object, pass it directly to createInterpreter:

const interpreter = runtime.createInterpreter({
  tokenizer,
  response_format: { type: "json_object" },
});

The runtime will try to extract the raw token byte vocabulary from common Hugging Face tokenizer shapes, including Transformers.js tokenizer JSON and getVocab()/tokenToId() style APIs.

Hugging Face Tokenizers

Use the exact tokenizer instance used by your model. The token ids sampled by the LLM are the ids passed to interpreter.commitToken(tokenId), so the tokenizer vocabulary must match the model vocabulary exactly.

With Transformers.js, pass the loaded tokenizer directly. Transformers.js tokenizers are callable objects; llguidance supports that shape and will inspect the tokenizer's metadata without requiring a wrapper/facade.

import { AutoTokenizer } from "@huggingface/transformers";
import { loadBundledLLGuidance } from "llguidance";

const tokenizer = await AutoTokenizer.from_pretrained("Xenova/gpt2");
const runtime = await loadBundledLLGuidance();

const interpreter = runtime.createInterpreter({
  tokenizer,
  response_format: { type: "json_object" },
});

The bridge looks for common Transformers.js fields and methods on the callable tokenizer, including _tokenizerJSON, get_vocab(), token_to_id(), eos_token_id, bos_token_id, and all_special_ids.

With @huggingface/tokenizers, pass the tokenizer object if it exposes tokenizer JSON, getVocab(), or idToToken()/getVocabSize():

import { Tokenizer } from "@huggingface/tokenizers";
import { loadBundledLLGuidance } from "llguidance";

const tokenizer = await Tokenizer.fromFile("tokenizer.json");
const runtime = await loadBundledLLGuidance();

const interpreter = runtime.createInterpreter({
  tokenizer,
  response_format: {
    type: "json_schema",
    json_schema: {
      type: "object",
      properties: {
        answer: { type: "string" },
      },
      required: ["answer"],
      additionalProperties: false,
    },
  },
});

Internally, llguidance needs raw token bytes, not decoded text. The bridge handles common Hugging Face formats, including byte-level BPE tokens like Ġword, SentencePiece-style tokens like ▁word, byte fallback tokens like <0xFF>, and added/special tokens.

If automatic extraction fails, build the tokenizer config yourself and pass it to runtime.createTokenizer():

const llgTokenizer = runtime.createTokenizer({
  tokens: rawTokenBytesById,
  eos_token_id,
  bos_token_id,
  special_token_ids,
});

You can also create an llguidance tokenizer explicitly:

const llgTokenizer = runtime.createTokenizer({
  // Each entry must be the raw byte representation for the token id at that index.
  tokens: [
    [123], // "{"
    [125], // "}"
    [34],  // '"'
    [58],  // ":"
    [44],  // ","
  ],
  eos_token_id: 4,
});

const interpreter = runtime.createInterpreter({
  tokenizer: llgTokenizer,
  response_format: { type: "json_object" },
});

JSON Object Mode

const interpreter = runtime.createInterpreter({
  tokenizer,
  response_format: { type: "json_object" },
});

Regex Mode

Regex constraints use Rust regex syntax as accepted by upstream llguidance.

const interpreter = runtime.createInterpreter({
  tokenizer,
  response_format: {
    type: "regex",
    regex: "[a-z]+",
  },
});

API

loadBundledLLGuidance()

Loads the WASM module and returns a runtime object.

Options:

• wasm: WASM binary source passed to wasm-bindgen, such as a URL, Response, bytes, or WebAssembly.Module.
• wasmUrl: alias for wasm.
• wasmFactory: wasm-bindgen JS factory module namespace, promise, or function returning one.
• wasmFactoryUrl: URL for the wasm-bindgen JS factory module, including blob: URLs.

runtime.createTokenizer(config)

Creates an llguidance tokenizer environment.

Supported config fields:

• tokens: array of raw token byte arrays, indexed by token id.
• eos_token_id or eosTokenId: end-of-sequence token id.
• bos_token_id or bosTokenId: optional beginning-of-sequence token id.
• special_token_ids or specialTokenIds: optional list of special token ids.

runtime.createInterpreter(config)

Creates a constrained decoding interpreter.

Supported config fields:

• tokenizer: tokenizer returned by createTokenizer, or a compatible existing tokenizer object.
• response_format: { type: "json_object" }, { type: "json_schema", json_schema }, or { type: "regex", regex }.

interpreter.computeMask()

Computes the next-token constraint.

Returns one of:

• { mask, vocabSize }, where mask is a Uint32Array bitset.
• { stop: true } when the grammar is complete.
• { backtrack, ffTokens } when llguidance returns an unconditional fast-forward splice.

interpreter.commitToken(tokenId)

Commits the sampled token and advances the grammar state.

Returns { stop, backtrack, ffTokens }.

Development

Build a fast development WASM artifact:

pnpm run build:llguidance-wasm:dev

Build the release artifact and stage the npm package in dist/:

pnpm run build:dist

Publish with an npm one-time password:

pnpm run publish <otp> [patch|minor|major]

The publish script bumps the root package.json version, rebuilds dist/, and publishes the staged package to npm as llguidance.