llguidance

Pure TypeScript constrained decoding for browsers and Node.js, based on guidance-ai/llguidance.

The dependency-free runtime supports regex constraints, json_object, and a practical JSON Schema 2020-12 profile. It does not claim full JSON Schema conformance or complete upstream llguidance API parity.

Install

pnpm add llguidance

Usage

import { loadBundledLLGuidance } from "llguidance";

const runtime = await loadBundledLLGuidance();
const interpreter = runtime.createInterpreter({
  // Use the exact tokenizer and token IDs used by the model.
  tokenizer: modelTokenizer,
  response_format: {
    type: "json_schema",
    json_schema: {
      type: "object",
      properties: {
        answer: { type: "string" },
      },
      required: ["answer"],
      additionalProperties: false,
    },
  },
});

const result = interpreter.computeMask();
if ("mask" in result) {
  // Apply result.mask to the model logits and sample an allowed token.
  const tokenId = sampleToken(result.mask, result.vocabSize);
  interpreter.commitToken(tokenId);
}

Each interpreter represents one mutable generation stream. Use separate interpreters for batch rows, beams, or forks.

Feature Bundles

The root entry includes both regex and JSON Schema engines plus the Python-style compatibility API. Applications that need only one constraint family can use a static feature subpath:

import { loadRegexGuidance } from "llguidance/regex";

const runtime = await loadRegexGuidance();

import { loadJsonGuidance } from "llguidance/json-schema";

const runtime = await loadJsonGuidance();

import { loadGrammarGuidance } from "llguidance/grammar";

const runtime = await loadGrammarGuidance();
const interpreter = runtime.createInterpreter({ tokenizer, grammar: lark });

The feature runtimes expose createTokenizer, createInterpreter, and get_version. The regex entry accepts only regex; the JSON entry accepts json_object and json_schema; the grammar entry accepts the documented practical Lark subset. These are independent static bundles with no dynamic imports, so excluded engines are not downloaded or parsed.

Response Formats

{
  type: "json_object";
}

{ type: "json_schema", json_schema: schema }

{ type: "regex", regex: "[a-z]+" }

See JSON_SCHEMA_PROFILE.md and REGEX_SUPPORT.md for exact support.

Tokenizers

The tokenizer vocabulary must match the model vocabulary exactly. Pass a compatible tokenizer object directly or create one from raw token bytes:

const tokenizer = runtime.createTokenizer({
  tokens: rawTokenBytesById,
  eos_token_id,
  bos_token_id,
  special_token_ids,
});

An EOS token ID is required for custom token arrays. See TOKENIZERS.md for supported tokenizer shapes and byte formats.

API

• loadBundledLLGuidance() returns the cached runtime.
• loadRegexGuidance(), loadJsonGuidance(), and loadGrammarGuidance() return cached lean feature runtimes.
• runtime.createTokenizer(config) creates a tokenizer handle.
• runtime.createInterpreter(config) creates one constrained-decoding stream.
• interpreter.computeMask() returns { mask, vocabSize } or { stop: true, reason: "accepted" | "dead_end" }.
• Interpreter configs accept a limits object with maxTokens, maxOutputBytes, maxRegexStates, maxRegexRepeat, maxSchemaDepth, and maxJsonDepth safety bounds.
• interpreter.commitToken(tokenId) advances the stream.
• Tokenizer, interpreter, and matcher handles expose idempotent free() and dispose().

The runtime also exposes selected Python-style classes and helpers. See API_MAPPING.md.

Documentation

• JSON Schema profile
• Regex support
• Grammar support
• Tokenizers
• Worker integration
• Architecture
• Compatibility with upstream llguidance
• Python-to-JavaScript API mapping
• Performance
• Development

The root entry intentionally does not include Lark support. Use the optional llguidance/grammar entry for its documented subset, or upstream llguidance when full grammar compatibility is required.

Development

pnpm install
pnpm dev

See DEVELOPMENT.md for build, test, typecheck, distribution, and benchmark commands.