@dikolab/parser

Compile a PARSEQ grammar document into a WebAssembly finite-state machine and parse text into a linear postorder (RPN) AST — one-shot or streaming, event-driven, and running the same in Node.js, Deno, and the browser.

This is the thin TypeScript wrapper over the Rust parser core (compiled to WASM with wasm-pack). It handles WASM loading for every runtime and re-exports the typed API.

Documentation

Full documentation is published on the project docs site:

• Overview — install, the WASM loading model, and a quick start.
• Grammar (PARSEQ) — the lexer + grammar document format you feed in.
• API documentation — every export, with signatures and examples.
• Examples — end-to-end one-shot and streaming usage.

Install

npm add @dikolab/parser          # npm / Node.js / bundlers
deno add jsr:@dikolab/parser     # Deno (JSR)

Quick start

The WebAssembly module must be instantiated once. Call init() and await it before touching any other export.

import { init, createStateMachine, parse } from "@dikolab/parser";

await init(); // load the WASM (Node fs / Deno fs / browser fetch — auto-detected)

const grammar = `%skip WS

%%lexer
NUMBER
PLUS
WS

%%%%

[0-9]+ := NUMBER
\\+ := PLUS
[ ]+ := WS

%%grammar
expr : expr PLUS NUMBER | NUMBER ;
`;

const fsm = createStateMachine(grammar); // compile once, reuse for many parses
const ast = parse("1 + 2 + 3", fsm); // Lexeme[]  (postorder RPN)
console.log(ast.at(-1)?.name); // "expr" — the accepted root

(For the full PARSEQ format — prelude directives, the lexer regex grammar, the BNF rules, and how the start/accept rule is chosen — see the Grammar (PARSEQ) reference.)

Runtimes

init() detects the host and loads parser_bg.wasm the right way:

• Node.js — reads the file with node:fs/promises.
• Deno — reads it with Deno.readFile.
• Browser (and any fetch-capable runtime) — streams it from the module URL.

The package ships as ES modules only.

API

Full reference with more examples: API documentation.

init(): Promise<void>

Instantiate the WASM module. Idempotent (loads once); await it before any other call.

class Parser

Compile a grammar once, then run it repeatedly — one-shot with parse, or streaming chunk-by-chunk with walk. Event handlers are registered only at construction, via the optional events map.

class Parser {
   constructor(source: string, events?: EventCallbackMap);
   parse(input: string): Lexeme[]; // one-shot; THROWS on error
   walk(chunk: string, last_chunk: boolean): WalkResult; // streaming; errors ride in the result
   reset(): void; // rewind the streaming cursor
   readonly completed: boolean; // true once streaming reached Accept
   free(): void; // release the WASM instance
}

One-shot functions

function createStateMachine(source: string): string; // PARSEQ -> reusable FSM envelope (JSON)
function parse(input: string, fsm: string): Lexeme[]; // tokenize + parse in one call

Types

interface Lexeme {
   type: "terminal" | "non_terminal";
   name: string;
   data: string | null; // absent fields are null (not undefined)
   child_count: number | null;
   rule_index: number | null; // non-terminal: which alternative reduced (1-based per non-terminal); terminal: null
   from_line: number;
   to_line: number;
   from_index: number;
   to_index: number;
}
interface WalkResult {
   pending: boolean;
   result: Lexeme | null;
   error: string | null;
}
interface EventCallbackMap {
   token?: (lexeme: Lexeme) => string | void; // override the token's data
   skipped?: (lexeme: Lexeme) => { name: string; data?: string } | void; // recover / discard
   shift?: (lexeme: Lexeme) => string | void;
   reduce?: (lexeme: Lexeme) => string | void;
   accept?: (lexeme: Lexeme) => void;
}

Streaming

Feed input a chunk at a time and pull the AST out one node per call — never holding the whole input (or tree) in memory.

import { init, Parser, type Lexeme } from "@dikolab/parser";
await init();

const p = new Parser(grammar, {
   reduce: (lx) => console.log("reduced", lx.name),
   accept: (lx) => console.log("root", lx.name),
});

const chunks = ["1 + ", "2 + ", "3"]; // arriving off a socket, say
const ast: Lexeme[] = [];
for (let i = 0; i < chunks.length; i++) {
   const last = i === chunks.length - 1;
   let r = p.walk(chunks[i]!, last); // feed this chunk
   while (r.result != null) {
      ast.push(r.result); // drain every node it produced…
      r = p.walk("", last); // …with empty-input calls
   }
   if (r.error != null) throw new Error(r.error); // walk reports; it never throws
   // result == null now: for a non-final chunk, r.pending is true → feed the next
}
console.log(p.completed); // true
p.reset(); // rewind to stream a fresh input; handlers stay registered

Two things to get right:

1. Drain on result, feed on pending — never loop on pending alone. While the tokenizer waits for the next chunk it returns { result: null, pending: true }, so a while (r.pending) loop spins forever. Drain with while (r.result != null), and only once result is null and pending is true do you feed the next chunk.
2. walk never throws. Tokenize/parse errors arrive in WalkResult.error (with pending = false) and latch until reset(). Only parse() throws — and new Parser(source) throws if the grammar fails to compile.

Support

If @dikolab/parser is useful to you, you can support its development:

Support development

License

MIT © 2026 Diko Consunji