streamable-json

A hyper-fast, low-memory JSON streaming parser built on the Web Streams API. It parses JSON entirely on the fly by transforming streams of strings into path-annotated JSON events.

Features

• Blazing Fast: Uses efficient state machines with fast-paths for string and number parsing.
• Low Memory Footprint: Never loads the entire plain-text JSON structure or AST into memory.
• Web Streams Native: Built entirely around TransformStream interfaces for universal compatibility.
• Path-Aware: Emits JSON tokens alongside their exact object or array path (e.g., ["users", 0, "name"]).
• Pure TypeScript: Strongly typed tokens and events out of the box.

Requirements

Modern JavaScript engine supporting the Web Streams API (ReadableStream, WritableStream, TransformStream).

Installation

bun install streamable-json

Quick Start

Parse a JSON stream and react to structural tokens on the fly.

import { JsonTokenizer, JsonParser, ParserEvent } from 'streamable-json';

const jsonString = '{"user": {"name": "Alice", "age": 30}}';

// 1. Create a readable stream from your JSON source
const readable = new ReadableStream<string>({
    start(controller) {
        controller.enqueue(jsonString);
        controller.close();
    }
});

// 2. Set up a writer to process the incoming parsed tokens
const writable = new WritableStream({
    write(event) {
        // e.g. path: ["$", "user", "name"], value: "Alice"
        if (event.type === ParserEvent.VALUE_CHUNK && event.path.at(-1) === "name") {
            console.log(`Found name: ${event.value}`);
        }
    }
});

// 3. Pipe the source through the Tokenizer then the Parser
await readable
    .pipeThrough(new TransformStream(new JsonTokenizer()))
    .pipeThrough(new TransformStream(new JsonParser()))
    .pipeTo(writable);

How It Works

streamable-json splits the parsing lifecycle into two distinct Web Stream Transformer classes:

1. JsonTokenizer: Consumes string chunks and splits them into discrete Tokens (START_OBJECT, STRING_VALUE_START, NUMBER_CHUNK, etc.). It utilizes an internal state machine to interpret structural tokens while buffering number and string primitives.
2. JsonParser: Consumes the sequence of Tokens emitted by the JsonTokenizer. It maintains a stack of keys to determine the exact absolute path of the current cursor, yielding ParsedEvents that associate values to their deeply nested key path.

Documentation

For an advanced look at the types, token emitters, and parser internals, see the API Reference.