parse-json-stream

High-performance stream parser that parses and yields incomplete JSON values as they arrive from a stream.

Installation

npm install parse-json-stream

Quick start

import { parseStream } from "parse-json-stream";

const response = await fetch("/api/data");
if (!response.body) {
  throw new Error("The response has no body");
}

for await (const chunk of parseStream(response.body)) {
  console.log(chunk.value);   // Current part of the value
  console.log(chunk.segments); // ["users", 0, "name"]
  console.log(chunk.path);    // "$.users[0].name"
  console.log(chunk.pointer); // "/users/0/name"
  console.log(chunk.type);    // "string"
}

chunk.value is a string fragment. A value can produce multiple chunks with the same path. Append these fragments to reconstruct the complete value.

Reference

parseStream(stream, init?)

Parses a supported stream and returns an AsyncGenerator<JSONChunk> that yields parsed chunks.

• stream: The stream to parse.
• init.signal: An optional AbortSignal that stops parsing.

parseStream accepts:

• ReadableStream
• WebSocket
• EventSource
• AsyncIterable<string | Uint8Array | ArrayBuffer>

Stream data can contain strings, Uint8Array values, or ArrayBuffer values.

JSONChunk

interface JSONChunk {
  readonly value: string;
  readonly type: "string" | "number" | "boolean" | "null";
  readonly segments: Array<string | number>;
  readonly path: string;
  readonly pointer: string;
}

• value contains the current string fragment.
• type contains the JSON value type.
• segments contains object keys and array indexes.
• path contains the JSONPath location, such as $.users[0].name.
• pointer contains the JSON Pointer location, such as /users/0/name.

React

Import useJSONStream from parse-json-stream/react.

useJSONStream(bufferProcessor)

Buffers parsed chunks until the next browser animation frame, then calls bufferProcessor with this type:

(chunks: Array<JSONChunk>) => void

The callback can run more than once for a stream. Each call contains only the new chunks collected since the previous call. If React renders the component with a new callback, the next batch goes to the new callback.

useJSONStream returns:

• fetchJSONStream(url, options?): Fetches a web address and consumes its response stream. options accepts standard RequestInit fields, such as method, headers, and body, except signal.
• consumeJSONStream(stream, init?): Consumes a supported stream. init.signal is an optional AbortSignal.

fetchJSONStream and consumeJSONStream do not change until the hook unmounts so that you can add them to React dependency arrays.

useJSONStream processes one JSON stream at a time. Starting a new stream stops the current parser. Starting a new fetch also cancels the current fetch request. When the component unmounts, the hook stops all active work and discards buffered chunks that have not been processed.

fetchJSONStream

fetchJSONStream fetches a web address and consumes its response stream:

import { useEffect, useState } from "react";
import { useJSONStream } from "parse-json-stream/react";

function FetchJSONStream() {
  const [text, setText] = useState("");

  const { fetchJSONStream } = useJSONStream((chunks) => {
    const nextText = chunks.map((chunk) => chunk.value).join("");
    setText((currentText) => currentText + nextText);
  });

  useEffect(() => {
    fetchJSONStream("/api/data").catch(console.error);
  }, [fetchJSONStream]);

  return <div>{text}</div>;
}

consumeJSONStream

consumeJSONStream consumes a stream that the component already has:

import { useEffect, useState } from "react";
import { useJSONStream } from "parse-json-stream/react";

function ConsumeJSONStream({ stream }: { stream: ReadableStream }) {
  const [text, setText] = useState("");

  const { consumeJSONStream } = useJSONStream((chunks) => {
    const nextText = chunks.map((chunk) => chunk.value).join("");
    setText((currentText) => currentText + nextText);
  });

  useEffect(() => {
    consumeJSONStream(stream).catch(console.error);
  }, [consumeJSONStream, stream]);

  return <div>{text}</div>;
}

Testing the React hook

useJSONStream schedules bufferProcessor with requestAnimationFrame. A test that controls timers must run the pending animation-frame callback before it checks the received chunks.

Patch global JSON object

Import parse-json-stream/patch to add parseStream to the global JSON object:

import "parse-json-stream/patch";

for await (const chunk of JSON.parseStream(stream)) {
  console.log(chunk.value);
}

License

ISC © 2024