JSON-Off-Thread

High-performance, asynchronous JSON utilities for Node.js using native addon + RapidJSON. Provides both standard promise-based parse/stringify and streaming APIs with cancellation, backpressure, and advanced type policies.

Features

• Non-blocking stringify and parse off the main thread.
• stringifyStream(object, options) incremental emission of JSON string chunks.
• parseStream(jsonString, options) incremental emission of top-level or deep nested values.
• Backpressure via Promise return (automatic) or manual resume() control.
• Cancellation tokens (createStringifyToken + cancelStringify).
• BigInt policies: wrap | string | number | ignore.
• Date policies: wrap | iso | epoch.
• Precise UTC ISO date format: YYYY-MM-DDTHH:MM:SS.mmmZ.
• Deep streaming parse traversal (mode: 'deep').

Installation

    npm i json-off-thread

API

stringify(value, [options]) -> Promise

Options:

• pretty: boolean pretty-print output.
• token: number cancellation token id.

parse(jsonString) -> Promise

Standard full parse returning JS value.

stringifyStream(value, options) -> Promise

Emits chunks of the JSON string during serialization. Options:

• onChunk(chunk: string[, control]) required. If returns a Promise, traversal waits until it resolves (automatic backpressure). If backpressureManual: true, a second control argument with resume() is provided instead.
• chunkSize: number threshold (>=1024) in bytes before flushing.
• pretty: boolean pretty-print.
• token: number cancellation token.
• backpressureManual: boolean manual mode.
• bigIntStrategy: 'wrap'|'string'|'number'|'ignore'
• dateStrategy: 'wrap'|'iso'|'epoch'

Returns: Final aggregated JSON string (or rejects on cancel/error).

parseStream(jsonString, options) -> Promise

Streams parsed values. Options:

• onValue(value: any[, keyOrIndex, control]) required.
  • Default: value is a JSON string snippet. keyOrIndex is not provided.
  • If emitNative: true: value is the JS object/value. keyOrIndex is the property name (string) or array index (number) if applicable.
• emitNative: boolean default false. If true, converts JSON to JS objects on the main thread before emission.
• mode: 'top'|'deep' default top.
• backpressureManual: boolean manual resume control.
• token: number cancellation token.

Resolves with the full original JSON string when complete.

createStringifyToken() -> number

Generates a token for cancellation.

cancelStringify(tokenId)

Sets the token flag to abort current operation (applies to both streaming and non-streaming when token passed).

Backpressure Modes

1. Automatic: Return a Promise from onChunk / onValue; traversal pauses until resolve/reject.
2. Manual: Set backpressureManual: true. Your handler receives (chunk, control) or (valueJSON, control); call control.resume() to continue.

BigInt Policy Details

• wrap: Encoded as {"$bigint":"<digits>"} for round-trip safety.
• string: Direct JSON string digits.
• number: 64-bit number (potential precision loss for >53-bit values).
• ignore: Serialized as null.

Date Policy Details

• wrap: {"$date":"<epoch_ms>"} as string.
• iso: Precise UTC ISO string YYYY-MM-DDTHH:MM:SS.mmmZ.
• epoch: 64-bit integer milliseconds since epoch.

Deep Parse Streaming

When mode: 'deep', parseStream performs a depth-first traversal emitting nested values (objects/arrays emitted after their children). Useful for progressive loading or indexing.

Example

const addon = require('json-off-thread');

// Streaming stringify with promise backpressure
addon.stringifyStream(largeObj, {
    chunkSize: 8192,
    onChunk: (chunk) => {
        // write to socket
        return writeAsync(chunk); // returns Promise
    },
    bigIntStrategy: 'wrap',
    dateStrategy: 'iso'
}).then(final => console.log('Done len', final.length));

// Manual backpressure
addon.stringifyStream(largeObj, {
    chunkSize: 4096,
    backpressureManual: true,
    onChunk: (chunk, control) => {
        queue.push(chunk);
        if (queue.length < 10) control.resume(); // continue immediately
        else flushQueue().then(()=>control.resume());
    }
});

// Streaming parse deep
addon.parseStream(JSON.stringify(largeStructure), {
    mode: 'deep',
    onValue: (json, control) => {
        index(json);
        control && control.resume();
    },
    backpressureManual: true
});

Notes / Limitations

• Default parse streaming emits serialized JSON snippets. Use emitNative: true to receive live JS values (note: conversion happens on main thread).
• Deep mode order: children before parent containers.
• Cancellation checks occur between emissions; very large single values (huge strings) may not be interruptible mid-write.
• Backpressure sleep interval is 5ms; adjust if you need tighter latency.

Future Ideas

• Adjustable sleep/yield strategy (adaptive).
• Max depth / size guards for security.
• Pure streaming (no aggregation) option returning metadata.

License

MIT (add your chosen license here).