payloadmesh

Adaptive, zero-runtime-dependency payload optimization for Node.js.

PayloadMesh chooses a compact representation per payload and, for repeated state updates, can automatically send deltas instead of retransmitting the full object. v0.2.0 adds stateful sessions, WebSocket integration, Express/Fastify response adapters, session sequence protection, and cumulative sequence benchmarking.

Why PayloadMesh

JSON is convenient, but repeated property names, text numbers, and full-object retransmission can waste bandwidth. Binary serializers reduce some of that cost, but they still usually serialize the complete object on every send.

PayloadMesh works one layer higher:

• chooses JSON, compact binary, gzip, or delta strategies automatically
• remembers previous state inside a session
• sends only changes when that is smaller
• detects out-of-order session packets with sequence numbers
• exposes wire-size and strategy telemetry
• integrates without runtime dependencies

Install

npm install payloadmesh

Requires Node.js 18+.

Quick start

import { encode, decode } from 'payloadmesh';

const packet = encode({
  productId: 1123,
  inventory: 42,
  location: 'Bentonville'
});

const value = decode(packet);

Stateful sessions

This is the recommended API for repeated updates.

import { createSession } from 'payloadmesh';

const sender = createSession();
const receiver = createSession();

const first = sender.encode({
  productId: 1123,
  inventory: 42,
  status: 'ACTIVE'
});

console.log(receiver.decode(first));

// PayloadMesh now knows the previous state and can choose a delta.
const second = sender.encode({
  productId: 1123,
  inventory: 41,
  status: 'ACTIVE'
});

console.log(receiver.decode(second));
console.log(sender.stats());

Sender and receiver maintain state independently. Session packets carry a 16-bit sequence number, so an out-of-order or skipped packet fails fast instead of silently applying a delta to the wrong base state.

Call session.reset() when starting a new logical stream or after reconnecting if both sides are not guaranteed to retain the same state.

WebSocket integration

PayloadMesh works with WebSocket-like objects such as the popular ws package without depending on it.

import { createWebSocketChannel } from 'payloadmesh';

const channel = createWebSocketChannel(ws, { mode: 'balanced' });

channel.onMessage((message) => {
  console.log('decoded:', message);
});

channel.send({ type: 'inventory', productId: 1123, inventory: 42 });
channel.send({ type: 'inventory', productId: 1123, inventory: 41 });

channel.send() returns the number of bytes sent. channel.stats() exposes message counts, total bytes, and delta usage.

Express integration

import express from 'express';
import { expressPayloadMesh } from 'payloadmesh';

const app = express();
app.use(expressPayloadMesh());

app.get('/inventory', (_req, res) => {
  res.payloadMesh({
    productId: 1123,
    inventory: 42,
    status: 'ACTIVE'
  });
});

The response content type is application/x-payloadmesh. By default, the adapter also emits x-payloadmesh-strategy and x-payloadmesh-savings headers. Disable those telemetry headers with { headers: false }.

Fastify integration

import Fastify from 'fastify';
import { fastifyPayloadMesh } from 'payloadmesh';

const app = Fastify();
await app.register(fastifyPayloadMesh);

app.get('/inventory', async (_request, reply) => {
  return reply.payloadMesh({
    productId: 1123,
    inventory: 42
  });
});

Get optimization stats

import { encodeWithStats } from 'payloadmesh';

const { buffer, stats } = encodeWithStats(data);

console.log(stats.selected);
// { strategy: 'binary', bytes: 824, encodeMs: 0.21 }

console.log(stats.savingsVsJsonPercent);

Benchmark a single payload

npx payloadmesh benchmark payload.json

With a previous state:

npx payloadmesh benchmark next.json --previous previous.json

Benchmark a real update sequence

Give the CLI a JSON array containing sequential states:

npx payloadmesh session-benchmark inventory-sequence.json

Example output from the repository's bundled 100-state inventory sequence:

States:                 100
Full JSON total:        47.70 KB
PayloadMesh total:      3.83 KB
Bandwidth saved:        43.87 KB
Savings vs full JSON:   91.97%
Delta packets selected: 99/100

This is a deliberately repetitive inventory-update example, not a universal performance claim. Savings depend on payload shape and how much changes between updates. Run the benchmark against your own production-like data before adopting a wire strategy.

Optimization modes

encode(data, { mode: 'cpu' });       // avoids gzip
encode(data, { mode: 'balanced' });  // default
encode(data, { mode: 'bandwidth' }); // tries compression more aggressively

balanced considers gzip for payloads of at least 4 KB and requires meaningful savings. bandwidth starts considering gzip at 512 bytes. cpu skips gzip entirely.

File CLI

npx payloadmesh encode payload.json payload.pmesh
npx payloadmesh inspect payload.pmesh
npx payloadmesh decode payload.pmesh restored.json

Current wire strategies

• JSON
• compact binary
• JSON + gzip
• compact binary + gzip
• delta + JSON
• delta + compact binary
• delta + JSON + gzip
• delta + compact binary + gzip

The binary format includes a per-message dictionary for object keys that repeat across records.

API

encode(value, options?)

Returns a Buffer containing a self-describing PayloadMesh packet.

decode(packet, options?)

Restores the original value. Standalone delta packets require { previous }.

createSession(options?)

Creates a stateful encoder/decoder with automatic previous-state management, packet sequencing, reset support, and cumulative statistics.

createWebSocketChannel(socket, options?)

Wraps a WebSocket-like object's send() and message events with a PayloadMesh session.

expressPayloadMesh(options?)

Express middleware that adds res.payloadMesh(value).

fastifyPayloadMesh(fastify, options?)

Fastify plugin that adds reply.payloadMesh(value).

encodeWithStats(value, options?)

Returns { buffer, stats }.

analyze(value, options?)

Benchmarks applicable strategies without returning the packet.

inspect(packet)

Returns packet version, session sequence, strategy, size, compression flag, and delta flag.

encodeBinary(value) / decodeBinary(buffer)

Low-level access to PayloadMesh's binary codec.

Supported values

The adaptive JSON path supports ordinary JSON-safe values. The binary path additionally preserves Buffer / Uint8Array, BigInt, Date, and undefined.

Circular structures and non-finite numbers are rejected.

Production notes

PayloadMesh is pre-1.0 and its binary wire format may evolve. It is designed for application transport, not long-term persistence. Stateful delta streams require both endpoints to share the same logical history; sequence numbers detect ordering problems, but applications should reset or renegotiate state after reconnects. Arrays are currently replaced as a whole when changed rather than patched element-by-element.

Roadmap

• automatic reconnect/resync checkpoints
• shared dictionaries across session messages
• array delta algorithms
• browser runtime
• streaming encoder/decoder
• schema learning
• strategy telemetry/learning
• optional integrations for additional transports

License

MIT