xron-format

Lossless data serialisation achieving up to ~80% token reduction for LLM contexts.

Installation

npm install xron-format

Quick Start

import XRON from 'xron-format';

const data = [
  { id: 1, name: 'Alice', role: 'Engineer' },
  { id: 2, name: 'Bob',   role: 'Designer' },
  { id: 3, name: 'Carol', role: 'Engineer' },
];

const compressed = XRON.stringify(data, { level: 'auto' });
const restored  = XRON.parse(compressed);
// deep-equals the original data

Compression Levels

| Level | Typical Reduction | Best For | |-------|-------------------|----------| | 1 | ~60% | Human-readable output | | 2 | ~70% | Balanced compact format | | 3 | ~80% | Maximum token savings | | auto | Best available | Picks optimal level per dataset (Never-Worse Guarantee) |

9-Layer Pipeline

1. Schema extraction — removes repeated property keys
2. Positional streaming — eliminates key tokens in tabular data
3. Dictionary encoding — deduplicates repeated string values
4. Type-aware encoding — compresses booleans, nulls, dates, UUIDs
5. Column templates — extracts common prefix/suffix patterns
6. Substring dictionary — deduplicates repeated substrings
7. Delta + repeat compression — compresses sequential numbers
8. Separator reduction — minimises field separator overhead
9. Tokeniser alignment — optimises BPE token boundaries

Guarantees

• Lossless: XRON.parse(XRON.stringify(data)) deep-equals data
• Never-Worse: Auto mode compares all XRON levels against raw JSON.stringify and returns whichever is shortest — if XRON would be larger (e.g. tiny payloads), it falls back to JSON
• Native BigInt support
• Zero dependencies (tiktoken is an optional peer for token counting)

Links

• GitHub
• Full documentation

License

MIT