📦 LYT

High-performance binary compression and encryption powered by BPE tokenization (the same algorithm behind GPT-4).

Compress any text or JSON into a compact binary format using OpenAI's o200k_base tokenizer + Varint encoding. Optionally encrypt with AES-256-GCM.

Install

npm install @gsusrh/lyt

Quick Start

Compress & Decompress

import { pack, unpack } from '@gsusrh/lyt';

const data = { users: ["Alice", "Bob"], count: 2 };

// Compress → Uint8Array
const binary = pack(data);
console.log(`Compressed to ${binary.length} bytes`);

// Decompress → original object
const original = unpack(binary);
console.log(original); // { users: ["Alice", "Bob"], count: 2 }

Encrypt & Decrypt (Seal / Open)

import { seal, open } from '@gsusrh/lyt';

const secret = "my-secure-key";
const data = "Sensitive information";

// Compress + Encrypt → Uint8Array
const sealed = seal(data, secret);

// Decrypt + Decompress → original
const result = open(sealed, secret);
console.log(result); // "Sensitive information"

API

| Function | Description | |----------|-------------| | pack(data) | Compress any value (string, object, array) to Uint8Array | | unpack(buffer) | Decompress a Uint8Array back to the original value | | seal(data, key) | Compress + encrypt with AES-256-GCM | | open(buffer, key) | Decrypt + decompress |

How It Works

1. Tokenization — Text is converted to token IDs using OpenAI's o200k_base BPE encoder
2. Varint Encoding — Token IDs are stored as variable-length integers (small tokens = fewer bytes)
3. Encryption (optional) — AES-256-GCM with scrypt key derivation, random IV, and auth tag

Performance

BPE compression is especially effective on natural language text and repetitive JSON structures, achieving 50-70% size reduction compared to raw UTF-8.

Requirements

• Node.js ≥ 18
• Server-side only (uses Node.js crypto and Buffer)

License

ISC — Created by Jesus Rodriguez