spd-lib-ts

SPD (Secure Packaged Data) — a post-quantum-hardened encrypted data format for Node.js.

Encrypt any JavaScript value (strings, numbers, objects, typed arrays, Maps, Sets, Dates, etc.) into a compressed, authenticated, tamper-proof container. Supports file storage, base64 string transport, chunked internet transfer, and streaming I/O for files larger than 2 GB.

Security model (v29)

| Layer | Algorithm | Notes | |---|---|---| | Key derivation | Argon2id (128 MiB, 6 iterations) | Raises brute-force cost against quantum search | | Key expansion | HKDF-SHA3-512 (domain-separated) | Separate AEAD and MAC keys from one master secret | | Encryption | XChaCha20-Poly1305 (256-bit key, per-entry) | 128-bit PQ security via Grover's algorithm | | Key commitment | CMT-4: SHA3-256(key ∥ nonce) prefix | Prevents partitioning oracle / multi-key attacks | | Authentication | HMAC-SHA3-512 (256-bit key, full-file) | 128-bit PQ security, domain-separated | | Name encryption | XChaCha20-Poly1305 (dedicated name key) | Entry names are never stored in plaintext | | Compression privacy | Pad-to-256B blocks before deflate | Mitigates CRIME/BREACH length-based oracle | | Salt wrapping | Argon2id + XChaCha20-Poly1305 | Encrypts the KDF salt under the passcode |

Key derivation chain:

1. Argon2id(passcode, random 16-byte salt) → 96-byte master secret
2. HKDF-SHA3-512(master, 'spd-aead-key-v1') → 32-byte AEAD key
3. HKDF-SHA3-512(master, 'spd-mac-key-v1') → 64-byte MAC key
4. Per-entry key: HKDF-SHA3-512(AEAD key, entry name) → 32-byte entry key

Performance (Node.js 22, Apple M3, standard profile)

| Operation | Throughput / Latency | |---|---| | Key derivation (Argon2id, 128 MiB) | ~380 ms (one-time per session) | | addData ~36 B | ~10k ops/s | | addData 10 KB | ~3k ops/s | | addData 100 KB | ~750 ops/s | | addData 1 MB | ~100 ops/s | | saveToFileStreaming / loadFromFileStreaming 10×10 KB | ~400 ms (dominated by KDF) | | getEntry (random-access lookup, binary file) | ~400 ms (dominated by KDF) | | addMany 1000 items (parallel workers) | ~1–2 s |

The dominant cost in all file operations is the single Argon2id call (~380 ms). CMT-4 commitment, compression padding, HKDF, and name encryption add < 5 µs per entry.

Install

npm install spd-lib-ts

Requires Node.js ≥ 18 and a C++ build toolchain for the argon2 native module.

Quick start

import { SPD } from 'spd-lib-ts';

const spd = new SPD();
await spd.setPassKey('MyStr0ng!Passphrase#2024');

await spd.addData('username', 'alice');
await spd.addData('apiKey', 'sk-abc123');
await spd.addData('config', { theme: 'dark', retries: 3 });

// Save to file (binary streaming format — recommended)
await spd.saveToFileStreaming('./vault.spd', 'MyStr0ng!Passphrase#2024');

// Load from file
const loaded = await SPD.loadFromFileStreaming('./vault.spd', 'MyStr0ng!Passphrase#2024');
const data = await loaded.extractData();
console.log(data.username); // 'alice'
loaded.destroy();

API reference

new SPD()

Creates a new empty SPD instance. You must call setPassKey() before adding or saving data.

spd.setPassKey(passcode: string): Promise<void>

Derives encryption and authentication keys from the passcode using Argon2id + HKDF. Must be called before any data operation.

Passcode requirements: minimum 12 characters, must contain at least 3 of: lowercase, uppercase, digits, special characters. Alternatively, a passphrase of 5+ space-separated lowercase words (e.g. from generatePassphrase()) is accepted.

await spd.setPassKey('MyStr0ng!Passphrase#2024');

spd.setKeyProfile(profile: 'standard' | 'paranoid'): void

Sets the Argon2id memory and time parameters before calling setPassKey.

| Profile | Memory | Iterations | Use case | |---|---|---|---| | 'standard' | 128 MiB | 6 | Default — interactive applications | | 'paranoid' | 2048 MiB | 16 | Long-term archive, server-side, offline |

spd.setKeyProfile('standard'); // default
spd.setKeyProfile('paranoid'); // maximum resistance

SPD.generatePassphrase(wordCount?: number): string

Generates a cryptographically random space-separated word passphrase using the EFF large wordlist. Default is 7 words.

const pass = SPD.generatePassphrase();       // 7 words, ~56 bits entropy
const pass = SPD.generatePassphrase(10);     // 10 words, ~80 bits entropy

spd.addData(name: string, value: unknown): Promise<void>

Encrypts and stores a single value. The name is sanitized (lowercased, non-alphanumeric chars replaced with _). Entry names are encrypted and never stored in plaintext (v29+).

Supported types: string, number, boolean, null, object, Array, Uint8Array, Uint16Array, Uint32Array, BigInt64Array, BigUint64Array, Float32Array, Float64Array, Map, Set, Date, RegExp, Error

await spd.addData('score', 42);
await spd.addData('tags', ['a', 'b', 'c']);
await spd.addData('created', new Date());
await spd.addData('raw', new Uint8Array([1, 2, 3]));
await spd.addData('lookup', new Map([['key', 'val']]));

spd.addMany(items: { name: string; data: unknown }[]): Promise<void>

Batch version of addData. Large batches (≥ 256 items) are encrypted in parallel using worker threads.

await spd.addMany([
  { name: 'firstName', data: 'Alice' },
  { name: 'age',       data: 30 },
  { name: 'prefs',     data: { darkMode: true } },
]);

spd.extractData(): Promise<Record<string, unknown>>

Decrypts and returns all stored entries as a plain object.

const data = await spd.extractData();
console.log(data.firstName); // 'Alice'

spd.saveToFileStreaming(path: string, passcode: string): Promise<void>

SPD.loadFromFileStreaming(path: string, passcode: string): Promise<SPD>

Recommended for all file I/O. Writes/reads the binary SPD wire format. Heap usage stays bounded regardless of file size. Required format for getEntry random-access lookups.

The file embeds an offset index tail (SPDx) for O(1) entry lookups without a sidecar file.

await spd.saveToFileStreaming('./large.spd', 'MyStr0ng!Passphrase#2024');
const spd = await SPD.loadFromFileStreaming('./large.spd', 'MyStr0ng!Passphrase#2024');

spd.saveToFile(path: string, passcode: string): Promise<void>

SPD.loadFromFile(path: string, passcode: string): Promise<SPD>

JSON-envelope file format. Suitable for files up to ~1–2 GB. Not compatible with getEntry.

await spd.saveToFile('./secrets.spd', 'MyStr0ng!Passphrase#2024');
const spd = await SPD.loadFromFile('./secrets.spd', 'MyStr0ng!Passphrase#2024');

SPD.getEntry(filePath: string, passcode: string, entryName: string): Promise<unknown>

Random-access lookup of a single entry from a binary SPD file (written by saveToFileStreaming). Uses the embedded SPDx index tail to jump directly to the entry — no full scan. MAC-verified before decryption.

const value = await SPD.getEntry('./vault.spd', 'MyStr0ng!Passphrase#2024', 'username');

spd.extractDataStreaming(tmpDir: string, callback): Promise<void>

Memory-efficient extraction using .ssf skeleton files. For each encrypted entry:

1. Decrypts + decompresses the entry into <tmpDir>/<name>.ssf (mode 0600)
2. Reads the .ssf file and calls your callback with the name and value
3. Deletes the .ssf file before moving to the next entry

At most one entry's bytes exist on disk or in RAM at a time.

import * as fs from 'fs';

fs.mkdirSync('/tmp/spd_scratch', { recursive: true });

await spd.extractDataStreaming('/tmp/spd_scratch', async (name, value) => {
  console.log(name, value);
});

tmpDir must exist before calling. Mode 0700 is recommended.

SPD.extractFromFileStreaming(filePath, passcode, tmpDir, callback): Promise<void>

True constant-memory extraction from a binary .spd file (written by saveToFileStreaming). Processes one entry at a time using .ssf skeleton files — the full plaintext never lives in RAM.

Peak RAM usage is proportional to the largest single entry, not the file size.

await SPD.extractFromFileStreaming(
  './huge.spd',
  'MyStr0ng!Passphrase#2024',
  '/tmp/spd_scratch',
  async (name, value) => {
    console.log(name, typeof value);
  }
);

spd.saveData(passcode: string): Promise<string>

Serializes the payload to a base64 string for in-memory or network transport.

const b64 = await spd.saveData('MyStr0ng!Passphrase#2024');

SPD.loadFromString(data: string, passcode: string): Promise<SPD>

Loads from a base64 string produced by saveData.

const spd = await SPD.loadFromString(b64, 'MyStr0ng!Passphrase#2024');

Chunked internet transfer

Split a payload into chunks for HTTP uploads or any transport with a body-size limit.

// Sender — chunk size is chosen automatically
const chunks = await spd.saveDataChunked('MyStr0ng!Passphrase#2024');
// chunks is string[] — send each element, then the manifest (last element) last

// Override chunk size when you have a strict body limit (e.g. 256 KB per request)
const chunks = await spd.saveDataChunked('MyStr0ng!Passphrase#2024', 256 * 1024);

// Receiver — pass all chunks in order including the manifest
const spd = await SPD.loadFromChunks(chunks, 'MyStr0ng!Passphrase#2024');

The last element of the array is always a JSON manifest ({ totalChunks, chunkSize, totalBytes, version }). The receiver validates chunk count and byte count before decrypting.

Auto chunk size (when chunkSize is omitted): targets ~32 chunks, clamped between 64 KB and 8 MB, rounded to the nearest 64 KB boundary.

spd.changePasscode(oldPasscode: string, newPasscode: string): Promise<void>

Re-derives keys under the new passcode and re-encrypts all data. Verifies the old passcode via MAC before doing anything.

await spd.changePasscode('MyStr0ng!Passphrase#2024', 'EvenStr0nger!Pass#9999');

spd.setHash(hash: 'sha3-512' | 'sha256' | 'sha512'): void

Sets the hash algorithm used for per-entry integrity checks. Default is 'sha3-512'. Must be set before adding data. The chosen algorithm is embedded in the payload and restored automatically on load.

spd.setHash('sha3-512'); // default, recommended

spd.setCompressionLevel(level: number): void

Sets the zlib deflate compression level (1–9). Default is 6 (balanced speed/size).

spd.setCompressionLevel(9); // maximum compression
spd.setCompressionLevel(1); // fastest, largest output

spd.setSigningKey(privateKeyPem: string): void / spd.setVerifyKey(publicKeyPem: string): void

Attach an Ed25519 signing key to the session. When set, saveToFile / saveData / saveToFileStreaming append an Ed25519 signature over the entire payload, and loadFromFile / loadFromString / loadFromFileStreaming verify it before decryption.

const { privateKey, publicKey } = crypto.generateKeyPairSync('ed25519', {
  publicKeyEncoding:  { type: 'spki',  format: 'pem' },
  privateKeyEncoding: { type: 'pkcs8', format: 'pem' },
});

spd.setSigningKey(privateKey);
await spd.saveToFileStreaming('./signed.spd', PASS);

const loaded = await SPD.loadFromFileStreaming('./signed.spd', PASS);
loaded.setVerifyKey(publicKey);
const data = await loaded.extractData(); // throws if signature invalid

spd.setEpochSize(bytes: number): void

Controls the epoch size (in bytes) for the key-ratcheting mechanism used in chunked exports. Defaults to 4 MB. Smaller values ratchet keys more frequently (higher security, lower throughput).

spd.destroy() / spd.clearCache()

Securely zeros all key material in place using sodium.memzero(), then clears all stored data. Call this when you are done with a session.

spd.destroy();

SPDWriter — streaming disk-backed writer

SPDWriter encrypts and writes entries one at a time to disk as they are produced — the full plaintext never exists in RAM simultaneously. Suitable for constructing large SPD files from a stream of records.

import { SPDWriter } from 'spd-lib-ts';

const writer = new SPDWriter('./output.spd', 'MyStr0ng!Passphrase#2024');
await writer.init();

await writer.addEntry('username', 'alice');
await writer.addEntry('config', { theme: 'dark' });
// ... add as many entries as needed

await writer.finalize();

After finalize(), the file contains an embedded SPDx index tail compatible with SPD.getEntry() — no sidecar file needed.

Options (passed as third argument to constructor):

| Option | Default | Description | |---|---|---| | compressionLevel | 9 | zlib deflate level | | hashAlgorithm | 'sha3-512' | Per-entry hash | | argon2Memory | 131072 (128 MiB) | Argon2id memory cost | | argon2Time | 6 | Argon2id time cost |

Call writer.destroy() to zero keys and delete the partial file if finalize() was never called.

SPDVault — in-memory key vault

SPDVault is a time-limited in-memory store for passcodes or keys. Keys expire automatically after a configurable timeout and are cleared on access renewal.

import { SPDVault } from 'spd-lib-ts';

const vault = new SPDVault(300_000); // 5-minute TTL

// Generate a random high-entropy key and store it
vault.genKey('session');

// Store your own key
vault.pushKey('myKey', 'MyStr0ng!Passphrase#2024');

// Retrieve (resets TTL)
const key = vault.pullKey('myKey'); // 'MyStr0ng!Passphrase#2024'

// Update (requires old value to match)
vault.updateKey('myKey', 'MyStr0ng!Passphrase#2024', 'NewPass!99');

// Delete one key
vault.destroyKey('myKey');

// Stop all timers (keys stay in memory)
vault.stop();

// Wipe everything
vault.destroy();

Generated keys are 500–699 characters of cryptographically random characters drawn from a 91-character charset using rejection sampling (no modulo bias).

SPDLegacy

SPDLegacy is a backwards-compatible class for reading files produced by SPD v1.x. It uses crypto_secretbox_easy (XSalsa20-Poly1305) and PBKDF2 key derivation. Do not use for new data — migrate to SPD instead.

import { SPDLegacy } from 'spd-lib-ts';

const spd = await SPDLegacy.loadFromFile('./old.spd', 'passcode');
const data = await spd.extractData();

Full example — save and load a config

import { SPD } from 'spd-lib-ts';
import * as path from 'path';

const FILE = path.join(process.env.HOME!, '.myapp', 'config.spd');
const PASS = process.env.APP_PASS!;

async function saveConfig(config: Record<string, unknown>) {
  const spd = new SPD();
  await spd.setPassKey(PASS);
  await spd.addMany(Object.entries(config).map(([name, data]) => ({ name, data })));
  await spd.saveToFileStreaming(FILE, PASS);
  spd.destroy();
}

async function loadConfig(): Promise<Record<string, unknown>> {
  const spd = await SPD.loadFromFileStreaming(FILE, PASS);
  const data = await spd.extractData();
  spd.destroy();
  return data;
}

Full example — chunked HTTP upload

// sender:
const spd = new SPD();
await spd.setPassKey(PASS);
await spd.addData('payload', largeObject);
const chunks = await spd.saveDataChunked(PASS, 256 * 1024); // 256 KB per chunk
spd.destroy();

for (let i = 0; i < chunks.length; i++) {
  await fetch('https://example.com/upload', {
    method: 'POST',
    body: JSON.stringify({ index: i, chunk: chunks[i] }),
  });
}

// receiver:
const spd = await SPD.loadFromChunks(receivedChunks, PASS);
const data = await spd.extractData();
spd.destroy();

Versioning

| npm version | SPD format version | Notes | |---|---|---| | 1.4.0 | v29 | CMT-4 key commitment, HKDF key expansion, encrypted entry names, 256-byte compression padding | | 1.3.1 | v26 | Hash algo + Argon2 params embedded in payload, secure key zeroing, null type support | | 1.3.0 | v25 | 512-bit Argon2id master secret, domain-separated AEAD + HMAC keys, HMAC-SHA3-512 | | < 1.3.0 | v24 | 256-bit Argon2id, single key for AEAD + MAC |

SPD format versions are not cross-compatible. v29 files cannot be read by older library versions. Use SPDLegacy to read v24 and earlier files.

Wire format (v29 binary)

Each entry in the binary file uses the following layout (all integers little-endian):

[2B encNameLen][encName][1B typeLen][type][2B hashLen][hash][3B nonceLen][nonce][8B dataLen][data]

• encName = [24B XChaCha20 nonce || CMT-4 ciphertext of entry name]
• data = [32B CMT block || XChaCha20-Poly1305 ciphertext of padded+compressed plaintext]

The full file is zlib-deflated and HMAC-SHA3-512 authenticated. A 72-byte header stores the plaintext length and MAC. An SPDx index tail is appended after the compressed stream for O(1) lookups.

License

ISC