@usecolf/protocol

Core COLF (Columnar Optimized List Format) v1 protocol definition, high-performance encoder, decoder, column optimizer, and statistics engine.

Overview

@usecolf/protocol provides the low-level foundation for the COLF specification. It converts standard array-of-objects JSON data into compressed, schema-mangled columnar objects (COLFDocument) with bitmap null-value encoding, dynamic key indexing, and precision type casting.

Features

• Columnar Transformation: Converts array of JavaScript objects into columnar arrays indexed by property keys.
• Zero-Dependency Core: Lightweight ESM package runnable in Node.js, browsers, and Edge runtimes.
• Automatic Optimization: Computes compression ratios, entropy scores, and null bitmaps.
• Roundtrip Fidelity: Restores original array objects without data loss.

Installation

npm install @usecolf/protocol
# or
pnpm add @usecolf/protocol

Quick Start

import { encode, decode } from '@usecolf/protocol';

const dataset = [
  { id: 1, name: 'Alice', role: 'admin', active: true },
  { id: 2, name: 'Bob', role: 'user', active: false },
  { id: 3, name: 'Charlie', role: 'user', active: true }
];

// Encode array of objects into COLF document
const colfDoc = encode(dataset, { mode: 'auto' });
console.log(colfDoc);
/*
{
  v: 1,
  r: 3,
  k: ['id', 'name', 'role', 'active'],
  c: [[1, 2, 3], ['Alice', 'Bob', 'Charlie'], ['admin', 'user', 'user'], [true, false, true]]
}
*/

// Decode COLF document back into original JavaScript objects
const restored = decode(colfDoc);
console.log(restored);

API Reference

encode(data: Record<string, unknown>[], options?: EncodeOptions): EncodeResult

Encodes an array of JSON objects into a COLFDocument.

decode(doc: COLFDocument, options?: DecodeOptions): Record<string, unknown>[]

Decodes a COLFDocument back into an array of JavaScript objects.

optimize(data: Record<string, unknown>[]): OptimizationResult

Analyzes dataset columns and produces optimal property key order and null bitmaps.

License

MIT © COLF Team