ISON Parser (JavaScript)

ISON (Interchange Simple Object Notation) - A token-efficient data format optimized for AI/LLM workflows.

Features

• 30-70% fewer tokens than JSON for structured data
• ISONL streaming format for fine-tuning datasets and event streams
• Native references for relational data (:ID syntax)
• Type inference for clean, minimal syntax
• Zero dependencies - works in Node.js and browser
• Built-in validation - Zod-like schema validation included

Installation

npm install ison-parser

Or via CDN:

<script src="https://unpkg.com/ison-parser/dist/ison-parser.js"></script>

Quick Start

Node.js

const ISON = require('ison-parser');

// Parse ISON
const isonText = `
table.users
id name email
1 Alice [email protected]
2 Bob [email protected]
`;

const doc = ISON.loads(isonText);
console.log(doc.blocks[0].rows[0].name); // "Alice"

// Convert to JSON
const jsonData = doc.toDict();

ES Modules

import ISON from 'ison-parser';

const doc = ISON.loads(isonText);

Browser

<script src="https://unpkg.com/ison-parser/dist/ison-parser.js"></script>
<script>
  const doc = ISON.loads(isonText);
  console.log(doc.toDict());
</script>

Built-in Validation (Zod-like)

ISON Parser includes a Zod-like validation module for schema validation:

const { i, document, ValidationError } = require('ison-parser/validation');

// Define schemas
const userSchema = i.object({
    id: i.int().min(1),
    name: i.string().min(1).max(100),
    email: i.string().email(),
    role: i.string().default('user')
});

// Validate data
const user = userSchema.parse({
    id: 1,
    name: 'Alice',
    email: '[email protected]'
});

// SafeParse (non-throwing)
const result = userSchema.safeParse(data);
if (result.success) {
    console.log(result.data);
} else {
    console.log(result.error.errors);
}

Schema Types

// Primitives
i.string()                    // String validation
i.string().min(1).max(100)    // With length constraints
i.string().email()            // Email validation
i.string().regex(/pattern/)   // Regex validation

i.number()                    // Number validation
i.int()                       // Integer validation
i.number().min(0).max(100)    // With range constraints
i.number().positive()         // Must be > 0
i.number().negative()         // Must be < 0

i.boolean()                   // Boolean validation
i.bool()                      // Alias

i.null()                      // Null validation

// ISON References
i.ref()                       // Reference validation (:id or :type:id)

// Complex types
i.object({ ... })             // Object with shape
i.array(i.string())           // Array of strings
i.table('name', { ... })      // ISON table block

Document Validation

Validate complete ISON documents with multiple blocks:

const schema = document({
    users: i.table('users', {
        id: i.int(),
        name: i.string(),
        email: i.string().email()
    }),
    config: i.object({
        debug: i.bool().default(false)
    }).optional()
});

const doc = schema.parse(ISON.loads(isonText).toDict());

Schema Composition

// Extend objects
const baseSchema = i.object({ id: i.int(), name: i.string() });
const userSchema = baseSchema.extend({ email: i.string().email() });

// Pick/Omit fields
const publicUser = userSchema.pick('id', 'name');
const safeUser = userSchema.omit('password');

// Custom refinements
const evenNumber = i.int().refine(n => n % 2 === 0, 'Must be even');

ISONL Streaming

ISONL is perfect for fine-tuning datasets, event streams, and logs:

const ISON = require('ison-parser');

// Parse ISONL
const isonlText = `table.examples|instruction response|"Summarize this" "Brief summary..."
table.examples|instruction response|"Translate to Spanish" "Hola mundo"`;

const doc = ISON.loadsISONL(isonlText);

// Stream records
const lines = isonlText.split('\n');
for (const record of ISON.isonlStream(lines)) {
  console.log(record.values);
}

Format Conversion

// ISON to ISONL
const isonl = ISON.isonToISONL(isonText);

// ISONL to ISON
const ison = ISON.isonlToISON(isonlText);

// JSON to ISON
const isonFromJson = ISON.jsonToISON(jsonObject);

// ISON to JSON
const jsonFromIson = ISON.isonToJSON(isonText);

API Reference

Core Functions

// Parse ISON string
const doc = ISON.loads(isonText);

// Serialize to ISON
const text = ISON.dumps(doc);

// Convert JSON to ISON
const ison = ISON.jsonToISON(jsonObject);

// Convert ISON to JSON
const json = ISON.isonToJSON(isonText);

ISONL Functions

// Parse ISONL
const doc = ISON.loadsISONL(isonlText);

// Serialize to ISONL
const isonl = ISON.dumpsISONL(doc);

// Stream ISONL
for (const record of ISON.isonlStream(lines)) {
  // process record
}

// Convert between formats
const isonl = ISON.isonToISONL(isonText);
const ison = ISON.isonlToISON(isonlText);

Classes

// Document - container for blocks
const doc = new ISON.Document();
doc.blocks.push(block);

// Block - single data block
const block = new ISON.Block('table', 'users', ['id', 'name'], rows);

// Reference - pointer to another record
const ref = new ISON.Reference('123', 'user');
console.log(ref.toISON()); // ":user:123"

// ISONLRecord - single streaming record
const record = new ISON.ISONLRecord('table', 'users', ['id', 'name'], values);

ISON Format

Tables (Structured Data)

table.users
id name email active
1 Alice [email protected] true
2 Bob [email protected] false

Objects (Key-Value)

object.config
timeout 30
debug true
api_key "sk-xxx"

References

table.orders
id customer_id total
O1 :C1 99.99
O2 :C2 149.50

ISONL Format

Each line is a self-contained record:

kind.name|field1 field2 field3|value1 value2 value3

Example:

table.users|id name email|1 Alice [email protected]
table.users|id name email|2 Bob [email protected]

Token Efficiency

| Records | JSON Tokens | ISON Tokens | Savings | |---------|-------------|-------------|---------| | 10 | ~200 | ~60-140 | 30-70% | | 100 | ~2000 | ~600-1400 | 30-70% | | 1000 | ~20000 | ~6000-14000 | 30-70% |

TypeScript Support

TypeScript declarations are included:

import ISON, { Document, Block, Reference } from 'ison-parser';

const doc: Document = ISON.loads(text);
const block: Block = doc.blocks[0];

Test Results

All 80 tests passing (33 parser + 47 validation):

Running ISON v1.0 Parser Tests
========================================
Results: 33 passed, 0 failed

Running ISON Validation Tests
========================================
Results: 47 passed, 0 failed

Run tests with:

npm test

Links

• Documentation | www.getison.com
• Specification
• ISONL Spec
• Python Package
• GitHub

License

MIT License - see LICENSE for details.