🤖 Agentic IO Parser

A framework-agnostic TypeScript library tailored for LLM payload Pre-processing and Post-processing. Built specifically to handle the unpredictability of LLM responses (hallucinations, malformed JSON, etc.) and to secure payloads before sending them out.

✨ Features

• Zero-Dependency Core: Lightweight and extremely fast.
• Strictly Typed (Zero-Any): Enforces precise JsonValue types across the entire pipeline.
• Deep Recursive Parsing: Operates securely on multi-level nested objects and arrays.
• Isomorphic: Runs everywhere — Node.js, Browsers, Edge Workers (Cloudflare/Vercel).

📦 Installation

npm install agentic-io-parser
# or
yarn add agentic-io-parser
# or
pnpm add agentic-io-parser

🛠️ Usage

1. Pre-Processing (Before sending to LLM)

Optimize token usage and protect sensitive data before it reaches the LLM.

PayloadMinifier

Recursively removes null values, empty arrays [], empty objects {}, and specific drop keys to save token cost.

import { PayloadMinifier } from 'agentic-io-parser';

const minifier = new PayloadMinifier({ dropKeys: ['__v', 'createdAt'] });
const payload = { user: { name: 'Alice', bio: null, posts: [] }, __v: 0 };

const minified = minifier.minify(payload);
// Output: { user: { name: 'Alice' } }

PayloadCompressor

Compresses large payloads by recursively mapping long keys to short aliases (e.g., a, b) and generating a decompression dictionary. Essential for reducing token consumption on large arrays of objects.

import { PayloadCompressor } from 'agentic-io-parser';

const compressor = new PayloadCompressor({ minLengthToCompress: 5 });
const payload = [{ customer_transaction_id: 123 }, { customer_transaction_id: 456 }];

const { compressed, dictionary } = compressor.compress(payload);
// compressed: [{ a: 123 }, { a: 456 }]
// dictionary: { a: 'customer_transaction_id' }

// Send 'compressed' and 'dictionary' to LLM.
// Once LLM responds with compressed keys, decompress it:
const original = compressor.decompress(compressedResponse, dictionary);

PiiScrubber

Automatically redacts sensitive information like Emails, Credit Cards, VN Phone Numbers, and JWT Tokens using deep traversal.

import { PiiScrubber } from 'agentic-io-parser';

const scrubber = new PiiScrubber();
const safeData = scrubber.scrub({ email: '[email protected]', phone: '0987654321' });
// Output: { email: '[REDACTED]', phone: '[REDACTED]' }

ContextCapper

Smartly truncates extremely long strings by preserving the head and tail, injecting ...[truncated X chars]... in the middle to give the LLM full context without blowing up the context window.

import { ContextCapper } from 'agentic-io-parser';

const capper = new ContextCapper({ maxLength: 50 });
capper.cap('VERY_LONG_STRING_..._ENDING');

MarkupCleaner

Strips out raw HTML (especially <script> and <style>) from payloads to prevent XSS injections and confusing the LLM.

import { MarkupCleaner } from 'agentic-io-parser';

const cleaner = new MarkupCleaner();
cleaner.clean('<p>Hello <b>World</b></p><script>alert(1)</script>');
// Output: 'Hello World'

2. Post-Processing (After receiving from LLM)

Extract, repair, and strictly validate the unpredictable outputs from LLMs.

JsonExtractor

Extracts JSON securely using a 3-tier fallback strategy (Markdown Block -> Deep Brackets -> Raw Trim).

import { JsonExtractor } from 'agentic-io-parser';

const extractor = new JsonExtractor({ strategy: 'first' });
const jsonString = extractor.extract('Here is your data: ```json\n{"a":1}\n``` Enjoy!');
// Output: '{"a":1}'

JsonRepair

Wraps AST-based jsonrepair to aggressively fix missing quotes, trailing commas, single quotes, and other common LLM hallucinated syntax errors.

import { JsonRepair } from 'agentic-io-parser';

const repair = new JsonRepair();
const validObj = repair.repair("{ name: 'Alice', age: 25, }"); 
// Output: { name: "Alice", age: 25 }

SchemaValidator (with Zod)

The ultimate boss. Validates the repaired JSON against a strict schema, automatically stripping hallucinatory extra keys and enforcing precise types.

import { z } from 'zod';
import { SchemaValidator } from 'agentic-io-parser';

const validator = new SchemaValidator();
const UserSchema = z.object({ name: z.string(), age: z.number() });

// Throws detailed ZodError if invalid, otherwise returns strictly typed object
const safeData = validator.validate(dirtyPayload, UserSchema);

ToneFormatter

Removes markdown asterisks, normalizes excessive whitespaces, and standardizes newlines for clean UI rendering.

import { ToneFormatter } from 'agentic-io-parser';

const formatter = new ToneFormatter();
formatter.format('**Hello**   World!\n\n\nHow are you?');
// Output: 'Hello World!\nHow are you?'

🧪 Testing

The library features comprehensive test suites ensuring resilience against edge cases.

pnpm test

📜 License

MIT License