json-to-toon-encoder-decoder

A lightweight, high-performance TypeScript library for encoding and decoding AI generative tokens between JSON and toon format. The toon format uses MessagePack serialization and zlib compression to significantly reduce token size for efficient transmission and storage.

Features

• 🚀 Fast encoding/decoding (< 100ms for 10KB tokens)
• 📦 Compact output with MessagePack + zlib compression
• 🔒 Built-in validation and error handling
• 🌐 Works in both Node.js (16+) and modern browsers
• 📘 Full TypeScript support with type definitions
• 🪶 Lightweight with minimal dependencies

Installation

npm install json-to-toon-encoder-decoder

Quick Start

import { encode, decode } from "json-to-toon-encoder-decoder";

// Encode a JSON token to toon format
const token = {
  userId: 12345,
  role: "admin",
  permissions: ["read", "write", "delete"],
  metadata: {
    createdAt: "2024-01-15T10:30:00Z",
    expiresAt: "2024-01-16T10:30:00Z",
  },
};

const toonToken = encode(token);
console.log(toonToken);
// Output: "toon_v1:eJyLrlZKzs8tyC..."

// Decode the toon token back to JSON
const decodedToken = decode(toonToken);
console.log(decodedToken);
// Output: { userId: 12345, role: 'admin', ... }

MCP Server

This package can also run as a Model Context Protocol (MCP) server, allowing AI assistants like Claude, OpenAI, and Gemini to use the encoding and decoding tools directly.

What is MCP?

The Model Context Protocol (MCP) is an open standard that enables AI assistants to securely connect to external tools and data sources. By running this package as an MCP server, AI assistants can encode and decode tokens on your behalf without you needing to write any code.

Hybrid Package Approach

This package works in two modes:

• Library Mode: Import and use the functions directly in your Node.js or browser applications (as shown above)
• Server Mode: Run as an MCP server that AI assistants can connect to and use as a tool

Both modes use the same underlying encoding/decoding logic, ensuring consistent behavior.

Supported AI Assistants

The MCP server works with any MCP-compatible client, including:

• Claude Desktop (Anthropic)
• OpenAI clients with MCP support
• Gemini clients with MCP support
• Any other MCP-compatible AI assistant or client

MCP Installation and Usage

Running the MCP Server

The easiest way to run the MCP server is using npx:

npx -y json-to-toon-encoder-decoder

This command will start the MCP server and listen for requests via stdio (standard input/output).

Configuration Examples

Claude Desktop Configuration

To use with Claude Desktop, add the following to your Claude configuration file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "json-to-toon": {
      "command": "npx",
      "args": ["-y", "json-to-toon-encoder-decoder"]
    }
  }
}

After updating the configuration, restart Claude Desktop. The tools will be available for Claude to use automatically.

Generic MCP Client Configuration

For other MCP-compatible clients:

{
  "mcpServers": {
    "json-to-toon": {
      "command": "npx",
      "args": ["json-to-toon-encoder-decoder"],
      "env": {}
    }
  }
}

Local Development Configuration

If you're developing or testing locally, you can point directly to the built server:

{
  "mcpServers": {
    "json-to-toon": {
      "command": "node",
      "args": [
        "/absolute/path/to/json-to-toon-encoder-decoder/dist/mcp/server.js"
      ]
    }
  }
}

Or use the npm script after building:

npm run build
npm run mcp

MCP Tools Documentation

When running as an MCP server, the following tools are available to AI assistants:

Tool: encode_json_to_toon

Encodes a JSON token to compressed toon format.

Parameters:

| Parameter | Type | Required | Description | | ------------------ | ---------------- | -------- | ------------------------------------------------------ | | token | object or string | Yes | The JSON token to encode (object or JSON string) | | compressionLevel | number | No | Compression level from 0 (none) to 9 (max). Default: 6 | | validate | boolean | No | Whether to validate the input token. Default: true |

Example Tool Call:

{
  "name": "encode_json_to_toon",
  "arguments": {
    "token": {
      "userId": 12345,
      "role": "admin",
      "permissions": ["read", "write"]
    },
    "compressionLevel": 6,
    "validate": true
  }
}

Success Response:

{
  "success": true,
  "toonToken": "toon_v1:eJyLrlZKzs8tyC9KTMnMzytRslIKzy/KSVGqBQAuRQh9",
  "length": 52
}

Error Response:

{
  "success": false,
  "error": "Invalid JSON token: token must be an object or valid JSON string",
  "errorCode": "INVALID_JSON_TOKEN"
}

Tool: decode_toon_to_json

Decodes a toon format token back to the original JSON object.

Parameters:

| Parameter | Type | Required | Description | | ----------- | ------- | -------- | ------------------------------------------------------------ | | toonToken | string | Yes | The toon token string to decode (must start with 'toon_v1:') | | validate | boolean | No | Whether to validate the toon token format. Default: true |

Example Tool Call:

{
  "name": "decode_toon_to_json",
  "arguments": {
    "toonToken": "toon_v1:eJyLrlZKzs8tyC9KTMnMzytRslIKzy/KSVGqBQAuRQh9",
    "validate": true
  }
}

Success Response:

{
  "success": true,
  "decodedToken": {
    "userId": 12345,
    "role": "admin",
    "permissions": ["read", "write"]
  }
}

Error Response:

{
  "success": false,
  "error": "Invalid toon token: token must start with 'toon_v1:' prefix",
  "errorCode": "INVALID_TOON_TOKEN"
}

Error Codes

MCP tool responses may include the following error codes:

Encoding Errors:

| Error Code | Description | | --------------------------- | ------------------------------------------------------ | | INVALID_JSON_TOKEN | The provided JSON token is invalid or cannot be parsed | | INVALID_COMPRESSION_LEVEL | Compression level must be between 0 and 9 | | SERIALIZATION_FAILED | Failed to serialize the JSON token using MessagePack | | COMPRESSION_FAILED | Failed to compress the serialized data | | BASE64_ENCODING_FAILED | Failed to encode compressed data to Base64 | | ENCODING_FAILED | General encoding failure |

Decoding Errors:

| Error Code | Description | | ------------------------ | -------------------------------------------------- | | INVALID_TOON_TOKEN | The provided toon token is invalid or corrupted | | BASE64_DECODING_FAILED | Failed to decode Base64 string | | DECOMPRESSION_FAILED | Failed to decompress the data (possibly corrupted) | | DESERIALIZATION_FAILED | Failed to deserialize MessagePack data | | DECODING_FAILED | General decoding failure |

MCP Troubleshooting

Testing the MCP Server Locally

To test if the MCP server is working correctly, you can run it manually:

# Build the project first
npm run build

# Run the MCP server
npm run mcp

The server will start and output: JSON to Toon MCP Server running on stdio

The server communicates via stdio, so you won't see interactive output. To properly test it, you'll need an MCP client or inspector tool.

Using MCP Inspector

You can use the MCP Inspector tool to test your server:

npx @modelcontextprotocol/inspector npx -y json-to-toon-encoder-decoder

This will open a web interface where you can:

• View available tools
• Test tool calls with different parameters
• See request/response messages
• Debug any issues

Common Configuration Issues

Issue: Claude Desktop doesn't show the tools

• Verify the configuration file path is correct for your OS
• Ensure the JSON syntax is valid (no trailing commas, proper quotes)
• Restart Claude Desktop completely after changing the config
• Check Claude's logs for error messages

Issue: "command not found" or "npx not found"

• Ensure Node.js and npm are installed: node --version and npm --version
• Make sure npx is available: npx --version
• Try using the full path to npx in your configuration

Issue: Server starts but tools don't work

• Verify you're using the latest version: npm install -g json-to-toon-encoder-decoder
• Check that the package built correctly: look for dist/mcp/server.js
• Try running the server manually to see error messages: npm run mcp

Issue: Tools work but give unexpected results

• Verify your input format matches the tool schema
• Check that JSON tokens are properly formatted
• Ensure toon tokens include the toon_v1: prefix when decoding
• Review error codes in the response for specific issues

Debugging Tips

1. Check Server Logs: The MCP server logs to stderr. When running manually, you'll see startup messages and any errors.

2. Validate Your Configuration: Use a JSON validator to ensure your MCP client configuration is valid JSON.

3. Test with Simple Inputs: Start with simple tokens to verify basic functionality:

   {
     "token": { "test": "value" }
   }

4. Check Node.js Version: Ensure you're running Node.js 16 or higher:

   node --version

5. Reinstall if Needed: If issues persist, try reinstalling:

   npm uninstall -g json-to-toon-encoder-decoder
   npm install -g json-to-toon-encoder-decoder

6. Use Local Build for Development: When developing, use the local build configuration to see detailed error messages and make debugging easier.

API Reference

encode(token, options?)

Encodes a JSON token to toon format.

Parameters:

• token (object | string): The JSON token to encode. Can be a JavaScript object or a JSON string.
• options (EncodeOptions, optional): Encoding options.

Returns: string - The encoded toon token with toon_v1: prefix.

Throws: TokenEncoderError if encoding fails.

Example:

import { encode } from "json-to-toon-encoder-decoder";

// Encode an object
const token = { userId: 123, role: "user" };
const toonToken = encode(token);

// Encode with custom compression level
const toonTokenHighCompression = encode(token, { compressionLevel: 9 });

// Encode a JSON string
const jsonString = '{"userId":123,"role":"user"}';
const toonTokenFromString = encode(jsonString);

// Encode without validation (faster, but less safe)
const toonTokenNoValidation = encode(token, { validate: false });

decode(toonToken, options?)

Decodes a toon token back to a JSON object.

Parameters:

• toonToken (string): The toon token string to decode (with or without toon_v1: prefix).
• options (DecodeOptions, optional): Decoding options.

Returns: object - The decoded JSON object.

Throws: TokenDecoderError if decoding fails.

Example:

import { decode } from "json-to-toon-encoder-decoder";

// Decode a toon token
const toonToken = "toon_v1:eJyLrlZKzs8tyC...";
const token = decode(toonToken);

// Decode without validation (faster, but less safe)
const tokenNoValidation = decode(toonToken, { validate: false });

Options

EncodeOptions

Configuration options for encoding tokens.

interface EncodeOptions {
  /**
   * Compression level (0-9)
   * - 0: No compression (fastest)
   * - 1-3: Fast compression
   * - 4-6: Balanced (default: 6)
   * - 7-9: Maximum compression (slowest)
   */
  compressionLevel?: number;

  /**
   * Whether to validate input before encoding
   * Default: true
   * Set to false for better performance if you're certain the input is valid
   */
  validate?: boolean;
}

DecodeOptions

Configuration options for decoding tokens.

interface DecodeOptions {
  /**
   * Whether to validate input before decoding
   * Default: true
   * Set to false for better performance if you're certain the input is valid
   */
  validate?: boolean;
}

Error Handling

The library provides three error classes with descriptive error codes:

TokenEncoderError

Thrown when encoding operations fail.

Error Codes:

| Code | Description | | --------------------------- | ------------------------------------------------------ | | INVALID_JSON_TOKEN | The provided JSON token is invalid or cannot be parsed | | INVALID_COMPRESSION_LEVEL | Compression level must be between 0 and 9 | | SERIALIZATION_FAILED | Failed to serialize the JSON token using MessagePack | | COMPRESSION_FAILED | Failed to compress the serialized data | | BASE64_ENCODING_FAILED | Failed to encode compressed data to Base64 | | ENCODING_FAILED | General encoding failure (catch-all) |

TokenDecoderError

Thrown when decoding operations fail.

Error Codes:

| Code | Description | | ------------------------ | -------------------------------------------------- | | INVALID_TOON_TOKEN | The provided toon token is invalid or corrupted | | BASE64_DECODING_FAILED | Failed to decode Base64 string | | DECOMPRESSION_FAILED | Failed to decompress the data (possibly corrupted) | | DESERIALIZATION_FAILED | Failed to deserialize MessagePack data | | DECODING_FAILED | General decoding failure (catch-all) |

ValidationError

Thrown when validation operations fail.

Error Codes:

| Code | Description | | ------------------- | ----------------------- | | VALIDATION_FAILED | Input validation failed |

Error Handling Example

import {
  encode,
  decode,
  TokenEncoderError,
  TokenDecoderError,
} from "json-to-toon-encoder-decoder";

try {
  const toonToken = encode({ userId: 123 });
  console.log("Encoded:", toonToken);
} catch (error) {
  if (error instanceof TokenEncoderError) {
    console.error("Encoding failed:", error.message);
    console.error("Error code:", error.code);

    switch (error.code) {
      case "INVALID_JSON_TOKEN":
        console.error("Please provide a valid JSON token");
        break;
      case "COMPRESSION_FAILED":
        console.error("Compression error - try a different compression level");
        break;
      default:
        console.error("Unexpected encoding error");
    }
  }
}

try {
  const token = decode("invalid-token");
  console.log("Decoded:", token);
} catch (error) {
  if (error instanceof TokenDecoderError) {
    console.error("Decoding failed:", error.message);
    console.error("Error code:", error.code);

    switch (error.code) {
      case "INVALID_TOON_TOKEN":
        console.error("Token format is invalid");
        break;
      case "DECOMPRESSION_FAILED":
        console.error("Token appears to be corrupted");
        break;
      default:
        console.error("Unexpected decoding error");
    }
  }
}

Advanced Usage

Using Classes Directly

For more control, you can use the TokenEncoder and TokenDecoder classes directly:

import { TokenEncoder, TokenDecoder } from "json-to-toon-encoder-decoder";

// Create encoder with custom options
const encoder = new TokenEncoder({
  compressionLevel: 9,
  validate: true,
});

const toonToken = encoder.encode({ userId: 123 });

// Create decoder with custom options
const decoder = new TokenDecoder({
  validate: true,
});

const token = decoder.decode(toonToken);

Validation

Use the TokenValidator class for standalone validation:

import { TokenValidator } from "json-to-toon-encoder-decoder";

// Validate JSON token
const jsonResult = TokenValidator.validateJsonToken({ userId: 123 });
if (!jsonResult.valid) {
  console.error("Invalid JSON token:", jsonResult.error);
}

// Validate toon token
const toonResult = TokenValidator.validateToonToken("toon_v1:eJyLrlZKzs...");
if (!toonResult.valid) {
  console.error("Invalid toon token:", toonResult.error);
}

// Validate Base64 string
const isValid = TokenValidator.isValidBase64("eJyLrlZKzs...");
console.log("Is valid Base64:", isValid);

Performance Characteristics

Speed

• Encoding: < 100ms for tokens up to 10KB
• Decoding: < 100ms for tokens up to 10KB
• Typical performance for 1KB token: 1-5ms (encode + decode)

Compression Ratio

The compression ratio depends on the data structure and content:

• Simple tokens (< 1KB): 40-60% size reduction
• Medium tokens (1-10KB): 60-80% size reduction
• Large tokens (> 10KB): 70-85% size reduction
• Highly repetitive data: Up to 90% size reduction

Example:

const token = {
  userId: 12345,
  role: "admin",
  permissions: ["read", "write", "delete"],
  metadata: { createdAt: "2024-01-15T10:30:00Z" },
};

const jsonString = JSON.stringify(token);
const toonToken = encode(token);

console.log("JSON size:", jsonString.length, "bytes");
// Output: JSON size: 145 bytes

console.log("Toon size:", toonToken.length, "bytes");
// Output: Toon size: 68 bytes

console.log(
  "Compression ratio:",
  ((1 - toonToken.length / jsonString.length) * 100).toFixed(1) + "%"
);
// Output: Compression ratio: 53.1%

Bundle Size

• Minified: ~45KB
• Minified + Gzipped: ~15KB
• Zero runtime dependencies beyond @msgpack/msgpack and pako

Browser Compatibility

The library works in all modern browsers that support:

• ES2015 (ES6)
• Uint8Array
• Base64 encoding/decoding (btoa/atob)

Tested browsers:

• Chrome 90+
• Firefox 88+
• Safari 14+
• Edge 90+

Node.js Compatibility

• Node.js 16.0.0 or higher
• Works with both CommonJS and ES Modules

TypeScript Support

The library is written in TypeScript and includes full type definitions:

import type {
  EncodeOptions,
  DecodeOptions,
  ValidationResult,
} from "json-to-toon-encoder-decoder";

const options: EncodeOptions = {
  compressionLevel: 6,
  validate: true,
};

License

MIT

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Support

For issues, questions, or feature requests, please open an issue on the GitHub repository.