@taml/encoder

Convert ANSI escape sequences to TAML (Terminal ANSI Markup Language) tags for further processing and manipulation.

TAML Ecosystem

TAML (Terminal ANSI Markup Language) is a lightweight markup language for styling terminal output with ANSI escape codes. For the complete specification, visit the TAML Specification Repository.

Package Dependencies

graph TD
    B["@taml/parser"] --> A["@taml/ast"]
    C["@taml/react"] --> A
    C --> B
    D["@taml/docusaurus"] --> C
    E["@taml/encoder"] --> F["@taml/cli"]
    E -.-> A
    E -.-> B
    style E fill:#e1f5fe,stroke:#01579b,stroke-width:2px

Related Packages

Core Infrastructure

• @taml/ast - Foundation package providing AST node types, visitor patterns, and tree traversal utilities for TAML documents.
• @taml/parser - Robust parser that converts TAML markup strings into typed AST nodes with comprehensive error handling and validation.

Input/Output Tools

• @taml/encoder - Converts raw ANSI escape sequences into clean TAML markup for further processing and manipulation.
• @taml/cli - Command-line tool for converting ANSI escape sequences to TAML format in batch operations.

Integration Packages

• @taml/react - React component that renders TAML markup as styled JSX elements with full TypeScript support and performance optimization.
• @taml/docusaurus - Docusaurus theme that automatically detects and renders TAML code blocks in documentation sites.

Installation

npm

npm install @taml/encoder

yarn

yarn add @taml/encoder

pnpm

pnpm add @taml/encoder

bun

bun add @taml/encoder

TypeScript Setup

This package includes TypeScript declarations out of the box. No additional setup is required for TypeScript projects.

// ESM
import { encode } from "@taml/encoder";

// CommonJS
const { encode } = require("@taml/encoder");

Quick Start

Here's a 5-minute introduction to converting ANSI escape sequences to TAML markup:

import { encode } from "@taml/encoder";

// Basic color conversion
const coloredText = encode("\x1b[31mError:\x1b[0m Operation failed");
console.log(coloredText); // "<red>Error:</red> Operation failed"

// Multiple formatting styles
const styledText = encode("\x1b[1m\x1b[31mBold Red\x1b[0m");
console.log(styledText); // "<bold><red>Bold Red</red></bold>"

// Real-world terminal output
const gitOutput = encode(
  "On branch \x1b[32mmain\x1b[0m\nChanges:\n\x1b[31mmodified: file.txt\x1b[0m",
);
console.log(gitOutput);
// "On branch <green>main</green>
// Changes:
// <red>modified: file.txt</red>"

// Progress indicators
const progress = encode(
  "Progress: [\x1b[32m██████████\x1b[0m\x1b[37m░░░░░\x1b[0m] 66%",
);
console.log(progress); // "Progress: [<green>██████████</green><white>░░░░░</white>] 66%"

// Log levels with colors
const logs = encode(
  "\x1b[32mINFO\x1b[0m: Success\n\x1b[33mWARN\x1b[0m: Warning\n\x1b[31mERROR\x1b[0m: Failed",
);
console.log(logs);
// "<green>INFO</green>: Success
// <yellow>WARN</yellow>: Warning
// <red>ERROR</red>: Failed"

Core Concepts

ANSI to TAML Conversion

The encoder transforms raw ANSI escape sequences into structured TAML markup, making terminal output easier to process, parse, and manipulate. This conversion enables:

• Clean Markup: Convert messy escape sequences to readable tags
• Nested Formatting: Proper handling of overlapping styles
• Safe Processing: Escape special characters for XML/HTML compatibility
• Further Processing: Enable parsing with @taml/parser

Supported ANSI Features

Colors

• Standard Colors (30-37): black, red, green, yellow, blue, magenta, cyan, white
• Bright Colors (90-97): brightBlack, brightRed, brightGreen, etc.
• 256-Color Palette (38;5;n): Mapped to closest standard color
• RGB Colors (38;2;r;g;b): Converted to nearest standard color

Background Colors

• Standard Backgrounds (40-47): bgBlack, bgRed, bgGreen, etc.
• Bright Backgrounds (100-107): bgBrightBlack, bgBrightRed, etc.
• Extended Backgrounds: 256-color and RGB background support

Text Styles

• Bold (1): <bold>text</bold>
• Dim (2): <dim>text</dim>
• Italic (3): <italic>text</italic>
• Underline (4): <underline>text</underline>
• Strikethrough (9): <strikethrough>text</strikethrough>

Reset Sequences

• Full Reset (0): Closes all open tags
• Foreground Reset (39): Removes color formatting
• Background Reset (49): Removes background formatting

Character Escaping

The encoder automatically escapes special characters for safe processing:

encode("5 < 10"); // "5 &lt; 10"
encode("Use &lt; for less-than"); // "Use &amp;lt; for less-than"
encode("\x1b[31m5 < 10\x1b[0m"); // "<red>5 &lt; 10</red>"

Usage Examples

Basic Color Conversion

import { encode } from "@taml/encoder";

// Standard colors
encode("\x1b[31mRed text\x1b[0m"); // "<red>Red text</red>"
encode("\x1b[32mGreen text\x1b[0m"); // "<green>Green text</green>"
encode("\x1b[34mBlue text\x1b[0m"); // "<blue>Blue text</blue>"

// Bright colors
encode("\x1b[91mBright Red\x1b[0m"); // "<brightRed>Bright Red</brightRed>"
encode("\x1b[92mBright Green\x1b[0m"); // "<brightGreen>Bright Green</brightGreen>"
encode("\x1b[94mBright Blue\x1b[0m"); // "<brightBlue>Bright Blue</brightBlue>"

Background Colors

// Standard backgrounds
encode("\x1b[41mRed Background\x1b[0m"); // "<bgRed>Red Background</bgRed>"
encode("\x1b[42mGreen Background\x1b[0m"); // "<bgGreen>Green Background</bgGreen>"

// Bright backgrounds
encode("\x1b[101mBright Red BG\x1b[0m"); // "<bgBrightRed>Bright Red BG</bgBrightRed>"
encode("\x1b[102mBright Green BG\x1b[0m"); // "<bgBrightGreen>Bright Green BG</bgBrightGreen>"

Text Formatting

// Individual styles
encode("\x1b[1mBold text\x1b[0m"); // "<bold>Bold text</bold>"
encode("\x1b[2mDim text\x1b[0m"); // "<dim>Dim text</dim>"
encode("\x1b[3mItalic text\x1b[0m"); // "<italic>Italic text</italic>"
encode("\x1b[4mUnderlined text\x1b[0m"); // "<underline>Underlined text</underline>"
encode("\x1b[9mStrikethrough text\x1b[0m"); // "<strikethrough>Strikethrough text</strikethrough>"

Nested and Combined Formatting

// Nested formatting
encode("\x1b[1m\x1b[31mBold Red\x1b[0m");
// "<bold><red>Bold Red</red></bold>"

encode("\x1b[4m\x1b[32mUnderlined Green\x1b[0m");
// "<underline><green>Underlined Green</green></underline>"

// Multiple color changes
encode("\x1b[31mRed\x1b[32mGreen\x1b[34mBlue\x1b[0m");
// "<red>Red</red><green>Green</green><blue>Blue</blue>"

// Complex combinations
encode("\x1b[1m\x1b[4m\x1b[31mBold Underlined Red\x1b[0m");
// "<bold><underline><red>Bold Underlined Red</red></underline></bold>"

Advanced Color Formats

// 256-color palette (mapped to standard colors)
encode("\x1b[38;5;196mBright Red\x1b[0m"); // "<brightRed>Bright Red</brightRed>"
encode("\x1b[38;5;46mBright Green\x1b[0m"); // "<brightGreen>Bright Green</brightGreen>"

// RGB colors (converted to nearest standard color)
encode("\x1b[38;2;255;0;0mRGB Red\x1b[0m"); // "<brightRed>RGB Red</brightRed>"
encode("\x1b[38;2;0;255;0mRGB Green\x1b[0m"); // "<brightGreen>RGB Green</brightGreen>"

// Background 256-color and RGB
encode("\x1b[48;5;196mRed Background\x1b[0m"); // "<bgBrightRed>Red Background</bgBrightRed>"
encode("\x1b[48;2;255;0;0mRGB Red BG\x1b[0m"); // "<bgBrightRed>RGB Red BG</bgBrightRed>"

Real-World Terminal Output

Git Status Output

const gitStatus = `On branch \x1b[32mmain\x1b[0m
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  \x1b[31mmodified:   src/app.ts\x1b[0m
  \x1b[31mmodified:   README.md\x1b[0m

Untracked files:
  \x1b[31mnew-feature.ts\x1b[0m`;

const tamlOutput = encode(gitStatus);
console.log(tamlOutput);
// "On branch <green>main</green>
// Your branch is up to date with 'origin/main'.
//
// Changes not staged for commit:
//   <red>modified:   src/app.ts</red>
//   <red>modified:   README.md</red>
//
// Untracked files:
//   <red>new-feature.ts</red>"

Application Logs

const logOutput = `\x1b[90m2024-01-15 10:30:15\x1b[0m \x1b[32mINFO\x1b[0m  Server started on port 3000
\x1b[90m2024-01-15 10:30:16\x1b[0m \x1b[33mWARN\x1b[0m  Database connection slow
\x1b[90m2024-01-15 10:30:17\x1b[0m \x1b[31mERROR\x1b[0m Authentication failed for [email protected]
\x1b[90m2024-01-15 10:30:18\x1b[0m \x1b[36mDEBUG\x1b[0m Cache miss for key: user:123`;

const tamlLogs = encode(logOutput);
console.log(tamlLogs);
// "<brightBlack>2024-01-15 10:30:15</brightBlack> <green>INFO</green>  Server started on port 3000
// <brightBlack>2024-01-15 10:30:16</brightBlack> <yellow>WARN</yellow>  Database connection slow
// <brightBlack>2024-01-15 10:30:17</brightBlack> <red>ERROR</red> Authentication failed for [email protected]
// <brightBlack>2024-01-15 10:30:18</brightBlack> <cyan>DEBUG</cyan> Cache miss for key: user:123"

Progress Indicators

const progressBar = `Downloading packages...
Progress: [\x1b[32m████████████████████\x1b[0m\x1b[37m░░░░░\x1b[0m] 80% (4/5)
\x1b[32m✓\x1b[0m [email protected]
\x1b[32m✓\x1b[0m [email protected]
\x1b[32m✓\x1b[0m @types/[email protected]
\x1b[32m✓\x1b[0m [email protected]
\x1b[33m⏳\x1b[0m @taml/[email protected]`;

const tamlProgress = encode(progressBar);
console.log(tamlProgress);
// "Downloading packages...
// Progress: [<green>████████████████████</green><white>░░░░░</white>] 80% (4/5)
// <green>✓</green> [email protected]
// <green>✓</green> [email protected]
// <green>✓</green> @types/[email protected]
// <green>✓</green> [email protected]
// <yellow>⏳</yellow> @taml/[email protected]"

Test Results

const testOutput = `\x1b[1mRunning tests...\x1b[0m

\x1b[32m✓\x1b[0m should handle basic colors
\x1b[32m✓\x1b[0m should handle nested formatting
\x1b[31m✗\x1b[0m should handle malformed sequences
  \x1b[90mExpected: "text"\x1b[0m
  \x1b[90mReceived: "\x1b[XYZtext"\x1b[0m

\x1b[1mTest Results:\x1b[0m
\x1b[32m2 passing\x1b[0m
\x1b[31m1 failing\x1b[0m`;

const tamlTests = encode(testOutput);
console.log(tamlTests);
// "<bold>Running tests...</bold>
//
// <green>✓</green> should handle basic colors
// <green>✓</green> should handle nested formatting
// <red>✗</red> should handle malformed sequences
//   <brightBlack>Expected: "text"</brightBlack>
//   <brightBlack>Received: "\x1b[XYZtext"</brightBlack>
//
// <bold>Test Results:</bold>
// <green>2 passing</green>
// <red>1 failing</red>"

Integration with TAML Ecosystem

With Parser

import { encode } from "@taml/encoder";
import { parse } from "@taml/parser";
import { getAllText, getElementsWithTag } from "@taml/ast";

// Convert ANSI to TAML, then parse to AST
const ansiText = "\x1b[31mError:\x1b[0m \x1b[1mFile not found\x1b[0m";
const tamlMarkup = encode(ansiText);
const ast = parse(tamlMarkup);

// Analyze the parsed content
const plainText = getAllText(ast);
const errorElements = getElementsWithTag(ast, "red");
const boldElements = getElementsWithTag(ast, "bold");

console.log("Plain text:", plainText); // "Error: File not found"
console.log("Error count:", errorElements.length); // 1
console.log("Bold count:", boldElements.length); // 1

With React

import { encode } from '@taml/encoder';
import { parse } from '@taml/parser';
import { TamlRenderer } from '@taml/react';

function TerminalOutput({ ansiText }: { ansiText: string }) {
  // Convert ANSI to TAML, then parse to AST
  const tamlMarkup = encode(ansiText);
  const ast = parse(tamlMarkup);

  return (
    <div className="terminal">
      <TamlRenderer ast={ast} />
    </div>
  );
}

// Usage
const logOutput = '\x1b[32mINFO\x1b[0m: Server started on port \x1b[1m3000\x1b[0m';
<TerminalOutput ansiText={logOutput} />

With CLI Tools

# Convert ANSI output to TAML
echo -e "\033[31mHello\033[0m \033[1mWorld\033[0m" | taml-cli encode

# Process with Node.js
echo -e "\033[31mError:\033[0m Failed" | node -e "
  const { encode } = require('@taml/encoder');
  let input = '';
  process.stdin.on('data', chunk => input += chunk);
  process.stdin.on('end', () => {
    console.log(encode(input.trim()));
  });
"

Complete Processing Pipeline

import { encode } from "@taml/encoder";
import { parse } from "@taml/parser";
import { visit, transform, getAllText } from "@taml/ast";

// Complete ANSI → TAML → AST → HTML pipeline
const ansiOutput =
  "\x1b[1m\x1b[31mERROR:\x1b[0m \x1b[4mConnection failed\x1b[0m";

// 1. Convert ANSI to TAML
const tamlMarkup = encode(ansiOutput);
console.log("TAML:", tamlMarkup);
// "<bold><red>ERROR:</red></bold> <underline>Connection failed</underline>"

// 2. Parse TAML to AST
const ast = parse(tamlMarkup);

// 3. Extract information
const plainText = getAllText(ast);
console.log("Plain text:", plainText); // "ERROR: Connection failed"

// 4. Transform to HTML
const html = transform(ast, {
  visitDocument: (node) =>
    node.children.map((child) => transform(child, this)).join(""),
  visitElement: (node) => {
    const content = node.children
      .map((child) => transform(child, this))
      .join("");
    return `<span class="taml-${node.tagName}">${content}</span>`;
  },
  visitText: (node) => node.content,
});

console.log("HTML:", html);
// '<span class="taml-bold"><span class="taml-red">ERROR:</span></span> <span class="taml-underline">Connection failed</span>'

// 5. Custom analysis
let errorCount = 0;
let warningCount = 0;

visit(ast, {
  visitElement: (node) => {
    if (node.tagName === "red") errorCount++;
    if (node.tagName === "yellow") warningCount++;
  },
});

console.log(`Found ${errorCount} errors, ${warningCount} warnings`);

API Reference

Core Function

encode(ansiText: string): string

Converts ANSI escape sequences in text to TAML markup with proper nesting support.

Parameters:

• ansiText - Input text containing ANSI escape sequences

Returns:

• String with ANSI sequences converted to TAML tags

Example:

const result = encode("\x1b[31mRed text\x1b[0m");
console.log(result); // "<red>Red text</red>"

Color Mapping

The encoder maps ANSI color codes to standardized TAML tag names:

Standard Colors (30-37, 40-47)

| ANSI Code | Foreground | Background | | --------- | ---------- | ----------- | | 30, 40 | black | bgBlack | | 31, 41 | red | bgRed | | 32, 42 | green | bgGreen | | 33, 43 | yellow | bgYellow | | 34, 44 | blue | bgBlue | | 35, 45 | magenta | bgMagenta | | 36, 46 | cyan | bgCyan | | 37, 47 | white | bgWhite |

Bright Colors (90-97, 100-107)

| ANSI Code | Foreground | Background | | --------- | --------------- | ----------------- | | 90, 100 | brightBlack | bgBrightBlack | | 91, 101 | brightRed | bgBrightRed | | 92, 102 | brightGreen | bgBrightGreen | | 93, 103 | brightYellow | bgBrightYellow | | 94, 104 | brightBlue | bgBrightBlue | | 95, 105 | brightMagenta | bgBrightMagenta | | 96, 106 | brightCyan | bgBrightCyan | | 97, 107 | brightWhite | bgBrightWhite |

Text Styles

| ANSI Code | Style Name | TAML Tag | | --------- | ------------- | ----------------- | | 1 | Bold | <bold> | | 2 | Dim | <dim> | | 3 | Italic | <italic> | | 4 | Underline | <underline> | | 9 | Strikethrough | <strikethrough> |

Reset Sequences

| ANSI Code | Reset Type | Effect | | --------- | ---------------- | ------------------------ | | 0 | Full Reset | Closes all open tags | | 39 | Foreground Reset | Removes color formatting | | 49 | Background Reset | Removes background color |

Extended Color Support

256-Color Palette (38;5;n, 48;5;n)

The encoder maps 256-color palette indices to the closest standard TAML color:

// Examples of 256-color mapping
encode("\x1b[38;5;196mBright Red\x1b[0m"); // "<brightRed>Bright Red</brightRed>"
encode("\x1b[38;5;46mBright Green\x1b[0m"); // "<brightGreen>Bright Green</brightGreen>"
encode("\x1b[48;5;21mBlue Background\x1b[0m"); // "<bgBlue>Blue Background</bgBlue>"

RGB Colors (38;2;r;g;b, 48;2;r;g;b)

RGB values are converted to the nearest standard TAML color:

// Examples of RGB color mapping
encode("\x1b[38;2;255;0;0mRGB Red\x1b[0m"); // "<brightRed>RGB Red</brightRed>"
encode("\x1b[38;2;0;255;0mRGB Green\x1b[0m"); // "<brightGreen>RGB Green</brightGreen>"
encode("\x1b[48;2;0;0;255mRGB Blue BG\x1b[0m"); // "<bgBlue>RGB Blue BG</bgBlue>"

Advanced Topics

Performance Considerations

Efficient Processing

// For large amounts of text, consider chunking
function processLargeAnsiText(text: string, chunkSize = 10000): string {
  if (text.length <= chunkSize) {
    return encode(text);
  }

  const chunks = [];
  for (let i = 0; i < text.length; i += chunkSize) {
    chunks.push(encode(text.slice(i, i + chunkSize)));
  }

  return chunks.join("");
}

Memory Management

// For streaming applications, process line by line
function processAnsiStream(lines: string[]): string[] {
  return lines.map((line) => encode(line));
}

// Example with file processing
import { createReadStream } from "fs";
import { createInterface } from "readline";

async function processAnsiFile(filename: string): Promise<string[]> {
  const fileStream = createReadStream(filename);
  const rl = createInterface({ input: fileStream });

  const results: string[] = [];
  for await (const line of rl) {
    results.push(encode(line));
  }

  return results;
}

Error Handling Patterns

Safe Encoding

function safeEncode(ansiText: string): string {
  try {
    return encode(ansiText);
  } catch (error) {
    console.warn("Failed to encode ANSI text:", error);
    // Return original text as fallback
    return ansiText;
  }
}

Validation

function validateAnsiText(text: string): boolean {
  // Check for valid ANSI escape sequences
  const ansiRegex = /\x1b\[[0-9;]*m/g;
  const matches = text.match(ansiRegex);

  if (!matches) return true; // No ANSI sequences is valid

  // Validate each sequence
  return matches.every((match) => {
    const params = match.slice(2, -1); // Remove \x1b[ and m
    return /^[0-9;]*$/.test(params);
  });
}

function encodeWithValidation(ansiText: string): string {
  if (!validateAnsiText(ansiText)) {
    throw new Error("Invalid ANSI escape sequences detected");
  }
  return encode(ansiText);
}

Integration Patterns

Batch Processing

// Process multiple ANSI strings efficiently
function batchEncode(ansiTexts: string[]): string[] {
  return ansiTexts.map((text) => encode(text));
}

// With error handling
function safeBatchEncode(
  ansiTexts: string[],
): Array<{ input: string; output: string; error?: string }> {
  return ansiTexts.map((input) => {
    try {
      return { input, output: encode(input) };
    } catch (error) {
      return {
        input,
        output: input,
        error: error instanceof Error ? error.message : "Unknown error",
      };
    }
  });
}

Custom Processing Pipeline

// Create a processing pipeline
class AnsiProcessor {
  private preprocessors: Array<(text: string) => string> = [];
  private postprocessors: Array<(text: string) => string> = [];

  addPreprocessor(fn: (text: string) => string): this {
    this.preprocessors.push(fn);
    return this;
  }

  addPostprocessor(fn: (text: string) => string): this {
    this.postprocessors.push(fn);
    return this;
  }

  process(ansiText: string): string {
    // Apply preprocessors
    let text = this.preprocessors.reduce((acc, fn) => fn(acc), ansiText);

    // Encode ANSI to TAML
    text = encode(text);

    // Apply postprocessors
    return this.postprocessors.reduce((acc, fn) => fn(acc), text);
  }
}

// Usage
const processor = new AnsiProcessor()
  .addPreprocessor((text) => text.trim())
  .addPreprocessor((text) => text.replace(/\r\n/g, "\n"))
  .addPostprocessor((text) => text.replace(/\n/g, "<br>"));

const result = processor.process("\x1b[31mError\x1b[0m\r\n");

Integration with Logging Libraries

// Custom log formatter
import { encode } from "@taml/encoder";

class TamlLogFormatter {
  format(level: string, message: string, timestamp?: Date): string {
    const time = timestamp ? timestamp.toISOString() : new Date().toISOString();
    const coloredLevel = this.colorizeLevel(level);
    const ansiOutput = `\x1b[90m${time}\x1b[0m ${coloredLevel} ${message}`;
    return encode(ansiOutput);
  }

  private colorizeLevel(level: string): string {
    switch (level.toUpperCase()) {
      case "ERROR":
        return "\x1b[31mERROR\x1b[0m";
      case "WARN":
        return "\x1b[33mWARN\x1b[0m";
      case "INFO":
        return "\x1b[32mINFO\x1b[0m";
      case "DEBUG":
        return "\x1b[36mDEBUG\x1b[0m";
      default:
        return level;
    }
  }
}

// Usage
const formatter = new TamlLogFormatter();
const tamlLog = formatter.format("ERROR", "Database connection failed");
console.log(tamlLog);
// "<brightBlack>2024-01-15T10:30:15.123Z</brightBlack> <red>ERROR</red> Database connection failed"

Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

# Clone the repository
git clone https://github.com/suin/taml-encoder.git
cd taml-encoder

# Install dependencies
bun install

# Run tests
bun test

# Build the project
bun run build

# Lint and format
bun run lint
bun run format

Testing

The project uses Bun for testing with comprehensive test coverage:

# Run all tests
bun test

# Run tests in watch mode
bun test --watch

# Run specific test file
bun test encoder.test.ts

Test Coverage

The encoder is thoroughly tested with:

• Basic ANSI sequences: Colors, backgrounds, styles
• Complex formatting: Nested tags, multiple resets
• Extended colors: 256-color palette, RGB colors
• Edge cases: Malformed sequences, empty input
• Real-world examples: Git output, logs, progress bars
• Character escaping: Special character handling

License

MIT © suin

Part of the TAML ecosystem - Visit the TAML Specification for more information about the Terminal ANSI Markup Language.