visuail

Build visuals for AI chats and keep the AI aware of what was rendered.

visuail now has two layers:

• visuail: core parsing + visual context ledger
• visuail/react: React renderers for built-in block types

Together they let you define a visual block once and use it for both UI rendering and AI-readable memory.

The Problem

AI chat apps that render rich visuals (tables, charts, cards) face a universal issue: the AI loses awareness of structured data in its own visual output on subsequent turns. Follow-up questions like "which one had the highest price?" force redundant tool calls because the AI can't see what it already showed.

Install

npm install visuail

Quick Start

import { createLedger } from "visuail";

const ledger = createLedger();

// Transform render blocks into AI-readable context
const result = ledger.transform(assistantMessage, 1);
// → { text: "...[V1:table "Products" | 3 rows x 4 cols\n  ...]...", nextCounter: 2 }

// Expand a visual later using the same ledger instance
const expanded = ledger.expand(messages, "V3");
// → "[V3:table "Orders" | 47 rows x 5 cols\n  ..."

React Rendering

import { RenderBlocks } from "visuail/react";

<RenderBlocks content={assistantMessage} />;

Built-in React renderers are included for:

• table
• cards
• detail
• kpi
• chart
• gallery
• docs
• timeline

You can also combine built-in and custom renderers:

import { RenderBlock } from "visuail/react";

<RenderBlock
  type={type}
  data={data}
  customRenderers={{
    parts: PartsBlock,
  }}
/>;

How It Works

1. Your AI generates responses with <!--render:TYPE JSON--> blocks
2. Before the next turn, call ledger.transform() on assistant messages
3. Render blocks become [V1:table ...] entries the AI can read and reference
4. Large datasets get adaptively compressed (configurable thresholds)
5. The same ledger instance can later call expand() to get full uncompressed data on demand

Important Behavior

• transform() preserves the original message text exactly except for replacing render blocks.
• Visual IDs stay stable within a ledger instance, even if you start numbering at V3 or continue across many messages.
• expand() can still scan raw assistant messages as a fallback, but the most reliable path is to use the same ledger instance that performed the transforms.
• visuail/react is intentionally generic; app-specific actions, links, and custom blocks should be added through wrapper components.

Built-in Block Types

| Type | Description | Compression | |------|-------------|-------------| | table | Rows and columns | > 15 rows → 10 sample rows | | chart | Bar, line, scatter, pie | Scatter > 20 points → 10 samples | | cards | Items with images and fields | None | | kpi | Metrics with trends | None | | detail | Field/value pairs with sections | Nested tables follow table rules | | gallery | Image collections | None | | docs | Document download links | None | | timeline | Event history | > 10 events → 8 samples |

Custom Block Types

const ledger = createLedger({
  blockTypes: {
    // Add a new type
    "sales-summary": (data, id, compress) => `[V${id}:sales-summary ...]`,
    // Override a built-in
    table: (data, id, compress) => `[V${id}:table CUSTOM]`,
  },
});

Custom type names may include letters, numbers, underscores, and hyphens, as long as they start with a letter.

Custom Compression

const ledger = createLedger({
  compression: {
    tableRows: 20,        // compress tables above 20 rows (default: 15)
    tableSampleRows: 5,   // keep 5 sample rows (default: 10)
    timelineEvents: 15,   // compress timelines above 15 events (default: 10)
  },
});

Rendering Notes

Built-in formatters append a short generic note like (rendered as structured table visualization) by default.

const ledger = createLedger({
  renderNotes: false, // disable notes completely
});

Or provide your own resolver:

const ledger = createLedger({
  renderNotes: (type, data) => {
    if (type === "table" && data.title) return `table widget "${data.title}"`;
    return `${type} shown in the app UI`;
  },
});

Render Block Format

Blocks use HTML comment syntax:

<!--render:table {"columns":["Name","Price"],"rows":[["Widget","$10"]]}-->

The AI emits these in its response text. They're invisible in HTML renderers but parseable by visuail.

Design Notes

• If a block type is unknown, visuail now emits a compact payload summary instead of dropping the data entirely.
• Built-in rendering notes are intentionally generic so they do not imply UI affordances your app may not actually support.
• The React renderer and the context ledger share the same render-block contract, so the UI schema and the AI-memory schema do not drift apart.

License

MIT