@tailmark/core

React-free primitives for windowed repair and incremental block lexing of growing Markdown.

bun add @tailmark/core

Session API

import { createLexSession } from "@tailmark/core";

const session = createLexSession();

const first = session.update("# Result\n\nStreaming para");
const second = session.update("# Result\n\nStreaming paragraph complete.");

console.log(second.blocks, second.stableCount, second.openFence);
session.reset();

createLexSession retains an opaque incremental cache. Use lexMarkdownBlocks(previousCache, content) when cache ownership needs to remain with your application, or repairMarkdown(content) when only streaming repairs are needed.

Windowed repair semantics

Default repair starts at the final safe, fence-aware blank-line window. Bytes before that window stay raw and cannot influence repair decisions in the open tail. The one-shot and session APIs use the same definition, so their equivalence is structural. On LF append-only streams with reusable boundaries, the repair and lex stages both do O(open tail) work.

This is intentionally different from applying Remend to the whole document on every update. A tail-local construct is repaired from tail-local context, and a synthetic repair can revert to its original raw bytes after that region settles outside the window.

The fallbacks are explicit:

• Supplying any custom repairs handler preserves its full-string contract, so repair and lex use the whole-document pipeline on every update.
• Content containing CRLF or CR uses the whole-document pipeline because Marked normalizes token raw offsets. Normalize line endings to LF upstream to use the windowed path.
• Documents made entirely from indented code may not expose a safe reusable boundary. They remain correct and honestly fall back to full work.
• Non-append edits, global link-definition state, or a failed boundary check also retreat to the correct full pipeline.

Custom repairs

Repairs run after Tailmark's built-in Remend pass, in the array order supplied. The priority field is accepted for Remend compatibility but does not reorder caller repairs.

import { createLexSession, type RepairHandler } from "@tailmark/core";

const closeNextSteps: RepairHandler = {
  name: "close-next-steps",
  handle(markdown) {
    const opening = "<TRAYCER_NEXT_STEPS>";
    const closing = "</TRAYCER_NEXT_STEPS>";
    if (!markdown.includes(opening) || markdown.includes(closing)) return markdown;
    return `${markdown}\n${closing}`;
  },
};

const session = createLexSession({ repairs: [closeNextSteps] });
const result = session.update("<TRAYCER_NEXT_STEPS>\n- Review the diff");

Streaming-equivalence guarantee

For every prefix, the blocks returned by a cache-threaded update are identical to a one-shot windowed repair and full lex of that prefix. Tailmark tests this character by character over growing tables, loose and nested lists, Setext headings, lazy continuations, duplicate link definitions, blockquotes, and open code fences in test/lexer.test.ts. The same suite contains timing-free work bounds for the windowed repair and lex paths, plus regressions for every documented fallback.