brookmd
v0.29.1
Published
Zero-dep streaming markdown for the browser. Rust→WASM core, Web Worker per stream, incremental parse with speculative closure.
Maintainers
Readme
brookmd
Zero-dep streaming markdown for the browser. Rust→WASM core, one Web Worker per stream, incremental parse with speculative closure for mid-stream constructs.
Drop in a streaming-aware renderer — React, Vue, Svelte, Solid, a framework-agnostic <brook-markdown> Web Component, or the vanilla DOM mount — wire each LLM stream to a BrookClient, and the markdown renders incrementally off the main thread, block by block, with stable identities so unchanged blocks never re-reconcile.
Parsing runs entirely off the main thread — each stream gets its own pooled Web Worker, so many concurrent LLM responses render without contending for the UI thread. On each token the parser re-parses only the active tail, not the whole document; patches cross the worker boundary as verified splices (not full re-sends, so emitted bytes stay O(n) even for one giant growing block); and heavy renderers (math, mermaid) are deferred until a block closes — code fences highlight as they stream, re-tokenizing only the last line rather than the whole block per chunk. The result is low retained memory and a main thread that stays responsive while streaming. See the live demo.
Beyond the browser: the same Rust core also powers experimental React Native, Swift (iOS/macOS), Kotlin/Android, Flutter, and C-ABI bindings — every platform speaks the same versioned wire, byte-for-byte. See the platform matrix in the repository README.
Install
bun add brookmd # or: npm i brookmd / pnpm add brookmdbrookmd ships compiled, non-minified ESM (dist/*.js + .d.ts types) plus
the compiled WASM — no raw .ts/.tsx source. The worker and WASM asset are
referenced with the web-standard new URL(asset,
import.meta.url) pattern, so any bundler with asset-module support resolves
them: Vite (the reference setup), webpack 5, Rollup (with asset
modules), Parcel, and Next.js (App Router — Turbopack and webpack;
verified on Next.js 16, see the Next.js callout below).
The streaming client (<BrookMarkdown> / BrookClient) is browser-only (it
constructs Web Workers). For server-side / static rendering of finished
content — SSR, React Server Components, build steps — use the worker-free,
synchronous brookmd/server entry. The framework packages — react,
vue, svelte, solid-js — are all optional peer dependencies; you only
need the one whose binding you import. The framework-free entries
(brookmd/client, brookmd/dom, brookmd/element, and brookmd/server) need
none. (The bare brookmd entry re-exports the React component surface, so it
pulls react — import from brookmd/client if you want a framework-free core.)
Vite — one-line config. Vite's dependency pre-bundling (esbuild) hoists the wasm-bindgen glue into
.vite/deps/, which breaks the relativenew URL("…_bg.wasm", import.meta.url)lookup so the worker can't load WASM (you'll see a 404 / "magic word" error). Exclude brookmd from pre-bundling:// vite.config.ts export default defineConfig({ optimizeDeps: { exclude: ["brookmd"] }, });No other bundler needs this — it's specific to Vite's optimizer.
Next.js (App Router) — one requirement. Works on Next.js with Turbopack (the default for both
next devandnext build) or webpack. Since 0.17.0 brookmd ships compiled ESM, so notranspilePackagesor other build config is needed — earlier versions required it only because the package shipped raw TypeScript, which Next does not compile insidenode_modules. That no longer applies.Use it from a Client Component.
<BrookMarkdown>uses React hooks (and spawns a Web Worker on mount), so it must carry"use client"— it can't be a Server Component. (It is still SSR-safe: on the server it renders an empty shell and only starts streaming after hydration, so there's no SSR crash — the constraint is hooks, not the worker.)"use client"; import { BrookMarkdown } from "brookmd/react"; export default function Answer({ stream }: { stream: AsyncIterable<string> }) { return <BrookMarkdown stream={stream} />; }Create the
streamin Client Component code, not in a Server Component. AResponse/ReadableStream/AsyncIterableisn't serializable, so it can't be passed as a prop from a Server Component (e.g.page.tsx) — that throws "Only plain objects can be passed to Client Components." Pass a serializable prop (a URL, the chat messages) from the server and open the stream on the client — e.g.stream={await fetch("/api/chat")}from a client effect, or theuseBrookStreamhook (see Quick start).That's it — Turbopack bundles the worker and emits the
.wasmto_next/static/mediaitself, so no extra asset/loader config is needed (and the ViteoptimizeDepsworkaround above does not apply). Bothnext devandnext build && next startare verified to spawn the worker, load the WASM, and stream markdown. Dev tip: open the app onlocalhost— Next dev blocks cross-origin dev resources (HMR, chunks) from other hosts (e.g.127.0.0.1) unless you add them toallowedDevOriginsinnext.config.
Quick start
import { BrookClient, BrookMarkdown } from "brookmd";
// One client per stream. Spawns a Web Worker that owns a Rust parser.
const client = new BrookClient();
// Feed chunks as they arrive from your SSE / fetch reader.
for await (const delta of streamFromAi()) {
client.append(delta);
}
client.finalize();In React — pass the stream straight to <BrookMarkdown>. It owns the client,
pipes the stream, supersedes it if it changes, and cleans up on unmount:
import { BrookMarkdown } from "brookmd/react";
export function ChatMessage({ stream }: { stream: AsyncIterable<string> }) {
return <BrookMarkdown stream={stream} />;
}stream accepts an AsyncIterable<string> (e.g. SSE deltas), a Response, or
a ReadableStream<Uint8Array> — so <BrookMarkdown stream={await fetch("/api/chat")} />
works too.
Need the client handle (for outline() / getMetrics() / a shared client)? Use
the useBrookStream hook — same lifecycle, returns the owned client:
import { BrookMarkdown, useBrookStream } from "brookmd/react";
export function ChatMessage({ stream }: { stream: AsyncIterable<string> }) {
const client = useBrookStream(stream);
return <BrookMarkdown client={client} />;
}Already holding a growing string? — useBrookMarkdownString
Many apps keep the streaming message as a single growing string prop (it
re-renders with the full text-so-far each token), not as a stream. Feed that
string straight in — useBrookMarkdownString diffs it for you and forwards only
the delta, so you don't hand-roll an append/reset bridge:
import { BrookMarkdown, useBrookMarkdownString } from "brookmd/react";
export function ChatMessage({ text, streaming }: { text: string; streaming: boolean }) {
const client = useBrookMarkdownString(text, { streaming });
return <BrookMarkdown client={client} />;
}It handles the two shapes a controlled string takes: a prefix-extension (the
common token-by-token growth) appends only the new suffix; a divergence (e.g.
the finished text swapped for a re-processed final string — bolded numbers,
wrapped tickers) resets and reparses. Pass streaming: false once the content is
final so the last block commits (a finished code fence then highlights). The
framework-neutral primitive is client.setContent(fullString, { done }) —
use it from any binding.
Transforming streamed content? If the enrichment runs live per token (e.g. bold every number as it arrives), do it at render time via
components— keep the markdown source append-only so parsing stays incremental. Re-transforming the whole string each token (so earlier bytes change) forcessetContentto reparse every tick (O(n²)); that's what render-time overrides avoid.setContent's reset path is for the once-at-the-end reprocess swap, not per-token rewrites. That swap is seamless: the current view stays on screen while the new string reparses — the document never blanks, scroll never moves, and blocks whose rendered content is unchanged keep their identity (and React keys), so only genuinely changed blocks re-render. (setContent("")is an explicit clear and resets immediately.)
When you want to drive the stream yourself, pass a client you own — the
component never destroys it:
import { useEffect, useState } from "react";
import { BrookClient, BrookMarkdown } from "brookmd";
export function ChatMessage({ stream }: { stream: AsyncIterable<string> }) {
const [client] = useState(() => new BrookClient());
useEffect(() => () => client.destroy(), [client]);
useEffect(() => {
const ac = new AbortController();
client.pipeFrom(stream, { signal: ac.signal }); // pipeFrom also accepts AsyncIterable
return () => ac.abort();
}, [client, stream]);
return <BrookMarkdown client={client} />;
}StrictMode note: a stream (SSE generator /
Response) can be consumed only once, so React StrictMode's dev-only double-mount may truncate it in development. Production mounts once and is unaffected.
Multiple concurrent streams just need multiple clients — each runs in its own worker, so they don't share main-thread budget.
Framework bindings
BrookClient is framework-neutral — it owns the worker and exposes
subscribe/getSnapshot. Pick a renderer to put its blocks on screen. Every
binding below is thin glue over the same incremental DOM renderer, so they
share one identity contract: a committed block's node is never recreated, only
the streaming tail re-renders.
One ownership rule across all bindings: the renderer's teardown (React
unmount, handle.destroy(), element disconnect, etc.) frees only the rendered
DOM and the subscription — it never destroys the client. You call
client.destroy() when you're done with the stream. (React's <BrookMarkdown>,
documented below, is the same.)
Vanilla / any framework — brookmd/dom
import { BrookClient } from "brookmd/client";
import { mountBrookMarkdown } from "brookmd/dom";
const client = new BrookClient();
const handle = mountBrookMarkdown(client, document.getElementById("out")!, {
stickToBottom: true,
});
// Feed it from a fetch/SSE reader:
const reader = (await fetch("/api/chat")).body!.getReader();
const dec = new TextDecoder();
for (;;) {
const { value, done } = await reader.read();
if (done) break;
client.append(dec.decode(value, { stream: true })); // stream:true carries multibyte across chunks
}
client.append(dec.decode());
client.finalize();
// Teardown: destroy BOTH — the renderer and the client you created.
handle.destroy();
client.destroy();Already holding a growing string? There's no framework reactivity to wrap,
so just call client.setContent(fullString, { done }) instead of the
append loop — it diffs internally (prefix → delta; divergence → reparse) and
finalizes on done. That's the same primitive the React/Vue/Svelte/Solid
controlled-string helpers wrap; in vanilla you call it directly.
mountBrookMarkdown(client, container, options?) returns { destroy(), refresh() }.
Options: components, sanitize, virtualize, stickToBottom, highlightCode
(default true), streamingHighlight (default true — highlight a code fence while
it is still streaming; see Streaming syntax highlighting),
batch (default true — one DOM write per requestAnimationFrame),
morphOpenBlocks (default false — morph a growing generic open block's subtree in
place instead of rebuilding it via innerHTML, so only the changed parts repaint
and focus/selection in the streaming tail survive; the rendered result is
equivalent to the default rebuild path).
Block-kind overrides use components keyed by block-kind (CodeBlock, Table,
Alert, Component, …) with values (props) => HTMLElement | string. Tag-level
(lowercase a/table/code) overrides are React-only — there's no virtual
tree on the fast innerHTML path; a block-kind override can rewrite the html
it's handed instead.
Web Component <brook-markdown> — brookmd/element
The universal binding — plain HTML, Angular, or any framework that renders DOM. Register once, then use the element:
import { defineBrookMarkdown } from "brookmd/element";
defineBrookMarkdown(); // defines <brook-markdown>; pass a custom tag name if you like<!-- zero-JS streaming straight from a URL -->
<brook-markdown src="/api/post.md" gfm-math stick-to-bottom></brook-markdown>
<!-- one-shot from inline text -->
<brook-markdown># Hello **world**</brook-markdown>// or caller-owned streaming — drive your own client:
const el = document.querySelector("brook-markdown");
el.client = myBrookClient; // element subscribes; never destroys it
el.components = { Thinking: (p) => myNode(p) };
myBrookClient.append(delta);Config flags are tri-state attributes: absent = library default;
gfm-math / gfm-math="true" / ="1" = on; gfm-math="false" / ="0" = off
(the only way to turn off a default-on flag such as gfm-alerts). It renders in
light DOM so your markdown CSS applies, and defineBrookMarkdown is a no-op under
SSR (no customElements). A self-owned element (src / markdown / inline
text / append()) is torn down on disconnect; a caller-supplied client is left
alone.
Angular consumes the same element — no separate package:
import { Component, CUSTOM_ELEMENTS_SCHEMA } from "@angular/core";
import { defineBrookMarkdown } from "brookmd/element";
defineBrookMarkdown(); // once at bootstrap
@Component({
standalone: true,
schemas: [CUSTOM_ELEMENTS_SCHEMA],
template: `<brook-markdown [attr.src]="url" stick-to-bottom></brook-markdown>`,
})
export class Answer { url = "/api/post.md"; }Controlled growing string? Assign a caller-owned client and drive it with
setContent — el.client = myClient; myClient.setContent(fullString, { done })
— the element subscribes and renders, you own the diffing. (The self-owned
markdown attribute is one-shot — it re-parses the whole document on each
change, so don't point it at a per-token-growing string; use a client +
setContent for that.)
Vue 3 — brookmd/vue
<script setup lang="ts">
import { onBeforeUnmount } from "vue";
import { BrookClient } from "brookmd/client";
import { BrookMarkdown } from "brookmd/vue";
const client = new BrookClient();
// feed client.append(delta) from your stream, then client.finalize()
onBeforeUnmount(() => client.destroy());
</script>
<template>
<BrookMarkdown :client="client" stick-to-bottom />
</template>Props: client (required), components, sanitize, virtualize,
stickToBottom. There's also a useBrookMarkdown composable returning a
container ref if you'd rather mount into your own element.
Already holding a growing string? useBrookMarkdownString owns a client and
diffs the string for you (the Vue analogue of the React hook — see
Controlled strings):
<script setup lang="ts">
import { BrookMarkdown, useBrookMarkdownString } from "brookmd/vue";
const props = defineProps<{ text: string; streaming: boolean }>();
// Pass getters so the composable tracks the live values; it owns + destroys the client.
const client = useBrookMarkdownString(() => props.text, () => ({ streaming: props.streaming }));
</script>
<template><BrookMarkdown :client="client" /></template>Svelte (4 & 5) — brookmd/svelte
A Svelte action — works in both v4 and v5, no .svelte build step:
<script lang="ts">
import { onDestroy } from "svelte";
import { BrookClient } from "brookmd/client";
import { brookMarkdown } from "brookmd/svelte";
const client = new BrookClient();
// feed client.append(delta) then client.finalize()
onDestroy(() => client.destroy());
</script>
<div use:brookMarkdown={{ client, stickToBottom: true }} />Growing string? The brookMarkdownString action owns a client and diffs the
string — use:brookMarkdownString={{ content, streaming }} (it destroys its
client on destroy, so no manual cleanup):
<script lang="ts">
import { brookMarkdownString } from "brookmd/svelte";
export let content: string; // the growing message
export let streaming: boolean; // false once complete → finalizes
</script>
<div use:brookMarkdownString={{ content, streaming, stickToBottom: true }} />Solid — brookmd/solid
import { onCleanup } from "solid-js";
import { BrookClient } from "brookmd/client";
import { BrookMarkdown } from "brookmd/solid";
const client = new BrookClient();
// feed client.append(delta) then client.finalize()
onCleanup(() => client.destroy());
<BrookMarkdown client={client} stickToBottom />;Growing string? createBrookMarkdownString owns a client and diffs the string
(the Solid analogue of the React hook), driving setContent from a
createEffect and destroying the client on cleanup:
import { BrookMarkdown, createBrookMarkdownString } from "brookmd/solid";
function Message(props: { text: string; streaming: boolean }) {
const client = createBrookMarkdownString(() => props.text, () => ({ streaming: props.streaming }));
return <BrookMarkdown client={client} />;
}The Solid binding's mount/teardown logic is tested, but its JSX component shell
has so far only been exercised through a real Solid (vite-plugin-solid) build
in development, not in CI — treat it as the newest of the bindings and file an
issue if your Solid setup trips on it. The component is a thin ref'd <div>;
if you hit a transform edge, mountBrookMarkdown from brookmd/dom inside
onMount/onCleanup is the zero-surprise fallback.
Server-side rendering
<BrookMarkdown> / BrookClient are browser-only (they spawn a Web Worker), but
the Rust→WASM core is a plain synchronous parser. So brookmd/server renders
finished markdown on the server with no worker and no async ceremony — Node
SSR, React Server Components, or a build step:
import { initBrook, renderToString } from "brookmd/server";
await initBrook(); // once at startup (loads the WASM)
const html = renderToString("# Hello\n\n**world**"); // sync HTML string, no workerFor React server rendering (RSC, static generation, or SSR), use
<BrookMarkdownStatic> from brookmd/server/react — a hookless, RSC-safe
component that renders finished content with the same components overrides
(inline/block component tags dispatch on the server too). It lives in its own
subpath so the core brookmd/server above stays importable with no react
installed:
import { initBrook } from "brookmd/server";
import { BrookMarkdownStatic } from "brookmd/server/react";
await initBrook();
export default function Doc({ md }: { md: string }) {
return (
<BrookMarkdownStatic
content={md}
config={{ inlineComponentTags: ["tik"] }}
components={{ tik: ({ symbol }) => <span className="ticker">{symbol}</span> }}
/>
);
}initBrook()— async, idempotent. In Node it reads the package's.wasmoff disk (Node'sfetchcan't loadfile://); on the web it fetches the bundler-resolved asset. On edge runtimes pass bytes yourself:initBrookSync(wasmBytes).renderToString(md, { config })— synchronous HTML string, zero React dependency (imports cleanly with noreactinstalled).parseToBlocks(md, { config })— the block array, for custom rendering.
Document assembly. A Block.html never ends with a newline — the terminator
that follows a top-level block belongs to the document, not the block.
renderToString therefore joins blocks with cmark's cr() rule: insert \n
before a block only when the output doesn't already end with one, and end the
document with one \n. An unconditional "\n".join(...) would double the
newline a raw HTML block serializes for itself. If you assemble
parseToBlocks output into a document string yourself, use the same rule —
it's what makes the output byte-identical to a reference CommonMark/GFM renderer
(652/652 CommonMark 0.31 and 24/24 GFM extension examples, byte-for-byte). See
WIRE.md §12.
<BrookMarkdownStatic content config components />(frombrookmd/server/react) — synchronous React tree for render-once contexts; render it with your framework's server renderer (renderToStaticMarkup, RSC, …). For live streaming, client-side code highlighting, or Mermaid, render<BrookMarkdown>on the client instead — it's a separate component. (If you SSR-then-hydrate, use the same component on both sides; the dedicated client renderers in<BrookMarkdown>don't hydrate<BrookMarkdownStatic>'s plainer markup.)
What it does
| Concern | brookmd | conventional main-thread renderer |
|---|---|---|
| Re-parse on each token | No — only the active tail | Yes, full string |
| Where parse runs | Web Worker (off main thread) | Main thread |
| Block identity across chunks | Stable monotonic IDs | New keys on every render |
| Mid-stream unclosed ``` / * / ** | Speculatively closed in render, replaced cleanly | Often renders raw or breaks |
| Half-streamed link [label](https://… | Label-only inert anchor (data-brook-pending), URL never leaks | Raw brackets + partial URL flash |
| Heavy renderers (syntax, math, mermaid) | Deferred until block close | Re-run per chunk |
| XSS sanitization | Allowlist in Rust + URL scheme check | Downstream sanitizer pass on the JS thread |
Streaming links
A link's destination is the last thing a model emits — [Earnings Call](https://…
often spans many tokens. While a link is still streaming, brookmd renders it as
an inert, label-only anchor: the label text inside an <a> with no href
(the half-typed URL never flashes on screen), marked so you can style it:
<a data-brook-pending="" target="_blank" rel="noopener noreferrer nofollow">Earnings Call</a>An anchor without an href gets no default link styling from the browser, so
without a rule for the marker the link would "pop" blue only when the URL
completes. The bundled theme (import "brookmd/styles.css") already styles it;
if you bring your own CSS, copy this:
.brook-md a[data-brook-pending] {
color: var(--brook-accent, #0969da); /* match your settled link's resting style */
cursor: default; /* not clickable yet */
}The moment the closing ) arrives, the href appears and data-brook-pending
is dropped — committed and finalized output never carry the marker, and the
finished block is byte-identical to a one-shot parse. Two composition notes:
urlTransform runs only on a real href, so it never sees a half-streamed URL
prefix — only the complete one (it may run again on re-renders while the
surrounding block is still open); decorators skip text
inside <a> by default (skipInside), pending or not.
Styling
brookmd emits semantic HTML under a .brook-md root and ships no CSS by
default — bring your own design system, or opt into the bundled theme:
import "brookmd/styles.css";It gives good-looking output out of the box, including the built-in syntax
highlighter's colors (without any CSS, highlight() renders uncolored). The
theme is scoped to .brook-md, zero-runtime, and does not change the rendered
HTML — skip the import and nothing is styled.
Next.js Pages Router:
brookmd/styles.cssis global CSS, which the Pages Router only allows importing frompages/_app. Import it there (App Router and other bundlers can import it from any component). Or skip it and bring your own.brook-mdstyles.
Re-theme by overriding a few CSS variables; it's light by default and switches to
dark automatically via prefers-color-scheme (force a mode with
class="brook-md brook-dark" or brook-light):
.brook-md {
--brook-accent: #7c3aed; /* links */
--brook-bg-code: #faf7ff; /* code background */
--brook-t-kw: #c026d3; /* syntax: keywords (also --brook-t-str/num/com/fn/ty/…) */
}Public API
BrookClient
class BrookClient {
constructor(options?: {
pool?: BrookPool;
config?: ParserConfig;
onError?: (err: { message: string; fatal?: boolean }) => void; // worker/parse + WASM-init errors
onBlock?: (block: Block) => void; // fires once per block as it commits
recovery?: boolean; // auto-heal a transient worker death (default true)
});
get failed(): Error | null; // terminal worker failure, else null (null through heals)
append(chunk: string): void; // queue text for parsing
pipeFrom( // read → append → finalize
src: ReadableStream<Uint8Array> | Response | AsyncIterable<string>,
opts?: { signal?: AbortSignal }, // abort to supersede (no finalize)
): Promise<void>;
finalize(): void; // mark stream complete
setContent( // drive from a controlled full string
full: string, // diffs vs last: prefix → append delta; else seamless
opts?: { done?: boolean }, // reset+reparse (view held, unchanged blocks keep identity)
): void; // done:true → finalize
reset(): void; // wipe and reuse
destroy(): void; // free this stream's parser
whenReady(): Promise<void>; // resolves once WASM loaded; rejects on init failure
subscribe(listener: () => void): () => void; // React-friendly store
getSnapshot(): Block[]; // ordered current blocks
getPersistable(source?: string): PersistableSnapshot; // capture the rendered doc as JSON
hydrate( // restore it: no worker, no parse
snapshot: PersistableSnapshot,
opts?: { source?: string }, // source ⇒ a live thread can resume
): void;
outline(): { level: number; text: string; id: number }[]; // heading table-of-contents (works mid-stream)
toPlaintext(): string; // rendered document as plain text (search / summaries)
getMetrics(): { bytes, patches, totalParseMs, throughputKBs,
retainedBytes, wasmMemoryBytes, ... };
}pipeFrom is the LLM-native shortcut — hand it a fetch response and it
reads, appends, and finalizes for you:
const client = new BrookClient();
await client.pipeFrom(await fetch("/api/chat")); // streams the body in, then finalizesPass onError to be notified of worker/parse errors and a fatal WASM-init
failure ({ fatal: true }); without it, errors are only console.error'd and a
load failure surfaces as a rejected whenReady(). Pass onBlock to run a side
effect each time a block commits (e.g. lazy-highlight a finished code block).
A transient worker death heals invisibly by default: if a worker dies
mid-stream (e.g. a stale hashed worker URL 404s after a redeploy), the client
buffers the driven document, re-acquires a fresh worker, and re-feeds it once —
the view stays on screen and onError does not fire. Only if the replacement
also dies is the failure terminal (onError with { fatal: true }, and
client.failed becomes the Error; it is null while healthy and through a
successful heal). Set recovery: false to disable the buffer and auto-recovery
(a fatal death is then immediately terminal) — worth it for memory-sensitive,
very large documents where retaining the full source is undesirable.
Per-stream config
const client = new BrookClient({
config: {
gfmAutolinks: true, // bare www./http(s):// URLs + emails → links (default true)
gfmAlerts: true, // > [!NOTE] → callouts (default true)
gfmTagfilter: false, // GFM disallowed raw HTML: escape <script>/<title>/… under unsafeHtml (default false)
gfmFootnotes: true, // [^1] + [^1]: → footnote section (default false)
gfmMath: true, // $…$ / \(…\) inline + $$…$$ / \[…\] display math (default false)
dirAuto: true, // per-block dir="auto" for RTL/bidi text (default false)
softBreaks: true, // a single \n renders as <br> (remark-breaks / chat convention; default false)
lenientLists: true, // marker + 6+ SPACES → item text, not indented code (default false)
a11y: true, // task-list <label> + <th scope="col"> a11y markup (default false)
unsafeHtml: false, // pass raw HTML through (default false — keep it false for untrusted input)
componentTags: ["Thinking", "Callout"], // BLOCK custom tags w/ markdown inside (default none)
inlineComponentTags: ["tik", "cite"], // INLINE custom tags (chips/citations) w/ markdown inside (default none)
htmlAllowlist: ["br", "sub", "sup"], // safe raw-HTML sanitizer: [] = allow all but dangerous; list = only those (default off)
dropHtmlTags: [], // tags removed entirely (comments always dropped when sanitizing; default off)
blockHtml: true, // extend the sanitizer to BLOCK raw HTML (<details>…); needs a list above (default false)
allowSchemes: ["file"], // un-block a default-blocked URL scheme (default none — see "Security")
blockData: true, // opt-in structured kind.data per block (default false — see "Structured block data")
},
});Omitted fields use the defaults above, so new BrookClient() is unchanged.
Config is applied when the stream's parser is created and is immutable for
that stream (reset() keeps it; use a new client for different flags).
When to enable each flag:
gfmAutolinks— on by default. Leave it on unless you want strict CommonMark.gfmAlerts— on by default. Leave it on unless you want strict CommonMark.gfmMath: true— when your LLM emits$…$or$$…$$(or LaTeX\(…\)/\[…\]). brookmd emits KaTeX-ready markup; you bring the KaTeX pass (orcomponents.MathBlock).gfmFootnotes: true— when your input uses[^1]references and[^1]:definitions. Off by default; see the footnote streaming caveat above.dirAuto: true— when content can be RTL / mixed-direction. Emits per-blockdir="auto"so the browser detects direction independently per block.lenientLists: true— when your LLM over-indents after a list marker. Strict CommonMark (§5.2) says a marker followed by 5 or more columns of whitespace starts an indented code block, so a model writing- const value = 1;renders as a<pre><code>block instead of a list item. This flag raises that cutoff to 6 columns of literal spaces: at 6+ the padding is absorbed into the item's content column and the text renders as the item's own markdown (inline formatting, links, and nested lists all parse normally). Off by default, so strict-CommonMark output is unchanged.Four cases stay strictly conformant by design — the flag is deliberately narrow, not a general "fix my indentation" pass:
| Input | Stays | Why | | --- | --- | --- | |
-+ exactly 5 spaces | code block | 5 columns is the §5.2 boundary itself; relaxing it would swallow genuine one-space-past-the-minimum code | |- ```js(fence on the marker line) | fenced code | the marker line opens a real fence — there is no over-indentation to undo | |-then code indented on a later line | code block | the decision reads only the marker's own line; a later-line indent is unambiguous authored code | |-\t\tfoo(tab padding) | code block | tab padding is a deliberate authoring choice, unlike model over-indentation which is always literal spaces |Excluding tabs is what keeps the divergence from CommonMark down to a single spec example (274,
1.+ 6 spaces). Everything else in the 652-example suite renders identically with the flag on or off; the conformance suites themselves run in strict mode and are unaffected.The rule is a pure per-line comparison made when the marker is first scanned, so it costs no lookahead and no re-parse while streaming.
a11y: true— opt-in accessibility markup that deviates from strict GFM byte-output: wraps task-list checkboxes in a<label>(screen-reader association) and addsscope="col"to table headers. Off by default so conformance output stays exact.unsafeHtml: true— only when rendering trusted HTML. For untrusted / LLM-produced HTML, pair this with<BrookMarkdown sanitize={…} />(DOMPurify or similar — see Security).gfmTagfilter: true— the GFM "Disallowed Raw HTML" extension, for use withunsafeHtml: the nine disallowed tags (<title>,<textarea>,<style>,<xmp>,<iframe>,<noembed>,<noframes>,<script>,<plaintext>) get their leading<escaped so they display as text instead of taking effect — opening and closing forms, any case, in blocks and inline. Off by default (strict CommonMark passes them through underunsafeHtml); it's a tag denylist, not a sanitizer — untrusted input still wantssanitize.componentTags: ["Thinking", …]— when your LLM emits block custom tags like<Thinking>…</Thinking>(on their own line) and you want their inner content parsed as markdown and dispatched to a React component. Safe withoutunsafeHtml(attributes are sanitized; allowlisted tags only).inlineComponentTags: ["tik", …]— same idea for inline custom elements that sit inside a paragraph, heading, list item, or table cell (ticker chips, citations,@mentions). See Inline component tags.htmlAllowlist/dropHtmlTags— render a safe subset of raw HTML (e.g.<br>,<sub>,<sup>) natively withoutunsafeHtml, drop specific tags, and drop HTML comments. See Safe raw HTML.blockHtml: true— extend that sanitizer to block-level raw HTML, so a<details><summary>…</summary>…</details>block renders as real elements instead of an escaped code block. Needs one of the two lists above to be set;<script>/<pre>/<style>/<textarea>blocks stay escaped. See Block-level raw HTML.allowSchemes: ["file"]— un-block a URL scheme brookmd blocks by default, for privileged hosts (Electron, extensions) that intercept link clicks instead of navigating. Script-executing schemes can never be re-enabled. See Un-blocking a scheme.
Footnotes (gfmFootnotes) work in streaming with one honest caveat: a
[^1] reference renders speculatively the moment it's seen (committed blocks
can't re-render), and the footnote section is emitted at finalize. So a
reference whose definition never arrives leaves a dangling link — the same
forward-reference cost as link reference definitions. Multiple references to
the same footnote each get a unique id (fnref-N, fnref-N-2, …) and the
definition lists one backref per reference. Remaining v1 limits:
single-block definitions (no continuation-indent / multi-paragraph) and no
nested footnotes. The section uses GitHub-style markup
(<section class="footnotes">, <sup class="footnote-ref">).
Math (gfmMath) recognizes both delimiter families LLMs emit — $…$ /
$$…$$ and LaTeX \(…\) / \[…\]. Inline math renders to
<span class="math math-inline">…</span>, display math to
<div class="math math-display">…</div> (and inline display to a math-display
span), each carrying the HTML-escaped LaTeX as its text content — exactly
what KaTeX's auto-render / rehype-katex consume. brookmd
stays zero-dep: it produces the KaTeX-ready markup and never processes the
body as markdown; you bring the KaTeX pass (or override components.MathBlock,
which receives the raw LaTeX as text). Single $ uses the pandoc rule so
prose and currency stay literal — the opener needs a non-space to its right, the
closer a non-space to its left and no digit after it, so $5 and $10 is not
math. A $$/\[ block is blank-line tolerant (multi-line \begin{aligned}…
stays one block) and renders incrementally while streaming, like a code fence.
Off by default (so $ in plain prose is untouched) — enable it per stream when
your model emits LaTeX.
Bidirectional text (dirAuto) emits dir="auto" on each block-level text
element (p, h1–h6, blockquote, ul/ol/li, table), so the browser
runs the Unicode bidi algorithm per block — an Arabic/Hebrew paragraph
renders RTL while the English one beside it stays LTR, with no JS direction
detection. Code blocks never get it (code is always LTR). This is the per-block
model GitHub uses; it's the right fix for the common failure mode of detecting
one direction for a whole mixed-language document. Off by default (strict
CommonMark output is unchanged); turn it on for RTL or mixed-direction content.
BrookMarkdown (React)
Subscribes to a BrookClient, renders each block keyed by its stable parser-assigned ID. Memoized so unchanged blocks never re-reconcile.
<BrookMarkdown client={client} />The root element accepts opt-in className (appended to brookmd), id,
role, and aria-live / aria-atomic. Set aria-live="polite" to make the
output a live region so screen readers announce streamed content as it settles —
polite coalesces rapid updates and does not read every token. The same
options exist on the DOM mount (mountBrookMarkdown(client, el, { ariaLive: "polite" })),
covering the Web Component and the Vue/Svelte/Solid adapters.
Custom components / overrides
Pass a components map to replace how elements render. Keys come in two
namespaces:
import { useMemo } from "react";
import { BrookClient, BrookMarkdown, type Components } from "brookmd";
function Message({ client }: { client: BrookClient }) {
// Memoize (or hoist to module scope). A fresh object every render busts
// BrookMarkdown's block memo, so every block re-parses on every patch.
const components: Components = useMemo(
() => ({
// tag-level (lowercase HTML names) — applied inside a block's HTML
table: (props) => <table className="rounded border" {...props} />,
a: (props) => <a target="_blank" rel="noreferrer" {...props} />,
h1: "h2", // a string value just swaps the tag
// block-kind (capitalized BlockKindTag) — replaces the whole block
CodeBlock: ({ text, language, open }) => (
<MyCodeBlockWithCopyButton code={text} lang={language} streaming={open} />
),
// GitHub alerts (`> [!NOTE]` / `[!TIP]` / `[!WARNING]` / `[!CAUTION]` /
// `[!IMPORTANT]`) — swap in your own callout component. The alert kind
// is on `block.kind.data.kind`; `html` is the rendered inner body.
Alert: ({ block, html }) => (
<MyCallout kind={(block.kind.data as { kind: string }).kind}>
<div dangerouslySetInnerHTML={{ __html: html }} />
</MyCallout>
),
}),
[],
);
return <BrookMarkdown client={client} components={components} />;
}Tag-level keys (table, thead, tr, td, a, code, pre, h1–h6,
ul, ol, li, blockquote, p, img, del, input, hr, …) replace that
element wherever it appears. The component receives the element's parsed
attributes (with class→className and style as an object) plus children.
Block-kind keys (CodeBlock, Mermaid, MathBlock, Alert, Paragraph,
Heading, List, Blockquote, Table, Rule, Html) replace the entire
block. The component receives BlockComponentProps: { block, html,
open, speculative }, plus text/language for code/math blocks — and meta,
the rest of a fence's info string (```ts title="src/main.ts"), for a
filename header (the alert type is at block.kind.data.kind).
One map, two prop contracts — the single biggest footgun. The keys above are looked up by TWO dispatchers. The block-kind dispatcher passes
BlockComponentProps(withblock); the element dispatcher, which is what makesa/code/tableoverrides work, passes the element's attributes andchildrenonly — noblock. The same name can hit both: aninlineComponentTagschip, or acomponentTagstag that lands inside a list item or blockquote (where it is a real nested Component block, rendered as an element inside its container's HTML), takes the element path. So an override that readsprops.block.…throwscan't access property "kind", block is undefinedfor those occurrences — intermittently, because it depends on where the model happened to put the tag.Write any override for a name that can appear in both positions defensively:
const Thinking = ({ block, children }) => block ? <Panel data={block.kind.data}>{children}</Panel> : <span>{children}</span>;Three things make this survivable rather than fatal: block-kind keys are typed to
BlockComponentProps, so a mismatched override is a compile error; a raw element whose name collides with a block-kind key (<Table>,<Alert>… — only reachable with raw-HTML passthrough on) is never dispatched to that override; and every block renders inside its own error boundary, so a throwing override costs that one block instead of unmounting the document. WireonBlockErrorto see them.
Rules worth knowing:
- There is no
nodeprop / no hast tree. Introspect viaclassName/data-*, or — better — opt into the typed structured-data channel (blockData: true) and readblock.kind.data(and the typedprops.table/heading/code/math/listfields) directly — no HTML re-parsing. - Overrides apply to the OPEN (streaming) block too, not just settled ones —
so a design-system renderer (Tailwind classes on
p/ul/li, inline<a>/<code>overrides) stays styled mid-stream. The tail's HTML is always well-formed (the parser speculatively closes it). If asanitizeis supplied it runs first, on every block. - No
componentsprop ⇒ the original fast path (innerHTML, byte-identical output). The HTML→React conversion runs only when you actually supply overrides, and is memoized per(block id, html)so committed blocks don't re-parse as the stream grows. - For code blocks the built-in highlighter is the default; it is bypassed
(so your override wins) when you pass
components.CodeBlock,components.pre, orcomponents.code.
Inline text decorators
Wrap or replace matched inline text while streaming — e.g. bold financial
figures — without writing your own HTML re-parser. A decorators entry runs
POST-parse on real inline text nodes only (never URLs, code, or markup), once
per committed block, so a long document stays O(n).
import { BrookMarkdown, wrapLink } from "brookmd";
// HOIST it (module scope) or memoize — a fresh identity each render busts the
// per-block memo and re-decorates every block on every patch (a dev warning fires).
const decorators = [
{ match: /\$[\d.]+[BMK]|FY\d{4}|\d+(?:[-–]\d+)?%/g, replace: (t) => <mark>{t}</mark> },
// Linkify a ticker — route the href through the safe helper (see below):
{ match: /\$[A-Z]{1,5}\b/g, replace: (t) => wrapLink(t, { href: `/sym/${t.slice(1)}` }) },
];
<BrookMarkdown client={client} decorators={decorators} />;- Trusted surface — not sanitized. A decorator's
replaceoutput is spliced straight into the tree and does not pass through brookmd's attribute sanitizer (React renders ajavascript:href without complaint). Treatdecoratorsexactly likecomponents: build only trusted nodes, and route any link href throughwrapLinkor the exportedsafeUrl. skipInsidedefaults to['a','code','pre','kbd']; override per decorator.- Per-text-node. A value split by inline markup (e.g.
$2.<em>5</em>B) is two text nodes and won't match across them — match against settled, contiguous text. - Matching is pure and stateless, so a value streamed char-by-char decorates
identically to a one-shot render. Same API on
brookmd/dom(mountBrookMarkdown(client, el, { decorators })); a decorator there returns aNodeor string.
urlTransform?: (url, { tag, attr }) => string rewrites href/src/poster
URLs as blocks render (proxy images, add UTM params). Its output is re-sanitized
(safeUrl(urlTransform(safeUrl(value)))), so a buggy transform can never emit a
javascript: / data:text/html URL. Hoist/memoize it for the same reason as
decorators.
Structured block data (setBlockData)
Set blockData: true in the per-stream config and each block carries typed
structured data on block.kind.data, also surfaced as typed fields on the
component props — so you build toolbars, tables of contents, charts, copy
buttons, etc. from data, never by re-parsing the rendered HTML (no hast tree,
no rehype). Off by default; when off, output and CommonMark/GFM conformance are
byte-identical, so non-users pay nothing.
| Kind | block.kind.data | prop | use |
|------|-------------------|------|-----|
| Table | { headers, rows, aligns }, cells { text, html } | props.table | sort / filter / transpose / CSV / chart |
| Heading | { level, text, id } | props.heading | table of contents with anchors |
| CodeBlock | { lang, meta?, code } | props.code | decoded source (copy / run) |
| MathBlock | { latex } | props.math | LaTeX source (re-render) |
| List | { ordered, start } | props.list | ordered-list numbering |
Each cell's text is inline-stripped plaintext (for sort/filter/CSV/logic);
html is the inline-rendered display HTML. The data streams with the
document — a growing table or a heading carries its structured data on every
patch, in lock-step with the HTML — something a batch HTML-AST cannot do.
// Table of contents from heading data — no DOM, works mid-stream:
const toc = client.getSnapshot()
.filter((b) => b.kind.type === "Heading" && b.kind.data)
.map((b) => b.kind.data as { level: number; text: string; id: string });Component tags
LLMs increasingly emit custom component tags like <Thinking>…</Thinking>. By
default these are inert (escaped, or — with unsafeHtml — raw HTML whose body
is not markdown). Opt in by allowlisting the tag names:
const client = new BrookClient({ config: { componentTags: ["Thinking", "Callout"] } });Now a listed tag is a markdown container: its inner content is parsed as
markdown, it spans blank lines up to its matching close tag (not split like a
raw HTML block), it nests, and a </Tag> inside a code fence stays content. It's
safe without unsafeHtml — the tag is allowlisted and its attributes are
sanitized (event handlers dropped, dangerous URL schemes → #).
Each renders as a Component block. Override it in React by tag name (or with
the generic Component fallback). The override receives tag, the sanitized
attrs, the inner content as ready-to-render children (the easy path), and
also html (the inner already-rendered markdown string, for
dangerouslySetInnerHTML):
<BrookMarkdown
client={client}
components={{
Thinking: ({ children }) => (
<details className="thinking">
<summary>Reasoning</summary>
{children}
</details>
),
}}
/>
childrenvshtml. AComponentoverride that renders neither shows empty (a common first-try gotcha). Preferchildren— a parsed React tree with nested overrides applied; reach fordangerouslySetInnerHTML={{ __html: html }}only when you need the raw string.attrskeys are React-form (class→className,for→htmlFor) so{...attrs}spreads cleanly. While streaming, both reflect the partial inner content and re-render as more arrives. With no override the block renders as<thinking …>…</thinking>. Tag names match case-sensitively; off unlesscomponentTagsis set.
Inline component tags
componentTags handles block containers (a <Thinking> on its own line). For
inline custom elements — ticker chips, citations, @mentions, inline tooltips
that sit inside a paragraph, heading, list item, or table cell — use
inlineComponentTags:
const client = new BrookClient({ config: { inlineComponentTags: ["tik"] } });
<BrookMarkdown
client={client}
components={{
tik: ({ symbol, children }) => <span className="ticker">{children ?? symbol}</span>,
}}
/>;Now Apple <tik symbol="AAPL">AAPL</tik> rose 2% (or self-closing
<tik symbol="AAPL"/>) dispatches the inline <tik> to components.tik: its
inner is parsed as inline markdown (the children), its attributes become
props, and it's safe without unsafeHtml (attributes sanitized, allowlisted
tags only). It works everywhere inline content does — including table cells.
Tag names match case-sensitively and dispatch verbatim to components[tag]
(<tik>→components.tik, <Cite>→components.Cite). The
two lists are independent: list a tag under componentTags for blocks,
inlineComponentTags for inline, or both for both.
Where an allowlisted tag actually lands:
| Position | Result |
| --- | --- |
| Own line, top level | block Component — override gets BlockComponentProps |
| Own line inside a list item / blockquote | real nested Component block, emitted as an element inside the container's HTML — the override is dispatched by element name, so it gets attributes + children and no block |
| Mid-paragraph, listed in inlineComponentTags | inline element — attributes + children, no block |
| Mid-paragraph, NOT listed in inlineComponentTags | escaped text |
| Inside a table cell | escaped text (cells are inline-only) |
An allowlisted tag in a position that is not supported degrades inertly
(escaped) — it never consumes surrounding content. But note rows 2 and 3: those
DO render, through the element path, which is why an override that reads
props.block must guard for its absence (see the two prop
contracts).
Link-bridge alternative. Before
inlineComponentTags, the way to get an inline custom element was the link bridge: emit[$AAPL](tik://AAPL)and overrideato render a chip when the href scheme matches. It's XSS-safe and renders inline-in-cells too —inlineComponentTagssimply replaces that workaround with first-class inline elements.
Safe raw HTML
LLMs emit a little raw HTML — <br>, <sub>/<sup>, <mark>, and HTML comments
as markers (<!--mk:id-->). unsafeHtml is all-or-nothing; instead opt into a
sanitizer that renders a safe subset natively. Setting htmlAllowlist and/or
dropHtmlTags (even to []) engages it:
// Render only these inline tags; escape everything else:
new BrookClient({ config: { htmlAllowlist: ["br", "sub", "sup", "mark"] } });
// Or allow everything except a built-in dangerous set:
new BrookClient({ config: { htmlAllowlist: [] } });- HTML comments are dropped — no more
<!--mk:id-->surfacing as escaped text — in every mode except bareunsafeHtmlpass-through. htmlAllowlist: ["br", …]renders only those inline tags; everything else is escaped.htmlAllowlist: [](empty) allows all tags except a built-in dangerous set (script,style,iframe,object,embed,form,svg,xmp,plaintext, … — non-overridable: allowlisting one still drops it).dropHtmlTags: ["mk", …]removes those tags entirely (markup gone; inner text stays as inert text).- Every rendered tag's attributes are sanitized:
on*handlers andstyle(a CSS beacon / clickjacking vector) are dropped, and dangerous URL schemes (javascript:, …, including multi-encoded) become#— inhref,src,srcset,poster,cite,action,data,longdescandbackground. - A further set of DOM-hazard attributes is dropped outright (case-insensitive)
— they neither execute nor carry a URL, but each lets authored markup reach
past the text it should be:
srcdoc(inline document injection),is(customized-built-in upgrade),autofocus/contenteditable(focus-steal, UI spoof),id/name(DOM clobbering — an element shadowingdocument.getElementByIdor a global),slot/part/exportparts(shadow-DOM injection),form,formaction,formenctype,formmethod,formnovalidate,formtarget(form hijack),xmlns/xlink:*(namespace escape hatch), andping(tracking beacon). Some are inert today only because the tag that gives them meaning is already in the dangerous set — they are dropped anyway so the policy never depends on that coincidence.class,title,alt,target,rel,data-*andaria-*are unaffected. - This applies to raw HTML only. Component tags stay
permissive: their attributes become framework props on
components[tag], so<Tab id="x">keepsid— the consumer's component decides whether it ever reaches the DOM.on*,styleand dangerous URL schemes are filtered there too. - Scope: inline raw HTML by default. Block-level raw HTML stays escaped
unless you also set
blockHtml(below). Tag matching is case-insensitive.
Block-level raw HTML (blockHtml)
A model that emits a disclosure widget on its own lines —
<details>
<summary>Sources</summary>
Three filings and a transcript.
</details>— produces an HTML block, not inline HTML, so the sanitizer above leaves it
escaped. Opt in with blockHtml and it renders as real elements:
new BrookClient({ config: { htmlAllowlist: [], blockHtml: true } });
// or restrict it:
new BrookClient({ config: { htmlAllowlist: ["details", "summary"], blockHtml: true } });- Only meaningful with the sanitizer engaged.
blockHtmlon its own does nothing; it extendshtmlAllowlist/dropHtmlTagsto block level. Defaultfalse, so existing sanitizer users keep escaped block HTML until they opt in. - Same policy, no exceptions. Tags go through the same allow / drop /
non-overridable-dangerous decision and the same hardened attribute policy as
inline raw HTML — a block-level
<div onclick=… id=… srcdoc=…>renders as a bare<div>. - Scope: CommonMark HTML block types 6 and 7 — a known block-level tag
(
<details>,<div>,<table>,<section>, …) or any other complete tag alone on its line. Types 1–5 stay escaped/dropped, as with the flag off: type 1 is the raw-text family (<script>,<pre>,<style>,<textarea>), where a browser reads everything after the opening tag as unparsed text — so a speculative close mid-stream is an mXSS vector — and types 2–5 (comments, processing instructions, CDATA, declarations) carry no renderable element at all. A block<script>is escaped in every configuration, including withscriptexplicitly allowlisted andunsafeHtmlalso on. - Streaming: speculative closers. While the block is still arriving, every
still-open element gets a closer appended, so the HTML you have received so
far is always a complete tree —
<div>\n<b>bolrenders as<div><b>bol</b></div>, and the closers simply stop being speculative when the author's own</b></div>lands (the emitted bytes don't change). A half-arrived tag (<spa,<a href="htt) renders as nothing until it completes, the same pending-invisible contract as a streaming markdown link's URL; if the stream ends on one, it settles as escaped text. Mis-nesting is repaired rather than propagated:<b><i></b>emits<b><i></i></b>, and a close tag matching nothing open is dropped. A type-6/7 block ends at a blank line even with tags open — the closers land there. - Markdown inside the HTML is not parsed (the body is text + tags). That —
full
rehype-rawsemantics — is a later stage.
Types
interface Block {
id: number;
kind: { type: "Paragraph" | "Heading" | "CodeBlock" | "List" | ...; data?: unknown };
html: string; // safe to inject via dangerouslySetInnerHTML
open: boolean; // still being built (last block in active tail)
speculative: boolean; // closed by inference, may be revised
start: number;
end: number;
}
// Override map for <BrookMarkdown components={...} />
type Components = Record<string, React.ComponentType<any> | string>;
// Props a block-kind override receives (e.g. components.CodeBlock)
interface BlockComponentProps {
block: Block;
html: string;
open: boolean;
speculative: boolean;
text?: string; // decoded source — CodeBlock / MathBlock
language?: string; // info string, first word — CodeBlock
meta?: string; // info string, the rest (`title="src/main.ts"`) — CodeBlock
}htmlToReact(html, components) and parseTrustedHtml(html) are also exported
for advanced use (e.g. rendering a single block's HTML to a React tree yourself).
highlight(code, lang)
Optional. Tiny native-RegExp tokenizer covering js/ts/tsx/jsx, rust, python, go, bash, sql, json, html, css. Unknown languages fall through to plain escaped text.
import { highlight } from "brookmd/highlight";
const html = highlight("const x = 1;", "ts");Streaming syntax highlighting
A code fence is highlighted while it streams, not only once it closes. On by
default; turn it off with streamingHighlight={false} (React) or
{ streamingHighlight: false } (the DOM mount options / Vue / Svelte / Solid),
which restores the plain-until-close behaviour.
It stays O(n) over the whole block rather than re-highlighting the fence on every chunk. An open block keeps a frozen prefix — markup for everything behind a checkpoint, which later bytes provably cannot rewrite — and re-tokenizes only the tail after it, about one source line's worth per patch. When the fence closes, the tokenizer resumes from that checkpoint instead of starting over, so a block that streamed in highlights near-instantly.
Two things worth knowing:
- The tail is speculative. A prefix of source does not tokenize like the same
prefix of a longer source:
const s = "hellois a stray quote plus an identifier until its closing quote lands, and123.456eis123plus loose fragments until a digit arrives. So the last line's colours can shift as bytes come in. Nothing behind the checkpoint ever changes. - The settled markup is byte-identical to
highlight(text, lang)either way. Turning this on or off changes when colour appears, never what it is.
Blocks past the highlighter's 50 000-character guard, unknown languages, and any
fence taken over by a components.CodeBlock / pre / code override are
unaffected — they behave exactly as before.
Coverage
CommonMark 0.31: 100% (652/652 spec examples), byte-exact — every section,
including the hard ones (nested/loose lists, link reference definitions, link
precedence, lazy blockquote continuation). Plus GFM extensions, also 24/24
byte-exact: tables, strikethrough, task lists, extended autolinks, GitHub
alerts (> [!NOTE] → styled callouts), footnotes ([^1] + [^1]:), and math
($…$, $$…$$, \(…\), \[…\]). Autolinks and alerts are on by default;
footnotes and math are opt-in per stream (see
Per-stream config).
Byte-exact means the full output string matches the reference renderer's
byte-for-byte — not a structurally normalized or whitespace-forgiving compare.
Both counts are the harnesses' default floors (CMARK_MIN_EXACT=652,
GFM_MIN_EXACT=24, pinned explicitly in the CI workflows), so a single byte of
regression fails the build even when the normalized tally stays green.
The only deviations from the reference output are deliberate brookmd choices,
folded by a documented canonicalize step applied to both sides before
comparison — the only transform on the byte-exact path, so it can erase our
intentional extras but never hide a structural divergence:
| Deliberate difference | Reference emits |
|---|---|
| target="_blank" rel="noopener noreferrer nofollow" on links (security-only) | no such attrs |
| data-lang="…" on code blocks (alongside class="language-…") | class only |
| HTML5 void elements (<br>) | XHTML self-closing (<br />) |
| style="text-align:…" on table cells | GFM's deprecated align="…" |
See crates/brookmd-core/tests/{cmark_spec,gfm_spec,footnotes,math}.rs for the
runners, the canonicalize source, and the floors.
GitHub alerts render to GitHub-compatible markup
(<div class="markdown-alert markdown-alert-note">…), so existing markdown CSS
styles them, and they're overridable as a block kind via components.Alert.
What it doesn't do
By design, not yet, or only partially:
- Raw HTML in markdown — escaped by default, not passed through. (Security
default. The
unsafeHtml: trueconfig flag disables the escape but must never be enabled for untrusted input without asanitizehook.) - Forward link references when streaming — a
[ref]used before its later[ref]: urldefinition can't resolve until the definition arrives; one-shot parsing handles it fully, streaming converges once the definition streams in. - Definition lists — out of scope for v1.
- KaTeX / Mermaid rendering — brookmd emits KaTeX-ready math markup
(
<span>/<div class="math …">withgfmMathon) and aMermaidslot, but stays zero-dep: bring your own KaTeX / mermaid pass (or acomponents.MathBlock/components.Mermaidoverride) for the actual SVG/MathML output.
Performance
Every realistic streaming shape (long paragraph, fenced code block, GFM table,
blockquote/alert, flat list, math fence, reference-heavy document) parses in
O(n) total work, not O(n²) — at every chunk size from 16 bytes (char-by-char)
up. Each shape has an incremental cache that mirrors the structure of the block
so that an append only does work proportional to the newly arrived bytes, not
the growing tail. See CHANGELOG.md for per-shape numbers and
the regression that prompted each cache; the canonical bench is
crates/brookmd-core/examples/bench.rs (cargo run --release --example bench).
Headline numbers are not durable across machines, but the curve is: chunk size shouldn't change the order of magnitude for any shape. If you hit one that does, file an issue with the input and chunking — that's the next bench scenario.
Wire delta mode (automatic)
Parse work was already O(n), but until 0.23.0 the bytes crossing the
worker→main-thread boundary were not: every append re-emitted the open
block's full HTML, O(n²/chunk) total for one block that grows across many
chunks (a long streaming list, a big code fence). Since 0.23.0 the parser
emits verified splices instead — {keep, append} deltas against the
block's previous emit, established by byte comparison so reconstruction is
byte-exact by construction — and the client reassembles full blocks before
anything else sees them. Zero API change; Block.html is always complete.
Measured at a 200 KB document in 256-byte chunks, a streaming list's total
patch traffic drops from 119.6 MB to 0.78 MB (153× less, 2.8× faster
end-to-end); an unclosed code fence from 80.1 MB to 0.58 MB (137×, 4.9×
faster). Fast-committing prose is unchanged. Emitted bytes are now gated
linear in CI alongside the parse-work counters. Raw-boundary consumers (the
WASM BrookParser, native bindings, C ABI) keep byte-identical v1 wire by
default and can opt in with setWireDelta(true) — see
WIRE.md §11.
Instant thread reopen — persist and hydrate
Reopening a long thread normally means re-feeding its whole source through the parser before the first paint: O(history) work between the click and the pixels, which is exactly when a chat UI feels frozen.
It is also unnecessary. A committed block is emitted once and is final, so the blocks a stream already produced are a complete description of the document — there is nothing to re-derive. Capture them, store them, restore them:
// When the thread closes (or at any checkpoint):
await db.put(threadId,