@steve31415/email-rewrite

Email HTML → Markdown preprocessing + LLM rewrite with URL compaction. Shared by the Plasticine Email and Radar apps.

Plasticine Way: ec0578a (2026-06-21)

What it does

1. preprocessForRewrite(html) — strips <blockquote>, <div.gmail_quote>, and <img>; walks the cheerio DOM to produce Markdown; caps at 50K chars.

2. compactUrls(md) / expandUrls(md, table, logger) — replaces each unique http(s) URL with https://z.com/<index><2-digit-hash> before the LLM call, restores them on output, and logs email_rewrite.url_expand_* at ERROR on hash mismatch or unknown index. Newsletter HTML is dominated by click-tracking URLs (often 400+ chars each); compaction cuts the prompt 70–80%.

3. runRewrite({...}) — the whole pipeline: preprocess → compact → LLM (via @steve31415/llm-failover) → expand. Returns { markdown, tokensUsed, provider, latencyMs, urlCount, disposition } | null. Returns null on empty input, max_tokens truncation, or provider-chain exhaustion; caller falls through to the no-rewrite path. Failures log email_rewrite.failed at ERROR, except an all-providers-declined outcome (every provider safety-refused / content-filtered, surfaced as LLMError.declined), which logs email_rewrite.declined at WARN as an expected external result. Requires @steve31415/llm-failover >=1.7.0.

   Pass optional dispositionOptions: string[] to let the rewrite prompt also assign a disposition: the model is instructed to optionally emit a first line <<<DISPOSITION: X>>> (X ∈ the caller's options) before the body, which is returned as disposition (or null when absent/unrecognized). The package stays domain-agnostic — the caller supplies the vocabulary.

Why URL compaction matters

Long synchronous Anthropic generations 524 through Cloudflare's outbound subrequest cap (~100s) before the app-level timeout can fire. URL compaction attacks the root cause — generation time — by cutting bytes the LLM has to emit. On the SF Chronicle "Bay Briefing" newsletter, this brought Sonnet's rewrite latency from a 524-timeout chain to ~28s direct, with zero loss of link fidelity.

Usage

import { runRewrite } from '@steve31415/email-rewrite';

const result = await runRewrite({
  rawHtml: '<p>email html here</p>',
  userPrompt: 'Rewrite this for clarity, preserve all links.',
  callSite: 'rewrite_production', // or 'rewrite_preview' — used for telemetry
  apiKeys: {
    anthropic: env.ANTHROPIC_API_KEY,
    gemini: env.GEMINI_API_KEY,
    openai: env.OPENAI_API_KEY,
  },
  logger,
  // Optional overrides — defaults: claude-sonnet-4-6 / 16384 / 240_000 ms.
  // For UI preview flows, drop to a fast cheap model:
  // model: 'gemini-3.1-flash-lite',
  // maxTokens: 8192,
});

if (!result) {
  // Truncation / failure — fall through to original content.
} else {
  console.log(result.markdown);
}

Development

npm install
npm test
npm run build

Publishing

Via npm Trusted Publishing (OIDC). Tag v<version> and push; the .github/workflows/publish.yml workflow handles the rest. Never npm publish manually.