next-markdown-mirror

Self-hosted Markdown for AI agents — serve clean Markdown instead of HTML when AI agents request your pages. Full docs

The Problem

AI agents waste tokens parsing your nav bars, footers, cookie banners, and ad scripts as "content." HTML boilerplate can 2-5x the token count vs clean Markdown — and AI tools citing your site produce lower-quality responses because of the noise.

Why not Cloudflare?

Cloudflare offers automatic Markdown conversion — but it requires their Pro plan at $20/month per domain ($240/year). For 5 domains, that's $1,200/year.

next-markdown-mirror is free and open source:

| | next-markdown-mirror | Cloudflare Pro | |-|---------------------|----------------| | 1 domain | $0 | $240/year | | 5 domains | $0 | $1,200/year | | 10 domains | $0 | $2,400/year |

Plus: self-hosted (deploy anywhere), full control over filtering and frontmatter, built-in JSON-LD extraction and llms.txt support.

Quick Start

Install:

pnpm add next-markdown-mirror
# or: yarn add next-markdown-mirror
# or: npm install next-markdown-mirror
# or: bun add next-markdown-mirror

App Router setup (Next.js 16+, 3 files)

1. Proxy — intercepts markdown requests and rewrites to the handler:

// proxy.ts
import { withMarkdownMirror } from 'next-markdown-mirror/nextjs';
export const proxy = withMarkdownMirror();

2. Route handler — fetches your HTML internally and converts to Markdown:

// app/md-mirror/[...path]/route.ts
import { createMarkdownHandler } from 'next-markdown-mirror/nextjs';

export const GET = createMarkdownHandler({
  baseUrl: process.env.NEXT_PUBLIC_SITE_URL!,
});

3. llms.txt — AI discovery file:

// app/llms.txt/route.ts
import { createLlmsTxtHandler } from 'next-markdown-mirror/nextjs';

export const GET = createLlmsTxtHandler({
  siteName: 'My Site',
  baseUrl: process.env.NEXT_PUBLIC_SITE_URL!,
  pages: [
    { url: '/', title: 'Home', description: 'Welcome page' },
    { url: '/about', title: 'About' },
  ],
});

Pages Router setup (3 files)

1. Middleware — intercepts markdown requests and rewrites to the API route:

// middleware.ts
import { createMarkdownMiddleware } from 'next-markdown-mirror/pages';
export default createMarkdownMiddleware();
export const config = { matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'] };

2. API route handler — fetches your HTML internally and converts to Markdown:

// pages/api/md-mirror.ts
import { createPagesMarkdownHandler } from 'next-markdown-mirror/pages';

export default createPagesMarkdownHandler({
  baseUrl: process.env.NEXT_PUBLIC_SITE_URL!,
});

3. llms.txt — AI discovery file:

// pages/api/llms.txt.ts
import { createPagesLlmsTxtHandler } from 'next-markdown-mirror/pages';

export default createPagesLlmsTxtHandler({
  siteName: 'My Site',
  baseUrl: process.env.NEXT_PUBLIC_SITE_URL!,
  pages: [
    { url: '/', title: 'Home', description: 'Welcome page' },
    { url: '/about', title: 'About' },
  ],
});

How it works

                    ┌──────────────────────────────┐
                    │        Your Next.js App       │
                    │                               │
  Accept: text/md   │  ┌─────────┐                  │
  ─────────────────►│  │  proxy   │  rewrite         │
        or ?v=md    │  │  .ts     │──────────┐       │
                    │  └─────────┘           │       │
                    │                        ▼       │
                    │              ┌──────────────┐  │
                    │              │ /md-mirror/  │  │
                    │              │ [...path]    │  │  text/markdown
                    │              │  route.ts    │──┼──────────────►
                    │              └──────┬───────┘  │  + YAML frontmatter
                    │                     │          │  + token count
                    │           fetch     │          │
                    │          (internal) │          │
                    │                     ▼          │
                    │              ┌──────────────┐  │
                    │              │  Your HTML   │  │
                    │              │    page      │  │
                    │              └──────────────┘  │
                    └──────────────────────────────┘

Features

• JSON-LD → YAML frontmatter — structured data extracted and prepended automatically
• llms.txt protocol — built-in AI discovery file generation
• GFM + extended Markdown — tables, task lists, definition lists, <details>, <mark>, and more
• Token counting — x-markdown-tokens response header with custom counter support
• Intelligent content filtering — strips nav, footer, scripts, ads, cookie banners, and icons
• Content-Signal header — tell AI agents how they may use your content

Core API

The converter works standalone without Next.js:

import { HtmlToMarkdown } from 'next-markdown-mirror';

const converter = new HtmlToMarkdown({
  baseUrl: 'https://example.com',
  extractJsonLd: true,
  contentSignal: 'ai-input',
});

const result = converter.convert(html);
// result.markdown    — converted Markdown with YAML frontmatter
// result.tokenCount  — estimated token count
// result.jsonLd      — extracted JSON-LD data
// result.title       — page title

Configuration

See the full configuration reference on the docs site.

Key options

| Option | Type | Default | Description | |--------|------|---------|-------------| | contentSelectors | string[] | ['main', 'article', '[role="main"]'] | CSS selectors for main content | | excludeSelectors | string[] | [] | Additional CSS selectors to exclude | | extractJsonLd | boolean | true | Extract JSON-LD as YAML frontmatter | | baseUrl | string | — | Base URL for resolving relative URLs | | contentSignal | ContentSignal | — | Content-Signal header value | | routePrefix | string | '/md-mirror' (App Router) / '/api/md-mirror' (Pages Router) | Internal route prefix |

Used by

• 108 Real Estate
• Kontra Digital
• Orbi
• Yolk Studio

Contributing

git clone https://github.com/jakubkontra/next-markdown-mirror.git
cd next-markdown-mirror
npm install
npm test          # run tests
npm run typecheck # type-check
npm run lint      # lint
npm run build     # build
cd test-app && npm install && npm run dev  # run test app on :3099

Built with

Built with the help of Claude Code.

License

MIT