@vercel/agent-readability

Detect AI agents. Serve them markdown. Audit your site against the Agent Readability Spec.

Install

npm install @vercel/agent-readability

Or audit without installing:

npx @vercel/agent-readability audit https://vercel.com/docs

Which adapter?

| You're deploying | Use | |---|---| | Next.js (any host) | ./next | | SvelteKit SSR (any host) | ./sveltekit | | Nuxt SSR (any host) | ./nuxt | | Any framework with prerendered/static pages on Vercel | ./vercel | | Astro / plain static site on Vercel | ./vercel | | Other frameworks | core API (shouldServeMarkdown) |

Framework server middleware (./sveltekit, ./nuxt) doesn't run for prerendered pages — use ./vercel when any page is statically generated. Next.js middleware runs for all page types on Vercel, so ./next works universally.

Quick Start

Next.js middleware.ts:

import { withAgentReadability } from '@vercel/agent-readability/next'

export default withAgentReadability({
  rewrite: (pathname) => `/api/docs-md${pathname}`,
})

export const config = { matcher: ['/docs/:path*'] }

SvelteKit hooks.server.ts:

import { handleAgentReadability } from '@vercel/agent-readability/sveltekit'
import { sequence } from '@sveltejs/kit/hooks'

export const handle = sequence(
  handleAgentReadability({ rewrite: (p) => `/api/docs-md${p}` }),
)

Nuxt server/middleware/agent.ts:

import { defineAgentMiddleware } from '@vercel/agent-readability/nuxt'

export default defineAgentMiddleware({
  getMarkdown: async (pathname) => {
    const doc = await fetchDoc(pathname)
    return doc.markdown
  },
})

Vercel (any framework) middleware.ts:

import { createAgentMiddleware } from '@vercel/agent-readability/vercel'

export default createAgentMiddleware({
  rewrite: (pathname) => `/agent-md${pathname}`,
})

export const config = {
  matcher: '/((?!api|_next|_nuxt|favicon|.*\\..*).*)',
}

Agents hitting /docs/* get Markdown when their Accept header prefers it or does not distinguish between HTML and Markdown. An explicit HTML preference is always respected.

For the Next.js and Vercel rewrite adapters, apply the canonical Link header in the destination route after it resolves a page. See Canonical URLs.

How Detection Works

Three layers, checked in order:

1. Known UA patterns. 30+ agents (ClaudeBot, GPTBot, Cursor, Perplexity, etc.)
2. Signature-Agent header. ChatGPT agent via RFC 9421.
3. sec-fetch-mode heuristic. Unknown bots lacking browser fingerprints.

Optimizes for recall over precision. Serving markdown to a non-AI bot is cheap. Missing an AI agent is not.

Core API

isAIAgent(request)

import { isAIAgent } from '@vercel/agent-readability'

const result = isAIAgent(request)
// { detected: true, method: 'ua-match' }

Accepts any object with headers.get() (Request, NextRequest, etc.)

Returns:

• { detected: true, method: 'ua-match' | 'signature-agent' | 'heuristic' }
• { detected: false, method: null }

acceptsMarkdown(request)

Returns true when the request prefers an offered Markdown representation over HTML. It honors quality values, wildcards, explicit q=0 rejections, specificity, and header order. When you provide custom mediaTypes, the first entry is the response content type and later entries are compatible request aliases.

import { acceptsMarkdown } from '@vercel/agent-readability'

if (acceptsMarkdown(request)) {
  return new Response(markdown, {
    headers: { 'Content-Type': 'text/markdown', 'Vary': 'Accept' },
  })
}

shouldServeMarkdown(request)

Combines detection and content negotiation. Explicit Accept preferences take priority. Agent detection is used only when the header is missing or does not distinguish between HTML and Markdown.

import { shouldServeMarkdown } from '@vercel/agent-readability'

const { serve, reason } = shouldServeMarkdown(request)
// serve: true, reason: 'agent' | 'accept-header'

createNotFoundResponse(path, options?)

Returns an agent-friendly markdown response with a real 404 status, so agents do not treat the missing URL as a page. Set status: 410 for removed content.

import { createNotFoundResponse } from '@vercel/agent-readability'

return createNotFoundResponse('/docs/missing', {
  baseUrl: 'https://example.com',
})

The response includes Content-Type: text/markdown; charset=utf-8, Vary: Accept, and X-Robots-Tag: noindex. Pass markdown and headers to preserve a custom smart not-found body, suggestions, and cache policy while the package owns the status.

generateNotFoundMarkdown(path, options?)

Generates the standard markdown body used by createNotFoundResponse(). Use it when another response abstraction must construct the final 404 or 410.

Keep suggested page links canonical (/docs/page) and negotiate markdown with Accept: text/markdown rather than exposing .md page URLs in not-found responses.

applyMarkdownHeaders(headers, options?)

Apply the response headers every markdown response should carry: Vary: Accept (token-deduped) and, when options.canonicalUrl is set, Link: <url>; rel="canonical" (skipped if a canonical Link is already present). Use in custom proxies that serve markdown without an adapter. canonicalLinkHeader(url) builds the raw Link value.

import { applyMarkdownHeaders } from '@vercel/agent-readability'

const headers = new Headers({ 'Content-Type': 'text/markdown' })
applyMarkdownHeaders(headers, { canonicalUrl: 'https://example.com/docs/page' })
return new Response(markdown, { headers })

Pattern Exports

AI_AGENT_UA_PATTERNS, TRADITIONAL_BOT_PATTERNS, SIGNATURE_AGENT_DOMAINS, and BOT_LIKE_REGEX are all exported.

Next.js Adapter

withAgentReadability(options, handler?)

Works with Next.js 14 and 15 (Pages and App Router).

import { withAgentReadability } from '@vercel/agent-readability/next'

export default withAgentReadability({
  docsPrefix: '/docs',
  rewrite: (pathname) => `/en/llms.mdx/${pathname.replace('/docs/', '')}`,
  onDetection: async ({ path, method }) => {
    await trackMdRequest({ path, detectionMethod: method })
  },
})

onDetection runs via event.waitUntil() and does not block the response.

The adapter adds Vary: Accept to rewrites but leaves the canonical Link header to the Markdown route by default. Apply the header after the route resolves a page, and omit it from 404 and 410 responses. Set canonicalUrl only when every rewritten path is guaranteed to exist.

Composing with existing middleware

export default withAgentReadability(
  { rewrite: (p) => `/md${p}` },
  (req, event) => i18nMiddleware(req, event),
)

Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | docsPrefix | string | '/docs' | URL prefix to intercept | | rewrite | (pathname: string) => string | required | Maps request path to markdown route | | onDetection | (info) => void \| Promise<void> | - | Analytics callback (runs in waitUntil) | | canonicalUrl | (pathname, request) => string \| URL \| null | - | Opt-in canonical URL for rewrites whose paths are guaranteed to exist |

agentReadabilityMatcher

Excludes Next.js internals and static files. Use for site-wide detection:

import { withAgentReadability, agentReadabilityMatcher } from '@vercel/agent-readability/next'

export default withAgentReadability({
  docsPrefix: '/',
  rewrite: (pathname) => `/md${pathname}`,
})

export const config = {
  matcher: agentReadabilityMatcher,
}

SvelteKit Adapter

handleAgentReadability(options)

Returns a Handle function. Requires SvelteKit 2+. Uses event.fetch() for zero-cost internal routing to your +server.ts markdown routes.

// hooks.server.ts
import { handleAgentReadability } from '@vercel/agent-readability/sveltekit'
import { sequence } from '@sveltejs/kit/hooks'

export const handle = sequence(
  handleAgentReadability({
    docsPrefix: '/docs',
    rewrite: (pathname) => `/api/docs-md${pathname}`,
    onDetection: ({ path, method }) => {
      console.log(`Agent detected: ${method} on ${path}`)
    },
  }),
)

Automatically guards against infinite loops (isSubRequest), skips client navigation requests (isDataRequest), and falls through if the rewrite target returns non-OK unless it is a markdown 404 or 410.

Doesn't run for prerendered pages (export const prerender = true). If any route is prerendered on Vercel, use the Vercel adapter.

Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | docsPrefix | string | '/docs' | URL prefix to intercept | | rewrite | (pathname: string) => string | required | Maps request path to +server.ts route | | onDetection | (info) => void \| Promise<void> | - | Fire-and-forget analytics callback | | canonicalUrl | (pathname, event) => string \| URL \| null | request origin + pathname | Canonical URL for the Link header; return null to disable |

Nuxt Adapter

defineAgentMiddleware(options)

Wraps defineEventHandler. Requires h3 1.8+ (ships with Nuxt 3/4). Uses a getMarkdown callback instead of rewrite since Nuxt has no zero-cost internal fetch.

Doesn't run for statically generated pages. If you use nuxt generate or { prerender: true } route rules, use the Vercel adapter instead — it runs at the edge before static assets are served.

// server/middleware/agent.ts
import { defineAgentMiddleware } from '@vercel/agent-readability/nuxt'

export default defineAgentMiddleware({
  docsPrefix: '/docs',
  getMarkdown: async (pathname, event) => {
    const doc = await queryContent(pathname).findOne()
    return doc.body
  },
})

getMarkdown can return a string (auto-wrapped with text/markdown headers) or a Response for full control.

Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | docsPrefix | string | '/docs' | URL prefix to intercept | | getMarkdown | (pathname, event) => string \| Response \| Promise<...> | required | Returns markdown content | | onDetection | (info) => void \| Promise<void> | - | Fire-and-forget analytics callback | | canonicalUrl | (pathname, event) => string \| URL \| null | from Host + X-Forwarded-Proto, URL-parser validated | Canonical URL for the Link header; return null to disable |

Vercel Adapter

createAgentMiddleware(options)

Framework-agnostic Vercel Routing Middleware. Runs before the cache, so it intercepts statically generated pages that bypass framework server middleware. Works for any framework on Vercel: Nuxt (static or SSR), SvelteKit, Astro, or plain static sites.

Requires @vercel/functions (peer dep).

// middleware.ts (project root, not in a framework directory)
import { createAgentMiddleware } from '@vercel/agent-readability/vercel'

export default createAgentMiddleware({
  docsPrefix: '/docs',
  rewrite: (pathname) => `/agent-md${pathname}`,
  onDetection: async ({ path, method }) => {
    await trackMdRequest({ path, detectionMethod: method })
  },
})

export const config = {
  matcher: '/((?!api|_next|_nuxt|favicon|.*\\..*).*)',
}

onDetection runs via waitUntil() from @vercel/functions and does not block the response.

The adapter adds Vary: Accept to rewrites but leaves the canonical Link header to the Markdown route by default. Apply the header after the route resolves a page, and omit it from 404 and 410 responses. Set canonicalUrl only when every rewritten path is guaranteed to exist.

Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | docsPrefix | string | '/docs' | URL prefix to intercept | | rewrite | (pathname: string) => string | required | Maps request path to markdown route | | onDetection | (info) => void \| Promise<void> | - | Analytics callback (runs in waitUntil) | | canonicalUrl | (pathname, request) => string \| URL \| null | - | Opt-in canonical URL for rewrites whose paths are guaranteed to exist |

Audit CLI

npx @vercel/agent-readability audit https://sdk.vercel.ai

25 weighted checks across 4 categories. Score 0-100. Failed checks include fix suggestions you can paste into your coding agent.

CI

- name: Audit agent readability
  run: npx @vercel/agent-readability audit ${{ env.SITE_URL }} --min-score 70 --json

Exit code 1 if score is below threshold.

| Flag | Description | |------|-------------| | --json | Output as JSON | | --min-score <n> | Exit with error if score < n |

Caching

Set Vary: Accept on markdown responses so CDNs don't serve cached HTML to agents (or cached markdown to browsers).

All four adapters set Vary: Accept automatically. The Next.js and Vercel adapters rewrite the URL, so your Markdown route handler must set Content-Type: text/markdown and declare the canonical URL after resolving the page.

Canonical URLs

Markdown bodies have no <link rel="canonical"> equivalent. Resolved Markdown responses should declare the canonical page URL with an HTTP Link header (RFC 8288):

Link: <https://example.com/docs/page>; rel="canonical"

The SvelteKit adapter inspects the resolved response and adds the header only for existing pages. The Nuxt adapter adds it when getMarkdown returns a string; its default comes from validated Host and X-Forwarded-Proto headers. Full Nuxt Response values retain their own headers.

The Next.js and Vercel adapters cannot inspect the rewrite destination's final status. They add Vary: Accept but leave the canonical header to the Markdown route by default, preventing missing pages from declaring themselves canonical. Use applyMarkdownHeaders after resolving the page, and call it without canonicalUrl for 404 or 410 responses. Their canonicalUrl options remain available as explicit opt-ins when every matched path is known to exist.

Edge Runtime

Core library and all adapters use Web APIs only. Works in Vercel Edge Runtime, Cloudflare Workers, and any Request/Response environment.

CLI requires Node.js 20+.

License

MIT