ward-protocol

WARD (Web Access Rights Declaration) — an open enforcement and evidence layer for AI crawler preferences.

WARD can physically remove marked content at the origin. Detection and identity verification are separate: a User-Agent match is a claim, not proof of who sent the request.

Install

npm install ward-protocol

Quick Start

1. Mark protected content in HTML

<div data-ward="redact" data-ward-reason="pii">
  <p>Max Mustermann</p>
  <p>Musterstrasse 1, 12345 Berlin</p>
</div>

2. Detect AI bots

import { inspectCrawlerIdentity } from 'ward-protocol'

const identity = inspectCrawlerIdentity(request.headers.get('user-agent') ?? '')
if (identity) {
  console.log(`${identity.bot.name}: ${identity.status}`) // claimed until verified
}

3. Redact HTML

import { redactHtml } from 'ward-protocol'

const result = redactHtml(html, {
  policyUrl: 'https://example.com/.well-known/ward.json',
})

console.log(`Redacted ${result.redactedCount} blocks`)
// result.html — cleaned HTML with PII removed

Optional: verify provider-published IP ranges

import { createPublishedIpVerifier } from 'ward-protocol'

// Only trust a value your proxy overwrites and your origin cannot receive
// directly from the public internet.
const verifyBot = createPublishedIpVerifier({
  getSourceIp: (request) => request.headers.get('cf-connecting-ip'),
})

The verifier supports the official range endpoints listed in the registry for OpenAI, Anthropic, and Perplexity. Missing evidence or an unavailable endpoint leaves the identity claimed; a completed comparison returns verified or mismatch. Raw source IPs are not included in verification evidence.

4. Next.js Middleware

// middleware.ts
import { createWardMiddleware, generateWardPolicy } from 'ward-protocol'

const policy = generateWardPolicy({
  training: false,
  summarization: true,
})

const ward = createWardMiddleware({
  policy,
  verifyBot,
  onBotDetected: (bot, req) => {
    console.log(`${bot.name} accessing ${new URL(req.url).pathname}`)
  },
})

export async function middleware(request: NextRequest) {
  const response = await ward(request)
  if (response) return response
  return NextResponse.next()
}

Next.js App Router: do not rewrite a streamed React Server Component response after rendering. Protected values can also exist in the RSC flight payload. Make the WARD decision in middleware, then omit protected values while rendering the Server Component. getward.org/impressum is the reference pattern.

5. Create ward.json

import { generateWardPolicy } from 'ward-protocol'

const policy = generateWardPolicy({
  publisher: 'My Company',
  training: false,
  summarization: true,
  contact: '[email protected]',
  rules: [
    { path: '/impressum', summarization: false },
  ],
})

// Save to public/.well-known/ward.json

API

Bot Detection

| Function | Description | |----------|-------------| | detectAiBot(userAgent) | Returns BotInfo or null | | isAiBot(userAgent) | Returns boolean | | AI_BOTS | Array of known AI bot entries | | inspectCrawlerIdentity(userAgent, options?) | Returns an identity labelled claimed, verified, or mismatch |

Policy

| Function | Description | |----------|-------------| | parseWardPolicy(json) | Parse and validate ward.json | | getPathRule(policy, path) | Get effective rule for a path | | generateWardPolicy(options) | Generate a ward.json policy | | evaluateWardRequest({ policy, path, identity }) | Resolve the policy to allow, redact, or observe |

Redactor

| Function | Description | |----------|-------------| | redactHtml(html, options?) | Remove data-ward="redact" content |

Options: message, policyUrl, attribute, redactValue

Middleware

| Function | Description | |----------|-------------| | createWardMiddleware(options?) | Create Next.js middleware |

Options: paths, excludePaths, redactMessage, onBotDetected, verifyBot, onDecision, policyPath, policy

What AI Bots Receive

Before (human browser):

<div data-ward="redact" data-ward-reason="pii">
  <p>Max Mustermann, Musterstrasse 1, Berlin</p>
</div>

After (AI bot):

<div data-ward="redact" data-ward-reason="pii">
  <!-- Content redacted per WARD policy | Reason: pii -->
</div>

The page structure stays intact. The bot knows a block exists, but PII is physically absent.

Specification

See SPECIFICATION.md for the full ward.json format specification.

Legal context

• EU DSM Directive Art. 4 permits rightsholders to reserve TDM rights in an appropriate manner, including machine-readable means.
• EU AI Act Art. 53 requires providers of general-purpose AI models to maintain a copyright-compliance policy, including identifying and complying with relevant rights reservations.

WARD is an experimental technical implementation. It is not an officially recognised compliance mechanism and does not prove that a requester used content for training.

See docs/ALPHA_CONTRACT.md for the exact product boundary and acceptance criteria.

License

MIT