nanocrawl-sdk

Monetize your content for AI. One file. Five minutes.

AI crawlers pay per page in USDC. Humans browse free. You earn revenue every time an agent reads your content.

npm install nanocrawl-sdk

Quick Start

1. Create one file in your Next.js project root:

// proxy.ts  (Next.js 16+)  or  middleware.ts  (Next.js 14-15)
import { withNanocrawl } from 'nanocrawl-sdk'

export const proxy = withNanocrawl({
  // Required
  sellerWallet: process.env.SELLER_WALLET,
  redisUrl: process.env.REDIS_URL,

  // Pricing
  pricing: { '/blog': 0.001, '/products': 0.002 },
  defaultPrice: 0.001,
  freeRoutes: ['/', '/about'],

  // Volume discounts — reward bots that crawl more
  volumeTiers: [
    { after: 10, discount: 0.20 },   // 20% off after 10 pages
    { after: 50, discount: 0.40 },   // 40% off after 50 pages
  ],
})

export const config = { matcher: ['/((?!_next|favicon).*)'] }

2. Set your env vars:

# .env.local
SELLER_WALLET=0xYourWalletAddress
REDIS_URL=rediss://...   # from console.upstash.com (free tier available)

3. Deploy. That's it. AI crawlers pay you per page. Humans browse free.

Volume Discounts

The more pages a bot crawls, the cheaper each page gets — encouraging deep, comprehensive crawling of your content.

• Page counts are tracked per wallet address in Redis
• Discounts apply automatically — no action needed from the bot
• The /.well-known/ai-pay manifest advertises your tiers so agents can plan budgets
• Bots that send X-Nanocrawl-Wallet header get their discounted price in the 402 response

Redis (Recommended for Production)

On Vercel, your app runs across multiple function instances. Redis ensures payments and volume tiers are tracked consistently across all of them.

# Get a free Redis URL from console.upstash.com, then add to Vercel:
REDIS_URL=rediss://default:[email protected]:6379

The SDK auto-detects REDIS_URL from your environment. You can also pass it explicitly:

withNanocrawl({
  sellerWallet: process.env.NANOCRAWL_SELLER_WALLET!,
  redisUrl: process.env.REDIS_URL,
})

Without Redis, everything still works but uses in-memory storage (fine for local dev).

Dashboard

Visit /nanocrawl on your deployed site. Connect MetaMask with your seller wallet to see:

• Total revenue and payment count
• Gateway balance (live from Circle)
• Top paying agents
• Revenue by route
• Full payment history
• Withdraw earnings to any supported chain

The dashboard is built into the SDK — no extra pages or API routes needed.

How It Works

Human   → browses your site normally (zero friction)
AI Bot  → gets 402 Payment Required
AI Bot  → pays via Circle Nanopayments (gas-free USDC)
AI Bot  → gets the page content
You     → earn USDC per page view

The SDK runs as Next.js middleware/proxy. It detects AI crawlers by user-agent, headers, and behavioral signals. Humans are never affected.

Payments settle through Circle's Gateway — gas-free USDC micropayments on the Arc network.

All Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | sellerWallet | string | required | Your EVM wallet address | | redisUrl | string | process.env.REDIS_URL | Redis connection URL | | pricing | Record<string, number> | — | Per-route prices in USDC (longest prefix wins) | | defaultPrice | number | 0.001 | Fallback price for unlisted routes | | freeRoutes | string[] | [] | Routes that bypass payment for crawlers | | volumeTiers | VolumeTier[] | [] | Volume discounts (see above) | | store | PaymentStore | — | Custom payment store (overrides Redis) | | gatewayUrl | string | Arc Testnet | Circle Gateway URL |

Testing

# Test with the NanoCrawl MCP server
npx nanocrawl
claude mcp add nanocrawl -- npx nanocrawl
# Then ask Claude: "browse https://your-site.vercel.app/blog/my-post"

The Ecosystem

| Package | Role | |---------|------| | nanocrawl-sdk (npm install nanocrawl-sdk) | You install this. Paywall + dashboard for your Next.js site. | | nanocrawl (npx nanocrawl) | AI agents use this. MCP server that gives them a wallet to pay you. |

Links

• NanoCrawl
• Circle Nanopayments
• x402 Protocol

License

MIT