@pickrate/collector

See which AI agents visit your site. One file, no data pipeline.

The analytics you already run can't see AI agents, because agents don't run JavaScript. This collector runs server-side, where it can see them, classifies each request (which agent, what it read), and sends it to Pickrate.

Two ways in:

• Next.js middleware — one line, for a Next app you deploy.
• Cloudflare Worker — put it in front of any site (including no-code hosts: Webflow, Wix, Shopify, Framer). A Worker sees the response, so it also captures the gaps (agents that asked for a page and got a 404).

Install (Next.js)

npm install @pickrate/collector

// middleware.ts
export { middleware } from "@pickrate/collector/next";
export const config = { matcher: ["/((?!_next|favicon).*)"] };

# .env
PICKRATE_KEY=sk_live_xxx   # your Pickrate secret key

That's it. Agent hits start showing up in your Pickrate dashboard. Human traffic classifies to nothing and pays zero cost.

Install (Express / Node)

Connect-style middleware, so it also works with plain Connect and a bare node:http server.

import express from "express";
import { middleware } from "@pickrate/collector/node";

const app = express();
app.use(middleware);          // as early as possible, before your routes

# .env
PICKRATE_KEY=sk_live_...

This adapter captures response status. It hooks the response's finish event, so it can tell you an agent asked for /llms.txt and got a 404 — which the Next.js middleware structurally can't, because middleware runs before the response exists. Nothing is reported until after the response is flushed, so it never delays a request, and next() is called synchronously on the way in.

Mounted routers are handled: it reads req.originalUrl before Express rewrites req.url to be mount-relative, so a router mounted at /docs still reports /docs/llms.txt. Registering the middleware twice is a no-op rather than a double count.

For Fastify, Koa, or Hono, call the core directly from whatever hook that framework gives you — see Custom instrumentation.

Install (Cloudflare Worker)

Run a Worker in front of your origin — works for any host, no-code included, and captures response status.

// src/index.js
import collector from "@pickrate/collector/worker";
export default collector;

# set the secret, then deploy in front of your zone
npx wrangler secret put PICKRATE_KEY   # your Pickrate sk_ key
npx wrangler deploy

The Worker fetches your origin, returns the response unchanged, and reports the agent hit (with its status) in the background.

What it captures

Every request from a known agent (by user-agent) or to a machine-readable surface (/llms.txt, /openapi.json, /.well-known/*, *.md, sitemap.xml, robots.txt). For each it sends an agent_interaction:

• actor — OpenAI GPTBot, Anthropic ClaudeBot, Perplexity, ...
• actorType — assistant (a person is waiting) vs crawler (background)
• surface — llms_txt, openapi_spec, well_known_endpoint, content, ...
• interaction — llms_txt_requested, page_requested, ...
• object / path — what was read

It never reports app routes, assets, or /api. Human page loads are ignored.

Custom instrumentation

Compose your own middleware, or report events the request path can't see (e.g. an MCP tool_invoked):

import { createCollector } from "@pickrate/collector";
const pickrate = createCollector({ key: process.env.PICKRATE_KEY! });

// From an MCP tool handler:
await pickrate.send({
  type: "agent_interaction",
  actor: "chatgpt",
  actorType: "assistant",
  protocol: "http",
  surface: "remote_mcp_server",
  interaction: "tool_invoked",
  object: "create_automation",
  path: "/mcp",
});

Notes

• Secret key. agent_interaction is server-detected, so the ingest requires a secret (sk_) key. The collector runs server-side, so the key never reaches a browser.
• Fail-safe. Every send is fire-and-forget and timeout-bounded. A logging failure never surfaces to a visitor.
• A misconfigured collector is loud, not silent. On any non-2xx from the ingest — a 401/403 (wrong/stale key) or a 422 (a publishable pk_ key whose events get rejected; the collector needs a secret sk_) — the collector logs a one-time warning to your server console. These are the top "installed but recording nothing" causes. Pass onError to route failures somewhere (fire-and-forget, never affects the response):

  createCollector({ key: process.env.PICKRATE_KEY, onError: (e) => reportToSentry(e) });

• Runtime. The core (@pickrate/collector) is framework-agnostic (Edge, Node, Cloudflare). The Next adapter uses waitUntil when available; the Node adapter reports on the response's finish event. Nothing blocks the response either way.
• Response status. The Node, Worker, and log-drain paths see it; Next middleware cannot, because it runs before the response exists. Status is what powers the "agents asked and got a 404" view.
• Gaps need the response. The Next middleware runs before the response, so it can't know a request 404'd — it reports the hit without a status. The Cloudflare Worker (or your own code, via report({ ..., status })) sees the response and captures status, which is what populates the "agents asked, you didn't serve" view. Log imports carry status too.

Development

npm install
npm test        # node:test, no network
npm run typecheck

The package ships hand-authored ESM at the repo root (index.js, next.js, worker.js, node.js) — no build step, so what you install is exactly what you can read here. The TypeScript in src/ is the same logic and carries the tests.

Change the classifier — or the node adapter — in both. src/parity.test.ts runs fixture sets through the shipped JS and the TypeScript and fails if they disagree, so drift breaks CI rather than shipping quietly.

Releases publish from GitHub Actions via npm trusted publishing — no npm token exists in this repo, and provenance attestations are generated automatically.

License

MIT © PurelySearch LLC