@datasaas/bot-tracker

Zero-dependency server-side bot and AI-crawler reporting for DataSaaS.

AI crawlers like GPTBot, ClaudeBot, and PerplexityBot fetch your raw HTML and never run JavaScript, so a browser analytics tag never sees them. This package reports them from your server instead. It runs one fire-and-forget check per request: it never awaits, never throws into your request path, and sends nothing for ordinary human traffic.

This is separate from the DataSaaS tracking snippet. The JS tag tracks humans; this tracks bots. Search crawlers that do render JavaScript (Googlebot, Bingbot, Applebot) are already captured by the JS tag with no server code.

Docs: full setup guide at datasaas.co/docs/bot-traffic.

Install

npm install @datasaas/bot-tracker

Replace ds_abc123 with your website id (Settings → Tracking).

Next.js (App Router middleware)

// middleware.ts
import { NextResponse } from "next/server";
import { trackBotFromRequest } from "@datasaas/bot-tracker";

export function middleware(req: Request) {
  trackBotFromRequest(req, { websiteId: "ds_abc123" }); // fire-and-forget
  return NextResponse.next();
}

// Match pages, not static assets — bots crawl pages, not chunks.
export const config = { matcher: ["/((?!_next/static|_next/image|favicon.ico).*)"] };

Any framework

trackBotRequest takes the fields directly, so it works in Express, Hono, Fastify, Cloudflare Workers, Bun, or Deno.

import { trackBotRequest } from "@datasaas/bot-tracker";

app.use((req, res, next) => {
  trackBotRequest(
    {
      userAgent: req.headers["user-agent"] || "",
      ip: (req.headers["x-forwarded-for"]?.split(",")[0] || req.socket.remoteAddress || "").trim(),
      path: req.path,
      host: req.headers.host,
      method: req.method,
    },
    { websiteId: "ds_abc123" }
  );
  next();
});

API

• trackBotFromRequest(req, opts) — extract fields from a Web Request/NextRequest and report it.
• trackBotRequest(fields, opts) — report from explicit fields.
• looksLikeBot(ua) — the local prefilter, exported for reuse. It is a cheap hint, not authoritative classification; the DataSaaS server is the source of truth.

opts:

• websiteId (required) — your DataSaaS website id.
• endpoint — override the ingest URL. Defaults to https://datasaas.co/api/bot; point it at your own instance if you self-host.
• fetchImpl — inject a fetch for tests or runtimes without a global one. On a runtime with no fetch, the call safely no-ops.

Serverless and edge runtimes

On a long-running server the report finishes on its own. On serverless or edge platforms the invocation is frozen the moment the response returns, which would cancel an in-flight report. Both functions return a promise that settles when the report finishes, so you can keep the runtime alive until it does.

Vercel / Next.js middleware — handled automatically. The package finds the request's waitUntil on the Next request context and registers the report itself, so the plain one-liner works:

export function middleware(req: Request) {
  trackBotFromRequest(req, { websiteId: "ds_abc123" });
  return NextResponse.next();
}

Cloudflare Workers — the execution context is passed to your handler, not global, so hand the returned promise to ctx.waitUntil:

export default {
  async fetch(request, env, ctx) {
    ctx.waitUntil(trackBotFromRequest(request, { websiteId: "ds_abc123" }));
    return fetch(request);
  },
};

How verification works

Your server posts the request's user_agent, ip, path, host, and method. DataSaaS matches the User-Agent to a known crawler and verifies the IP against that operator's published ranges, so a spoofed GPTBot header is recorded as unverified and can be filtered out. Operators without a published IP list (e.g. Meta, most Chinese crawlers) are always recorded as unverified by design.