@shipwake/ai-crawl

Server-side AI crawler tracking for Shipwake. It observes requests handled by your backend, filters static assets and unknown agents locally, and sends a small event to Shipwake without blocking the visitor response.

The package does not replace window.signal.track(...): browser analytics still measures people and SPA navigation, while this package measures server-observed AI and search crawlers.

Install

npm install @shipwake/ai-crawl

Next, Cloudflare or any Fetch-style runtime

import { trackAICrawlerResponse } from '@shipwake/ai-crawl';

export default async function handler(request, event) {
  const response = await next(request);

  trackAICrawlerResponse(request, response, event, {
    siteKey: process.env.SHIPWAKE_SITE_KEY,
    serverToken: process.env.SHIPWAKE_SERVER_TOKEN,
    apiUrl: process.env.SHIPWAKE_AI_CRAWL_URL,
  });

  return response;
}

When the runtime exposes waitUntil, the request is scheduled there. Otherwise it is still fire-and-forget and errors are swallowed by the helper.

Express

import { createExpressAICrawlerMiddleware } from '@shipwake/ai-crawl';

app.use(createExpressAICrawlerMiddleware({
  siteKey: process.env.SHIPWAKE_SITE_KEY,
  serverToken: process.env.SHIPWAKE_SERVER_TOKEN,
  apiUrl: process.env.SHIPWAKE_AI_CRAWL_URL,
}));

The middleware calls next() immediately and observes the final status code on finish.

Laravel and Inertia

For a Laravel app, use the companion shipwake/laravel-ai-crawl package. Register its middleware on the web group; it works for Blade, Inertia and React SPA requests because the observation happens around the Laravel response, not inside React.

siteKey, serverToken and apiUrl are required. The site key identifies the Shipwake site and is the same public key used by the browser tracker. Generate or rotate the private server token from the Shipwake dashboard, then keep it only in server-side environment variables. It is sent as the Bearer credential.