@aiedx/fetchlens-cloudflare
v0.1.2
Published
AI bot detection middleware for Cloudflare Workers & Pages — detect 26+ AI crawlers at the edge
Downloads
328
Maintainers
adssidhu86Readme
@aiedx/fetchlens-cloudflare
AI bot detection middleware for Cloudflare Workers & Pages. Detects 26+ AI agents (GPTBot, ClaudeBot, PerplexityBot, and more) hitting your site in real-time at the edge.
Why
Google Analytics, Plausible, and every other analytics tool are 100% blind to AI bot traffic. These bots don't run JavaScript, so they never appear in your dashboards. FetchLens detects them at the edge before they even reach your origin.
Install
npm install @aiedx/fetchlens-cloudflareSetup — Cloudflare Pages
Add functions/_middleware.ts to your project:
import { createPagesMiddleware } from '@aiedx/fetchlens-cloudflare'
export const onRequest = createPagesMiddleware({
siteId: 'yourdomain.com',
apiEndpoint: 'https://fetchlens.ai',
siteTag: 'fl_pub_your_key',
})That's it. Every request to your Pages site now runs through FetchLens.
Setup — Cloudflare Workers
Use withFetchLens as your Worker's default export:
import { withFetchLens } from '@aiedx/fetchlens-cloudflare'
export default withFetchLens({
siteId: 'yourdomain.com',
apiEndpoint: 'https://fetchlens.ai',
siteTag: 'fl_pub_your_key',
})Get your Site Tag by scanning your domain at fetchlens.ai.
What it detects
| Provider | Bots | |----------|------| | OpenAI | GPTBot, ChatGPT-User, OAI-SearchBot | | Anthropic | ClaudeBot, Claude-SearchBot, Claude-User, Claude-Code, anthropic-ai | | Perplexity | PerplexityBot, Perplexity-User | | Google | Google-Extended, Google-CloudVertexBot | | Apple | Applebot-Extended | | Meta | meta-externalagent, meta-externalfetcher, FacebookBot | | Others | DuckAssistBot, MistralAI-User, Bytespider, cohere-ai, Amazonbot, PetalBot, Diffbot, PhindBot, YouBot, CCBot |
Features
- Zero-latency — detection runs asynchronously via
waitUntil, no impact on page load - Vulnerability scan blocking — automatically blocks probes for
.php,/wp-admin,.env, etc. with a 403 (configurable) - Edge-native — runs on Cloudflare's global network, 300+ PoPs
- Works with any origin — static sites, GitHub Pages, R2, or any backend
- 26+ AI bot signatures — covers all major AI crawlers and training bots
Configuration
createPagesMiddleware({
siteId: 'yourdomain.com', // Required: your domain
apiEndpoint: 'https://fetchlens.ai', // Required: FetchLens API URL
siteTag: 'fl_pub_xxx', // Your Site Tag (from install guide)
blockVulnScans: true, // Block vulnerability probes (default: true)
detectReferrals: true, // Detect AI referrals via Referer/UTM (default: true)
})How it works
- Every request passes through Cloudflare's edge
- FetchLens extracts the user agent, headers, geo, and request metadata
- The payload is sent asynchronously to the FetchLens API (zero latency impact via
waitUntil) - The API identifies the bot using 26+ known signatures + heuristic detection
- Results appear on your FetchLens dashboard
Also add the client snippet
For AI referral detection (when users click links from ChatGPT, Claude, etc.), add the JS snippet to your HTML <head>:
<script src="https://fetchlens.ai/fetchlens.js" data-key="fl_pub_your_key" defer></script>The middleware catches bots (server-side), the snippet catches referrals (client-side). Together they give you the full picture.
Requirements
- Cloudflare Workers or Pages project
- Node.js 18+
License
Proprietary. Copyright (c) 2026 AIEDX Private Limited.