CrawlerLane

AI crawler intelligence API for founders, builders and websites.

CrawlerLane shows what AI assistants, search engines and crawlers are doing on your website — and turns that bot traffic into product, SEO and AI visibility signals.

CrawlerLane is not spyware. Only analyze logs from websites you own. Bot classification is heuristic unless verified by reverse DNS or trusted infrastructure checks.

Install

npm install @talocode/crawlerlane

CLI

npx crawlerlane classify --user-agent "ChatGPT-User"
npx crawlerlane report --file access.log --domain talocode.site --format markdown

SDK

import { CrawlerLaneClient } from '@talocode/crawlerlane'

const crawlerlane = new CrawlerLaneClient({
  apiKey: process.env.TALOCODE_API_KEY,
  baseUrl: process.env.TALOCODE_BASE_URL,
})

const report = await crawlerlane.report.generate({
  domain: 'talocode.site',
  logs,
})

Talocode Cloud

Hosted API namespace: /v1/crawlerlane/*

export TALOCODE_API_KEY=your_key
export TALOCODE_BASE_URL=https://api.talocode.site

Local open-source usage works without Talocode Cloud when CRAWLERLANE_ALLOW_LOCAL_UNAUTH=true.

Privacy

• IP addresses are redacted/hashed by default
• Do not submit logs from sites you do not own
• Do not use CrawlerLane to bypass robots.txt, CAPTCHAs, or paywalls

License

MIT © Talocode