crawbots

crawbots makes one Crawbots Proxy session usable in Node.js, browsers, CLI tools, CI, and system services. It requires Node.js 20+ and a personal API token created in the Crawbots portal.

Install

npm install crawbots

Use directly in Node.js

import crawbots from "crawbots";

await crawbots.proxy.connect(process.env.CRAWBOTS_PROXY_TOKEN);

// Same healthy exit while active. Passing the same id in another process
// reuses the same durable sticky session.
const session = await crawbots.proxy.select("DE", "billing-worker-1");

const response = await session.fetch("https://api.ipify.org?format=json");
console.log(await response.json());

select(country, sessionId?) creates or reuses a country-specific sticky session. random(sessionId?) does the same using automatic routing. If you omit the ID, Crawbots generates and returns one.

const automatic = await crawbots.proxy.random();

// Every newly opened connection receives a fresh eligible route.
const rotating = crawbots.proxy.rotating("US");

disconnect() closes the SDK-local HTTP dispatcher only. It does not revoke a reusable sticky session. Call session.revoke() when the affinity must end.

System-wide and service configuration

Every selected session exposes standard proxy URLs and environment variables. Use them with services, curl, CI, browsers, or any application that supports HTTP CONNECT or SOCKS5 proxy configuration.

const config = session.system();
process.env.HTTP_PROXY = config.env.HTTP_PROXY;
process.env.HTTPS_PROXY = config.env.HTTPS_PROXY;
process.env.ALL_PROXY = config.env.ALL_PROXY;

For shell, CI, launchd, or systemd configuration:

export CRAWBOTS_PROXY_TOKEN="cbp_live_..."
crawbots proxy env --country DE --session billing-worker-1

Add --json for configuration automation. The CLI prints configuration; it never silently changes OS-level proxy settings. This avoids changing every application on a machine without explicit operator approval.

Keep the API token and the emitted proxy URLs in a secret manager. Both grant proxy access.