@hunt-labs/bytekit-sdk

Official TypeScript SDK for the ByteKit API — web data infrastructure for AI agents and training pipelines: scraping, raw fetch, screenshots, sitemaps, page monitors, and web search.

Install

npm install @hunt-labs/bytekit-sdk

ESM-only. This package ships a single ESM build ("type": "module") with no separate CommonJS bundle. Use it from an ESM project ("type": "module" in your package.json, or a .mjs entrypoint). From CommonJS you can either load it with a dynamic await import('@hunt-labs/bytekit-sdk'), or — on Node ≥20.19 / ≥22.12, which resolve require() of ESM — call a top-level require('@hunt-labs/bytekit-sdk') directly (the package's exports map carries a default condition for exactly this). On older Node, a top-level require() throws ERR_REQUIRE_ESM; use the dynamic import() there.

Requires Node.js 18+ (the client uses the global fetch).

Authentication

Create an API key at app.bytekit.com (keys are prefixed sk_live_) and pass it to the client constructor:

import { RapidCrawl } from '@hunt-labs/bytekit-sdk';

const client = new RapidCrawl({ apiKey: process.env.BYTEKIT_API_KEY! }); // sk_live_...

The key is sent as Authorization: Bearer <apiKey> on every request. Keep it server-side — it is not safe to ship in a browser bundle.

The client class is named RapidCrawl for historical reasons (ByteKit's previous product name). It is the SDK's only entrypoint class.

Base-URL override

Requests go to https://api.bytekit.com by default. Point the client at staging, a local gateway, or a proxy with the baseUrl option:

const client = new RapidCrawl({
  apiKey: process.env.BYTEKIT_API_KEY!,
  baseUrl: 'https://api-stg.bytekit.com', // default: https://api.bytekit.com
});

Usage

Scrape a page

const result = await client.scrape.create({
  url: 'https://example.com',
  formats: ['markdown', 'html'],
});

Fetch an async scrape by id

const scrape = await client.scrape.get('sc_123abc');

Raw fetch (unprocessed response body)

const raw = await client.fetch.create({ url: 'https://example.com' });

Web search

const results = await client.search.query({ query: 'ai web scraping', limit: 10 });
for (const hit of results.results) {
  console.log(hit.position, hit.title, hit.url);
}

Screenshot a page

const shot = await client.screenshots.create({
  url: 'https://example.com',
  full_page: true,
});
const ready = await client.screenshots.get('ss_123abc');

Bulk jobs

scrape and fetch each expose a bulk sub-resource that takes a URL list and a webhook to notify on completion:

const job = await client.scrape.bulk.create({
  urls: ['https://example.com', 'https://example.org'],
  webhook_url: 'https://your.app/hooks/bytekit',
});
const status = await client.scrape.bulk.get('bulk_123abc');

client.fetch.bulk.create() / client.fetch.bulk.get() follow the same shape.

Bulk screenshots are not a screenshots.bulk sub-resource — they go through the separate top-level client.bulk resource instead:

const shots = await client.bulk.create({
  urls: ['https://example.com', 'https://example.org'],
  webhook_url: 'https://your.app/hooks/bytekit',
});
const shotStatus = await client.bulk.get('bulk_123abc');
await client.bulk.screenshots.list('bulk_123abc');

Page monitors

const monitor = await client.monitors.create({
  url: 'https://example.com/pricing',
  interval_type: 'daily',
  webhook_url: 'https://your.app/hooks/bytekit',
});
await client.monitors.list();
await client.monitors.delete('mon_123abc');

Sitemap crawl

const crawl = await client.sitemap.create({ url: 'https://example.com' });
const sitemap = await client.sitemap.get('sm_123abc');

Account, usage, and API keys

const account = await client.account.get();
const keys = await client.account.apiKeys.list();

Error handling

Any 4xx/5xx response throws a RapidCrawlError carrying the HTTP status and the API's error code:

import { RapidCrawl, RapidCrawlError } from '@hunt-labs/bytekit-sdk';

try {
  await client.scrape.create({ url: 'https://example.com' });
} catch (err) {
  if (err instanceof RapidCrawlError) {
    console.error(err.status, err.code, err.message);
  }
}

Documentation

Full API reference: bytekit.com/docs.

License

MIT — see LICENSE.