Berrycrawl TypeScript SDK

The official TypeScript SDK for turning websites into clean content, structured data, screenshots, search results, site maps, and brand profiles.

Documentation · Dashboard · GitHub

Installation

npm install berrycrawl

Quickstart

Create an API key in the Berrycrawl dashboard, set it as BERRYCRAWL_API_KEY, then:

import { Berrycrawl } from "berrycrawl";

const berrycrawl = new Berrycrawl({ apiKey: process.env.BERRYCRAWL_API_KEY });
const page = await berrycrawl.scrape({ url: "https://example.com" });

Common tasks

Scrape one page

const page = await berrycrawl.scrape({
  url: "https://example.com/pricing",
  formats: ["markdown", "links"],
});

Crawl a website

Crawls run as background jobs. The response contains the job ID you can use with the jobs API.

const job = await berrycrawl.crawl({
  url: "https://example.com/docs",
  limit: 50,
});
console.log(job.id);

Search the web

const results = await berrycrawl.search({
  query: "best headless browser libraries",
  limit: 10,
});

Map a website

const map = await berrycrawl.map({
  url: "https://example.com",
  search: "documentation",
});

Retrieve a brand profile

const brand = await berrycrawl.brand.retrieve({
  url: "https://stripe.com",
});

Extract structured data

const extraction = await berrycrawl.extract({
  urls: ["https://example.com/products"],
  prompt: "Extract every product name, price, and description",
});
console.log(extraction.id);

Error handling

import { BerrycrawlError } from "berrycrawl";

try {
  await berrycrawl.scrape({ url: "https://example.com" });
} catch (error) {
  if (error instanceof BerrycrawlError) {
    console.error(error.statusCode, error.body);
  }
  throw error;
}

Configuration

const berrycrawl = new Berrycrawl({
  apiKey: process.env.BERRYCRAWL_API_KEY,
  timeoutInSeconds: 90,
  maxRetries: 3,
});

License

Apache 2.0. See LICENSE.