scrapeunblocker

v0.6.0

Published

10 hours ago

Official Node.js client for the ScrapeUnblocker web scraping API - JavaScript-rendered pages that bypass Cloudflare, DataDome, PerimeterX, Akamai and more.

ScrapeUnblocker Node.js client

Official Node.js / TypeScript client for the ScrapeUnblocker web scraping API.

Every request is fully JavaScript-rendered in a real browser and routed through premium proxies, so it bypasses Cloudflare, DataDome, PerimeterX, Akamai, Kasada and similar anti-bot systems - from one simple call. You are only billed for successful requests.

Highest success rate on the market (95%+ on live production traffic)
Rendered HTML or parsed JSON - no per-site parsers to maintain
Zero runtime dependencies, fully typed, ESM + CommonJS

Install

npm install scrapeunblocker

Requires Node.js 18+.

Quickstart

import { ScrapeUnblockerClient } from "scrapeunblocker";

const su = new ScrapeUnblockerClient(); // reads SCRAPEUNBLOCKER_KEY, or { apiKey: "YOUR_API_KEY" }

// Rendered HTML for any URL
const html = await su.getPageSource("https://example.com");

// Structured JSON instead of HTML (products, listings, search results, ...)
const product = await su.getParsed("https://www.amazon.com/dp/B08N5WRWNW");
console.log(product.pageType); // "product"
console.log(product.data);

CommonJS works too:

const { ScrapeUnblockerClient } = require("scrapeunblocker");

Get your API key at app.scrapeunblocker.com. The free trial does not require a credit card.

Authentication

Set an environment variable and the client picks it up:

export SCRAPEUNBLOCKER_KEY="YOUR_API_KEY"

const su = new ScrapeUnblockerClient(); // reads SCRAPEUNBLOCKER_KEY

Fetch rendered HTML

const html = await su.getPageSource("https://www.nordstrom.com/browse/women/clothing/dresses", {
  proxyCountry: "US", // route through a specific country
  timeSleep: 3,       // wait extra seconds after load
});

Browser steps

Drive the page in a real browser after it loads - click, type, scroll, or wait for content - by passing an ordered steps array. The client JSON-encodes it for you.

const html = await su.getPageSource("https://example.com/search", {
  steps: [
    { action: "wait_for", selector: "#search", timeout_ms: 5000 },
    { action: "type", selector: "input[name=q]", value: "wireless earbuds", clear: true },
    { action: "click", selector: "button[type=submit]" },
    { action: "wait_for_text", value: "results" },
    { action: "scroll", value: "bottom" },
  ],
});

Available actions:

| Action | Fields | |---|---| | wait_for | selector, selector_type? (css | xPath | className | tagName, default css), timeout_ms? | | wait_for_text | value (text), timeout_ms? | | wait | value (ms) | | click | selector, selector_type?, timeout_ms? | | type | selector, selector_type?, value (text), clear?, timeout_ms? | | select | selector, selector_type?, value, timeout_ms? | | press_key | value (Enter, Tab, Escape, Backspace, Delete, Space, ArrowUp/Down/Left/Right, Home, End, PageUp, PageDown) | | scroll | value ("bottom" or an integer pixel amount) |

A request with steps runs once and is non-idempotent. If a step fails, the call rejects with a ValidationError (HTTP 422) whose body is the JSON { error: "step_failed", step_index, action, reason, selector, html }:

import { ValidationError } from "scrapeunblocker";

try {
  await su.getPageSource(url, { steps: [{ action: "click", selector: "#missing" }] });
} catch (err) {
  if (err instanceof ValidationError) {
    const detail = JSON.parse(err.body ?? "{}");
    console.log(detail.step_index, detail.reason); // 0 "selector not found"
  }
}

List elements

Set listElements: true to get structured JSON - { url, count, elements } - instead of HTML. The return type narrows automatically:

const { url, count, elements } = await su.getPageSource("https://example.com", {
  listElements: true,
  steps: [{ action: "wait_for", selector: ".card" }],
});
console.log(count, elements);

Get parsed JSON

const result = await su.getParsed("https://www.walmart.com/ip/12345");
console.log(result.pageType); // e.g. "product"
console.log(result.source);   // how it was extracted
console.log(result.data);     // the fields

// If a parse ever comes back wrong, force a fresh set of rules:
const fresh = await su.getParsed(url, { refreshRules: true, rulesHint: "price is missing" });

Google search (SERP)

const serp = await su.serp("web scraping api", { pagesToCheck: 2, proxyCountry: "US" });

Google Local (Maps)

const local = await su.googleLocal("coffee shops in chicago", { proxyCountry: "US", gl: "us" });
for (const biz of (local as any).results) {
  console.log(biz.name, biz.rating, biz.reviews, biz.address);
}

Google Images

const images = await su.googleImages("golden retriever puppy", { proxyCountry: "US", pages: 3 });
for (const img of (images as any).results) {
  console.log(img.imageUrl, img.sourceDomain, img.title);
}

Meta Ad Library

Look up an advertiser's ads in the Meta (Facebook) Ad Library as JSON:

const ads = await client.metaAdLibrary("Nike", { country: "US" });

Scope the search with country, activeStatus, mediaType and cap the result with maxAds. Omitted options fall back to the API's own defaults.

Oopbuy product search

Search 1688, Taobao or Oopbuy official listings as JSON:

const goods = await su.oopbuySearch("wireless earbuds", {
  channel: "1688",     // "1688" (default), "taobao" or "official"
  page: 1,
  pageSize: 20,        // max 60
  sort: "best_selling", // "default", "price_asc", "price_desc" or "best_selling"
});
for (const item of (goods as any).results) {
  console.log(item.title, item.price, item.monthSold, item.url);
}

Oopbuy trademark-blocks brand keywords (e.g. "nike") at its own backend. Those come back as a successful 200 with keywordRejected: true and an empty results array - Oopbuy's own genuine response, not an error.

Amazon

Product and search data as JSON, priced in the marketplace's own currency:

// One product by ASIN (or { url: "https://www.amazon.de/dp/B0BSHF7WHW" })
const product = await su.amazonProduct({ asin: "B0BSHF7WHW", marketplace: "amazon.com" });
console.log(product.title, product.price, product.currency, product.rating);

// Keyword search
const results = await su.amazonSearch("wireless headphones", { sort: "price_asc" });
for (const item of results.results) {
  console.log(item.title, item.price, item.currency, item.asin);
}

proxyCountry defaults to the marketplace's home country (amazon.com -> US, amazon.de -> DE), so prices come back in the right currency with no configuration.

eBay search

Listings from any of the 19 regional eBay marketplaces as JSON:

const items = await su.ebaySearch("iphone 13", {
  marketplace: "ebay.com",
  condition: "used",
  sort: "newly_listed",
});

if (items.exactMatches) {
  for (const item of items.results) {
    console.log(item.title, item.price, item.currency, item.condition);
    console.log("  seller:", item.seller?.username, item.seller?.feedbackPercent);
  }
}

exactMatches is false when eBay found nothing for the keyword and answered with its own loosely-related suggestions instead, so check it before using the listings.

TikTok

const profile = await su.tiktokProfile("nasa", { maxVideos: 5 });     // exact stats + newest videos
const video = await su.tiktokVideo("https://www.tiktok.com/@nasa/video/7665075736742530317", { includeTranscript: true });
const tag = await su.tiktokHashtag("nasa", { maxVideos: 10 });
const results = await su.tiktokSearch("space telescope", { maxResults: 25 });   // TikTok's own ranking
const comments = await su.tiktokComments("https://www.tiktok.com/@nasa/video/7665075736742530317", { maxComments: 40 });
console.log(profile.stats.followers, video.stats.plays, tag.stats.views, results.resultsCollected, comments.totalComments);

Profiles and hashtags list up to 10 videos in a couple of seconds from TikTok's server-rendered widget; ask for more (up to 200) and the real grid is scrolled in a browser session.

Cookies and the serving proxy

const page = await su.getPageWithCookies("https://example.com");
console.log(page.html, page.cookies, page.proxy);

Images

import { writeFile } from "node:fs/promises";
const bytes = await su.getImage("https://example.com/photo.jpg");
await writeFile("photo.jpg", bytes);

Skyscanner plugins

Flights, hotels and car hire as JSON:

const locations = await su.skyscanner.flightLocations("London");

const flights = await su.skyscanner.flights({
  origin: "London", dest: "New York",
  depart_date: "2026-09-01", adults: 1, currency: "USD",
});

const hotels = await su.skyscanner.hotels({ destination: "Madrid", checkin: "2026-09-01", checkout: "2026-09-03" });
const cars = await su.skyscanner.carhire({ pickup: "Madrid", pickup_datetime: "2026-09-01T10:00", dropoff_datetime: "2026-09-03T10:00" });

Southwest plugin

Southwest Airlines fares as raw booking JSON. Pass IATA airport codes and dates; leave return_date out for a one-way search:

const results = await su.southwest.flights({
  origin: "DAL", dest: "HOU",
  depart_date: "2026-10-20", return_date: "2026-10-27",
  adults: 1, fare_type: "dollars", proxy_country: "US",
});

Parameters: origin and dest (IATA codes), depart_date and optional return_date (YYYY-MM-DD; omit for one-way), adults (1-8, default 1), fare_type ("dollars" (default) or "points"), proxy_country (default "US") and max_attempts (1-5, default 3).

Error handling

Non-2xx responses reject with typed errors, all subclasses of ScrapeUnblockerError.

import {
  ScrapeUnblockerClient,
  BlockedError,
  PaymentRequiredError,
  RateLimitError,
  UpstreamOutageError,
} from "scrapeunblocker";

const su = new ScrapeUnblockerClient();
try {
  await su.getPageSource("https://example.com");
} catch (err) {
  if (err instanceof BlockedError) {
    // 403: the target blocked every bypass path (not billed)
  } else if (err instanceof PaymentRequiredError) {
    // 402: quota, credit limit, or a failed payment - fix billing
  } else if (err instanceof RateLimitError) {
    // 429: slow down
  } else if (err instanceof UpstreamOutageError) {
    // 503: the target site itself is down - retry later
  }
}

| Error | Status | Meaning | |---|---|---| | InvalidRequestError | 400 | Bad URL, unsupported scheme, or the API key header was not sent | | AuthenticationError | 401 | Key not recognised - typo, stray whitespace, or a rotated key | | NoSubscriptionError | 401 | Key is fine, but the account has no active plan | | PaymentRequiredError | 402 | Billing block - base class for the three below | | QuotaExceededError | 402 | The plan's requests for this period are used up | | CreditLimitExceededError | 402 | Unpaid balance is past the account's credit limit | | PaymentFailedError | 402 | A card payment was declined three times | | BlockedError | 403 | Blocked by bot protection on every path | | NotFoundError | 404 | Page loaded but held no image (getImage only) | | BrowserTimeoutError | 408 | Our browser run timed out before the page was ready | | UnsupportedContentError | 415 | The URL serves something other than HTML | | ValidationError | 422 | Missing or wrong-typed parameter; body holds the detail array | | RateLimitError | 429 | Too many requests | | UpstreamOutageError | 503 | The target origin is down | | ServerError | 5xx | Unexpected server error, including a 504 upstream timeout | | ScrapeTimeoutError | - | This client gave up locally before the API answered | | ConnectionError | - | Could not reach the API |

Transient failures (429, 502, 503, 504 and network errors) are retried automatically with exponential backoff. A 401 or 402 is never retried - it clears when the key or the billing state changes, not on another attempt. Neither is billed or counted against your quota, because the request is refused before anything is scraped.

Billing errors (402)

The three billing blocks share a status code and differ only in their message, so the client throws a dedicated error for each:

import {
  CreditLimitExceededError,
  PaymentFailedError,
  QuotaExceededError,
} from "scrapeunblocker";

try {
  await su.getPageSource("https://example.com");
} catch (err) {
  if (err instanceof QuotaExceededError) {
    // plan quota (plus any overage allowance) is used up for this period
  } else if (err instanceof CreditLimitExceededError) {
    // unpaid balance passed the account credit limit
  } else if (err instanceof PaymentFailedError) {
    // card declined three times - update the payment method
  }
}

When more than one applies, the most serious wins: failed payment outranks credit limit, which outranks quota. All three lift by themselves once the billing state changes - access returns within about a minute, and the API key stays the same. One catch worth knowing: subscribing to a new plan does not clear PaymentFailedError, because the old unpaid invoice stays open until it is paid.

Full details for every status code: docs.scrapeunblocker.com/errors.

Configuration

new ScrapeUnblockerClient({
  apiKey: undefined,   // or SCRAPEUNBLOCKER_KEY env var
  baseUrl: "https://api.scrapeunblocker.com",
  timeout: 180000,     // ms; protected pages can be slow
  maxRetries: 2,
});

License

MIT