web-scraping-api-sdk

Node client for the ScrapingBee web scraping API. Zero dependencies, typed, Node 18+. Handles proxies, headless-browser rendering and anti-bot for you, and adds nine dedicated scrapers that hand back parsed JSON.

Independent client, separate from ScrapingBee's own scrapingbee SDK. Key from app.scrapingbee.com, 1,000 credits free.

npm install web-scraping-api-sdk

Endpoints, parameters, credit costs and response fields below were checked against the live API on 2026-08-25.

First request

const { ScrapingBeeAPI } = require("web-scraping-api-sdk");

const api = new ScrapingBeeAPI(process.env.SCRAPINGBEE_API_KEY);

const page = await api.scrape("https://news.ycombinator.com", { render_js: false });
console.log(page.status, page.cost, page.requestId);
console.log(page.text().slice(0, 500));

Auth rides in the Authorization: Bearer header. The api_key query parameter is deprecated upstream and this client never sends it.

Recipes

Never overpay for a page

Let the API walk the proxy ladder and bill only the rung that worked. Nothing worked means nothing charged.

const page = await api.auto("https://example.com", 25);
console.log(page.autoCost);   // 1, 5, 10, 25, or 0 if every tier failed

Pull search results

const { organic_results } = (await api.google("web scraping api", "classic", { country_code: "us" })).json();
organic_results.forEach(r => console.log(r.position, r.title, r.url));

Swap the second argument for a different surface: news, maps, images, lens, shopping, ai_mode or ads. Alongside organic_results you get ai_overviews, top_ads, shopping_ads, questions, top_stories, news_results, local_results, knowledge_graph and related_searches.

Surface-specific landing pages: News . Images . Lens . Shopping . AI Mode . Ads . Google feature page

Watch the edges: news will not run with device: "mobile", lens wants an image URL as the search term, and ai_mode truncates past 400 characters.

For a lighter, flat-10-credit alternative there is Fast Search, which returns answer_box, organic, people_also_ask, related_searches and top_stories:

await api.fastSearch("web scraping api");

Track ecommerce prices

const search = (await api.amazonSearch("laptop stand")).json();
const asin = search.products[0].asin;

const product = (await api.amazonProduct(asin)).json();
console.log(product.title, product.buybox, product.rating);

const walmart = (await api.walmartProduct("10450114")).json();
console.log(walmart.price, walmart.currency, walmart.out_of_stock);

amazonProduct accepts an ASIN or a product URL and answers with asin, brand, bullet_points, buybox, category, coupon, currency, delivery and description. An unknown ASIN comes back as HTTP 404 with {"error": "Product not found"}. Walmart products carry price, price_strikethrough, rating, rating_count, fulfillment, gtin, seller_id and images.

Amazon: search . ASIN . reviews . offers . best sellers . sellers . feature page

Walmart: prices . search results . inventory . sellers . feature page

Read video data and transcripts

await api.youtubeSearch("web scraping tutorial", { duration: "long", upload_date: "this_year" });
const meta = (await api.youtubeMetadata("dQw4w9WgXcQ")).json();
const subs = (await api.youtubeSubtitles("dQw4w9WgXcQ")).json();

Pass a bare video id to the last two, not a full watch URL; that trips people up. What comes back covers the channel (channel_id, channel_url), the engagement counters (like_count, comment_count), and the publishing side (upload_date, categories, tags, thumbnails, formats, age_limit, is_live), plus title, description and duration. Flat 5 credits.

Video scraper . transcripts . comments . captions . feature page

Capture AI answers

const gpt = (await api.chatgpt("what is a web scraping api")).json();
const gem = (await api.gemini("what is a web scraping api")).json();
console.log(gpt.results_markdown, gpt.llm_model);
console.log(gem.results_markdown, gem.citations);

ChatGPT gives results_text, results_markdown, results_json, full_html, llm_model and prompt. Gemini gives results_text, results_markdown, citations, full_html and prompt. 15 credits each. ChatGPT citations are best effort, not guaranteed on every call. ChatGPT . Gemini

Feed a page to an LLM

const md = await api.scrape("https://example.com", { return_page_markdown: true });

Markdown out of the box beats stripping tags yourself when you are building RAG input. For selector-driven or prompt-driven extraction instead, see data extraction and AI web scraping:

await api.scrape("https://news.ycombinator.com", {
  extract_rules: { titles: { selector: ".titleline > a", type: "list" } },
});
await api.scrape("https://example.com", { ai_query: "the main heading" });  // +5 credits

Take a screenshot

const png = await api.screenshot("https://example.com", true);
require("fs").writeFileSync("shot.png", png);

Methods and cost

| Method | Credits | | --- | --- | | scrape(url, params) | 1 no JS, 5 with JS, 10 or 25 premium, 75 stealth, +5 for AI params | | auto(url, maxCost, params) | whichever tier succeeds, 0 if none do | | screenshot(url, fullPage, params) | same ladder as scrape | | google(search, searchType, params) | 15, or 10 with light_request | | fastSearch(search, params) | 10 | | amazonSearch / amazonProduct / amazonPricing | 5 light, 15 standard | | walmartSearch / walmartProduct | 10 light, 15 standard | | youtubeSearch / youtubeMetadata / youtubeSubtitles | 5 | | chatgpt(prompt, params) | 15 | | gemini(prompt, params) | 15 | | shopee(url, params) | 75 | | agenticSearch(prompt, limit, params) | 3,750 | | usage() | free, 6 calls per minute |

Shopee takes a product URL. Agentic employee search is the priciest call in the catalogue, so check usage() before looping it. Plan limits at scrapingbee.com/pricing.

What comes back

| Property | Header | Meaning | | --- | --- | --- | | cost | Spb-cost | credits charged | | autoCost | Spb-auto-cost | Auto-Mode charge, 0 on total failure | | requestId | Spb-request-id | identifies the call to support | | resolvedUrl | Spb-resolved-url | final URL after redirects |

Plus text(), json(), buffer(), status and headers.

Failures

Non-2xx throws ScrapingBeeError with statusCode, body and requestId. A 500 is never charged, so retrying it costs nothing.

const { ScrapingBeeError } = require("web-scraping-api-sdk");

try {
  await api.scrape("https://example.com", { stealth_proxy: true });
} catch (error) {
  if (error instanceof ScrapingBeeError) console.error(error.statusCode, error.requestId);
}

Notes

Public pages only. Scraping behind a login is prohibited by the ScrapingBee terms. Do not hand your API key to AI coding assistants.

Docs . Pricing . Examples in 8 languages

MIT.