Scraping AI JavaScript/TypeScript SDK (scraping-ai)

A type-safe TypeScript SDK for interacting with Scraping AI service API endpoints. Uses native fetch (Node 18+), no HTTP dependency.

This SDK focuses strictly on core service execution endpoints (pipeline orchestration, web extraction, crawling, finder, keywords, etc.), excluding account management, billing, payments, and administrative routes.

Installation

npm install scraping-ai-sdk

Quick Start (One-Shot Helpers)

1. Web Extraction in One Call

import { ScrapingAIClient } from "scraping-ai";

const client = new ScrapingAIClient({ apiKey: "your_api_key_here" });

const data = await client.extract("https://example.com", {
  schema: { title: "string", price: "number" },
});
console.log(data.results);

2. Page Crawling in One Call

const crawlResult = await client.crawl("https://example.com", { useBrowser: true });
console.log(crawlResult.results);

3. Discover URLs in One Call

const found = await client.findUrls("https://example.com", { maxDepth: 2 });
console.log(found.results);

4. Keyword Generation in One Call

const keywords = await client.generateKeywords("dummy_context", { numKeywords: 15 });
console.log(keywords.keywords);

5. Schema Generation in One Call

const schema = await client.generateSchema("Extract title, author, date");
console.log(schema.schema);

Service Modules Overview

• One-shot helpers: client.extract(...), client.crawl(...), client.findUrls(...), client.generateKeywords(...), client.generateSchema(...).
• client.extractor — data extraction module (run, status, result).
• client.crawler — page crawling module (run, status, result).
• client.finder — URL discovery module (run, status, result).
• client.keywords — keyword generation module (run, status, result).
• client.schema — JSON Schema generation module (run, status, result).
• client.ranker — relevance ranking module (run, status, result).
• client.searchUrl — search URL generator module (run, status, result).
• client.searchResults — search results discovery module (run, status, result).
• client.exports — S3 exports management (create, list, get).
• client.inputStates (alias client.pipeline) — multi-step pipeline orchestration (create, list, get, patch, runFlow, waitForTask).
• client.data — query extracted records (getByState, exportByState).

License

MIT