octoparse-client-js — the official DataHub JavaScript SDK

The official JavaScript client for Octoparse DataHub (npm package octoparse-client): discover, run and consume Data Apps. Written in TypeScript and shipped as compiled JS plus type declarations, so plain JS projects can use it directly. A small self-contained project with zero runtime dependencies (Node's built-in fetch only) that speaks only the platform's public /v1 REST API. Method-for-method aligned with the Python SDK (the octoparse-client pip package).

Requires Node >= 20.17 (ESM distribution; from this version CJS can require it directly).

Installation

npm install octoparse-client

Usage

import { Client } from "octoparse-client";

// baseUrl defaults to the production endpoint https://api-datahub.octoparse.com;
// pass it explicitly for local / staging environments
const client = new Client({ apiKey: "demo-key" });

// Discover
const result = await client.search("reviews", { limit: 10 });
const detail = await client.getApp("demo/reviews-store-query");   // or the card's app_id

// Run (wait for the terminal state) and consume. Time parameters are always in
// milliseconds (timeout / pollInterval; the Python SDK uses seconds). A timeout
// throws TimeoutError but does not cancel the server-side run.
const run = await client.call("demo/partner-reviews-api", { product: "p-9001" }, {
  timeout: 120_000, raiseOnFailure: true,
});
for await (const record of client.iterateRecords(run.run_id)) {
  console.log(record.content, record.rating);
}

// Inventory and reconciliation
for await (const r of client.iterateRuns({
  runKind: "production", createdFrom: "2026-08-01T00:00:00Z",
})) {
  console.log(r.run_id, r.state, r.billing);
}
console.log(await client.billing({ groupBy: "data_app", tzOffset: 480 }));

// Result data is kept for 90 days by default; mark datasets you want to keep
await client.setDatasetRetention(run.dataset_id, true);

call() starts a run and polls until it reaches a terminal state; for non-blocking semantics use run() to get a run_id, then getRun() / cancel() yourself. Apps shared with you point-to-point do not appear in marketplace results; query them with search("", { sharedWith: "me" }). All methods and the error mapping live in src/client.ts.

App references come in two forms, accepted interchangeably by getApp() / run() / call() and by the dataApp filter of the run listings: the two-part <namespace>/<app_name> reference (taken from the card's namespace and app_name, human-readable, breaks once the publisher renames the App); and the stable identifier app_id (app_<hex>, taken from the card's / detail's app_id field, immune to renames). For long-term integrations such as config files or scheduled jobs, pin the app_id.

Configuration

When new Client() is constructed without baseUrl / apiKey, they fall back to the OCTOPARSE_BASE_URL / OCTOPARSE_API_KEY environment variables (template in .env.example; the SDK does not load .env itself, the caller does), then to the production endpoint https://api-datahub.octoparse.com / anonymous access.

Bring your own token (tokenProvider, advanced)

The standard credential for programmatic access is the API Key. Callers that already hold a valid identity JWT elsewhere (gateway forwarding, the login state of an MCP OAuth authorization-code flow) can inject their self-managed credential with tokenProvider, which is mutually exclusive with apiKey: pass a zero-argument function returning the currently valid token, or implement the TokenProvider interface (token(minTtl?), renewal is the implementation's job). For providers with minTtl semantics, the SDK by default requires a remaining token lifetime of at least 12h before submitting a run (otherwise it first asks the provider for a fresher token; tune with the runTokenMinTtl millisecond value). The platform snapshots the credential at enqueue time and replays it for billing at the terminal state, so the fresher the token at submission, the smaller the window in which a long run's billing credential expires. The SDK ships no username/password-to-token implementation (ROPC is deprecated by OAuth 2.1 and the platform no longer accepts plaintext account credentials).

Tests

npm test             # offline unit tests (injected fetch mock, no server required)
npm run typecheck    # strict type checking (including tests)