@scrape-admin/client

Typed API client for @scrape-admin/web. It wraps the REST API (teams, members, API keys, browsers, AI connections, collections, profiles, entries and runs) so other apps and scripts don't have to hand-roll fetch calls and response types. It is plain TypeScript with no node: imports, so it runs on Node, Bun and Cloudflare Workers.

Installation

npm install @scrape-admin/client

Usage

import { createScrapeAdmin, ScrapeAdminError } from '@scrape-admin/client'

const sa = createScrapeAdmin({
  baseURL: 'https://scrape.example.com',
  teamId: 'team_…',
  apiKey: process.env.SCRAPE_ADMIN_API_KEY!,
})

const team = await sa.team.get() // { id, name, role }
const entry = await sa.entries.create(collectionId, {
  name: 'Widget @ Shop',
  sourceUrl: 'https://shop.example.com/widget',
  cronExpression: '0 */6 * * *',
})
await sa.entries.update(collectionId, entry.id, { active: false })

| Option | Description | |-------------|--------------------------------------------------------------------------| | baseURL | Origin of the Scrape Admin web app. | | teamId | Team the root of the client is bound to. Optional for user API keys. | | apiKey | Team (sa_…) or user (sau_…) API key, sent as Authorization: Bearer <apiKey>. | | userAgent | User-Agent header. Default scrape-admin-client. | | fetch | Custom fetch implementation (tests, Workers service bindings). |

User API keys

There are two kinds of API key:

• A team API key (sa_…) belongs to one team and has a role in it. Pass that team's teamId.
• A user API key (sau_…) acts as the user who created it. It reaches every team the user is a member of, with the user's role there, and it can create teams.

Without a teamId, the client exposes the account-level API. forTeam(teamId) returns a client for one team with every namespace below:

const sa = createScrapeAdmin({ baseURL, apiKey: process.env.SCRAPE_ADMIN_USER_KEY! })

const team = await sa.teams.create({ name: 'Acme' }) // you become its owner
const acme = sa.forTeam(team.id)

const { key } = await acme.apiKeys.create({ name: 'ci', role: 'admin' }) // shown only once
await acme.members.invite({ email: '[email protected]', name: 'Ada', role: 'member' })
await acme.collections.create({ name: 'Products' })

| Namespace | Methods | |-----------------|-----------------------------------------------------------------| | teams | list(), create({ name }) | | myInvitations | list(), accept(id), reject(id) (invitations to you) | | — | forTeam(teamId) |

With a teamId, these are available too, alongside the team namespaces. A team key can't create teams (teams.create rejects with forbidden) and has no invitations of its own. User API keys themselves can only be managed from the web app while signed in.

API

| Namespace | Methods | |---------------|-----------------------------------------------------------------------------------------------------------------------------------------| | team | get(), dashboard(), update({ name }), remove(), search(q?) | | members | list(), update(memberId, { role }), remove(memberId), invite({ email, name, role? }) | | invitations | list(), revoke(id) (invitations sent by the team) | | apiKeys | list(), create(body), update(id, patch), remove(id), rotate(id) | | browsers | list(), get(id), create(body), update(id, patch), remove(id, { detach? }), test(id), setDefault(id \| null) | | aiConnections | list(), get(id), create(body), update(id, patch), remove(id, { detach? }), test(id), setDefault(id \| null) | | collections | list(), get(id), create(body), update(id, patch), remove(id) | | profiles | list(), get(id), create(body), update(id, patch), remove(id), run(profile), validate(profile), generate(body, wait?), startGenerate(body), repair(id, body?, wait?), startRepair(id, body?) | | aiRuns | get(id), wait(id, { interval?, timeout?, signal? }), cancel(id) | | entries | list(c), get(c, e), create(c, body), update(c, e, patch), remove(c, e), refresh(c, e), sync(body), runs(c, e, { page?, pageSize? }) | | runs | list({ page?, pageSize?, status?, collectionId? }), get(c, e, runId) | | — | checkUrl(url) |

• update methods send a partial body with PATCH.
• Team management needs a role: team.update, members, invitations, apiKeys, profiles.generate, profiles.repair, and every browsers and aiConnections method except list and get need owner or admin, and team.remove needs owner. Only an owner can grant owner, and an API key can never get a higher role than the caller. Otherwise the call rejects with forbidden.
• apiKeys.create and apiKeys.rotate return the secret key. It is not returned again.
• entries.sync({ entries, remove?, removeMode?, refresh?, force? }) syncs up to 250 entries across the team's collections in one call:
  • With an id, an entry gets only its changed fields written. Without one, or with an unknown id, it is created.
  • remove deletes the listed entries, or deactivates them with removeMode: 'deactivate'.
  • refresh runs every synced entry once.
  • It returns one result per item (ref is echoed back) and never fails the whole call for one item.
• profiles.run(profile) runs a profile once against its url without saving it and returns { id, result }. Pass a type argument when you know the result shape: sa.profiles.run<MyResult>(profile).
• profiles.validate(profile) takes an object or a JSON/YAML string. Problems come back in the response ({ valid, results, … }), not as an HTTP error.
• profiles.generate({ url, fields, instructions?, aiConnectionId?, browserId? }) has the server's AI agent write a profile and profiles.repair(id, body?) fix a saved one. The agent runs in the background on the server (it takes minutes); both poll until it is done (every 3s, up to 15 minutes; tune with { interval, timeout, signal }) and save nothing: pass savable to profiles.create or update to profiles.update. A spent 24h budget rejects with rate_limited, a failed or cancelled run with ai_run_failed, and giving up with timeout (the run keeps going). startGenerate / startRepair return the AI run ({ id, status: 'running' }) right away; poll it with aiRuns.get(id) or aiRuns.wait(id) and stop it with aiRuns.cancel(id).
• browsers and aiConnections manage where profiles run and which model generates them. Credentials are write-only, remove returns what used the resource, and setDefault(null) switches back to Scrape Admin Cloud.
• entry.data and run.result are typed unknown. They hold the ResultBag of the latest run and are left out of list responses (entries.list, entries.runs): fetch the entry or run by id to read them.

Request and response types (ScrapeAdminEntry, ScrapeAdminCreateEntry, ScrapeAdminRun, …) are exported from the package root and derived from @scrape-admin/web-schema.

Errors

Every method rejects with a ScrapeAdminError:

| Field | Description | |----------|----------------------------------------------------------------------------------------------| | code | unauthorized, forbidden, not_found, validation, rate_limited, network, ai_run_failed, timeout or unknown | | status | HTTP status when the server answered, otherwise undefined | | body | Parsed response body of the failed request |

The original error is not attached as cause, because ofetch errors carry the request headers. This keeps the API key out of logs.

try {
  await sa.entries.get(collectionId, entryId)
}
catch (error) {
  if (error instanceof ScrapeAdminError && error.code === 'not_found') {
    // …
  }
  throw error
}

Migrating from createHttpAdapter

createHttpAdapter (the 0.1.x API) still works but is deprecated. It is now backed by createScrapeAdmin, so it sends updates as PATCH, no longer logs requests, and rejects with ScrapeAdminError.

| 0.1.x | Now | |------------------------------------------|--------------------------------------| | createHttpAdapter({ … }) | createScrapeAdmin({ … }) | | client.run(profile) | sa.profiles.run(profile) | | client.profiles.getById(id) | sa.profiles.get(id) | | client.profiles.delete(id) | sa.profiles.remove(id) | | client.collections.getById(id) | sa.collections.get(id) | | client.collections.delete(id) | sa.collections.remove(id) |

Scripts

Run these from the repo root:

pnpm --filter '@scrape-admin/client' build      # bundle with unbuild
pnpm --filter '@scrape-admin/client' dev        # bundle in watch mode
pnpm --filter '@scrape-admin/client' test       # vitest
pnpm --filter '@scrape-admin/client' typecheck  # tsc --noEmit
pnpm --filter '@scrape-admin/client' lint       # eslint