xyte

The Zyte API POST /v1/extract, on any proxy.

Same request JSON, same response JSON, same error codes as https://api.zyte.com/v1/extract. The difference is where traffic exits: Oxylabs, Bright Data, Decodo, IPRoyal, ZenRows, ScrapingBee, ScraperAPI, 30+ other presets, any http://user:pass@host:port, no proxy at all, or Zyte itself. You keep Zyte's conveniences: geolocation, sticky sessions, cookies, ban detection and retries, browser rendering with actions and network capture, markdown, and (with an LLM) AI extraction in Zyte's schemas.

Runs on Node.js ≥ 22 and Bun.

Three ways to use it:

• Library: new Xyte({ proxy }).extract(request) in your code.
• Drop-in server: xyte serve, then point existing Zyte code at it by changing only the base URL.
• In front of Zyte: keep Zyte for the sites that need it, send the rest through cheaper exits, with one client.

Why it exists

xyte is a personal project: a library that makes one scraping flow easier to write. One request format for every exit, fallbacks from a cheap exit to a better one when a site bans, and the same code whether the request goes to Zyte, to a proxy, or nowhere special. It speaks Zyte's API because that API is well designed and well documented, so code moves between xyte and Zyte in either direction by changing a base URL. It doesn't try to compete with or replace Zyte: Zyte is one of its providers (zyteApi()), often the right one for hard sites. It isn't affiliated with Zyte (disclaimer).

Docs site: me.xantiagoma.com/xyte (search, every page below).

Docs: coverage of the Zyte API · providers · combining providers · actions · observability · configuration · browser & stealth · AI extraction · deploying · mounting in your app · migrating from Zyte · API reference · examples

npm i @xtandard/xyte
# browser requests (browserHtml, screenshot, actions, networkCapture):
npm i playwright && npx playwright install chromium
# proxies on Node.js (Bun has proxy support built into fetch); undici 8 needs Node 22.19+:
npm i undici

On npm it's @xtandard/xyte (npm turns xyte down as too close to vite and bytes); the command it installs is xyte.

Library

import { Xyte, oxylabs, decodeBody } from '@xtandard/xyte';

const xyte = new Xyte({
  proxy: oxylabs({ username: process.env.OXYLABS_USERNAME!, password: process.env.OXYLABS_PASSWORD! }),
});

// Raw HTTP (cheap, fast)
const page = await xyte.extract({ url: 'https://example.com', httpResponseBody: true, geolocation: 'US' });
page.statusCode; // 200; a target 404 is a successful response with statusCode 404, as on Zyte
decodeBody(page.httpResponseBody!); // the HTML (bodies are base64, as on Zyte)

// Rendered in a browser, with Zyte actions
const rendered = await xyte.extract({
  url: 'https://example.com/app',
  browserHtml: true,
  actions: [
    { action: 'waitForSelector', selector: { type: 'css', value: '#results' }, timeout: 10 },
    { action: 'scrollBottom' },
  ],
  networkCapture: [{ filterType: 'url', matchType: 'contains', value: '/api/', httpResponseBody: true }],
});

await xyte.close(); // closes the shared browser

Closing the client

The first render launches a browser that the client reuses, and it keeps your process alive until you close the client:

await using xyte = new Xyte({ proxy }); // closed when the block ends, even on an exception
// or, where that syntax doesn't run (plain JavaScript on Node.js 22):
const xyte = new Xyte({ proxy });
try { … } finally { await xyte.close(); }

The client's lifecycle explains await using, where it runs, the return await trap, services (one client, closed on SIGTERM), Redis stores and tests.

Many at once

Many at once: for await (const r of xyte.extractMany(requests)) … yields results as they finish ({ index, response } or { index, error }), at most concurrency in flight, from any list or async source; extractAll() collects them in order. From a shell: xyte batch urls.txt > out.jsonl.

Requests and responses are typed from Zyte's OpenAPI spec. All options (retries, timeout, concurrency, isBan, onRetry…) are in configuration.

Drop-in server

XYTE_PROVIDER=oxylabs OXYLABS_USERNAME=… OXYLABS_PASSWORD=… npx @xtandard/xyte serve --port 8011

- const res = await fetch('https://api.zyte.com/v1/extract', {
+ const res = await fetch('http://localhost:8011/v1/extract', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', Authorization: 'Basic ' + btoa(apiKey + ':') },
    body: JSON.stringify({ url, httpResponseBody: true }),
  });

As an HTTP proxy, like Zyte's :8011: xyte serve --proxy-port 8012, then curl -x http://localhost:8012 -U key: --cacert <(xyte ca) https://site (proxy mode).

Long renders or queues: POST /v1/jobs (the same body, plus an optional callbackUrl) answers 202 { id } at once; GET /v1/jobs/{id} has the result, which is also POSTed (signed) to callbackUrl (jobs).

Set XYTE_API_KEYS=key1,key2 to require a key (sent the way Zyte expects it: basic-auth username). Docker images (full with Chromium, or slim HTTP-only) and Cloud Run / Fly / Render / App Platform / Modal configs: deploy. To serve it from your own app: createHandler(xyte) (Hono, Bun.serve, Next.js…) or toNodeHandler(…) (Express, Fastify), see mounting.

CLI one-offs:

npx @xtandard/xyte extract https://example.com --raw              # HTTP, prints the body
npx @xtandard/xyte extract https://example.com --browser --country GB
npx @xtandard/xyte extract --json '{"url":"https://example.com","browserHtml":true}'
npx @xtandard/xyte doctor --country US   # each configured provider: credentials, exit IP/country, quota

Providers

| Preset | Exit | Geolocation / session encoded as | | --- | --- | --- | | direct() (default) | this machine | none (geolocation can't apply: you get a warning) | | proxyUrl(url \| urls[]) | any proxy; several rotate across retries | none | | oxylabs({ username, password, type? }) | pr.oxylabs.io:7777 / dc.oxylabs.io:8000 | customer-U-cc-US-sessid-…-sesstime-10 | | brightData({ customerId, zone, password }) | brd.superproxy.io:33335 | brd-customer-…-zone-…-country-us-session-… | | decodo({ username, password }) (alias smartproxy) | gate.decodo.com:7000 | user-U-country-us-session-…-sessionduration-10 | | iproyal({ username, password }) | geo.iproyal.com:12321 | password P_country-us_session-…_lifetime-10m | | zenrows({ apiKey }) | ZenRows API proxy mode | premium_proxy=true&proxy_country=us&session_id=… | | scrapingBee({ apiKey }) | ScrapingBee proxy mode | premium_proxy=True&country_code=us&session_id=… | | scraperApi({ apiKey }) | ScraperAPI proxy mode | scraperapi.country_code=us.session_number=… | | zyteApi({ apiKey, name? }) | Zyte API itself | the request, forwarded unchanged | | xyteApi({ endpoint, apiKey, name? }) | another xyte serve (a deployment with its own proxy) | the request, forwarded; its attempts come back in the report | | template('http://u[-country-{country:lower}]-session-{session}:p@host:port') | any provider, from its URL format | placeholders + optional [parts] | | custom(name, ctx => url) | anything you build | you decide |

…and NodeMaven, SOAX, NetNut, Webshare, Rayobyte, PacketStream, Evomi, DataImpulse, Massive, Proxy-Seller, Thordata, Anyip, ABCProxy, ProxyEmpire, RapidProxy, Swiftproxy, 922 S5, LunaProxy, plus Crawlbase, Scrape.do, ScrapingAnt, ScrapeOps, Scrapfly, Oxylabs Web Unblocker and Zyte's proxy mode, hosted browsers (Bright Data Scraping Browser, Browserless, ZenRows, Oxylabs, NodeMaven), and API-only services (Scrappey, ScrapingBot). The full catalog, with credential formats, env names and which ones were checked against a live account: providers.

From env (what the CLI uses): XYTE_PROVIDER=oxylabs plus OXYLABS_USERNAME / OXYLABS_PASSWORD, XYTE_PROXY_URL=http://u:p@host:port, or XYTE_PROXY_TEMPLATE=….

The scraping-API presets (ZenRows, ScrapingBee, ScraperAPI) re-sign TLS, so they set insecure automatically. They bill per request, so in a browser every sub-request of the page is billed. Use them with httpResponseBody.

Mixing providers, Zyte included

No single provider is the cheapest and gets through everywhere. Three combinators mix them behind one client (details and recipes):

import { direct, oxylabs, rotate, route, Xyte, zyteApi } from '@xtandard/xyte';

const residential = oxylabs({ username, password });
const zyte = zyteApi({ apiKey: process.env.ZYTE_API_KEY! });

const xyte = new Xyte({
  proxy: route(
    {
      'wolt.com': direct(),                               // answers anyone: free
      'restaurantguru.com': rotate([residential, zyte]),  // Zyte only after a ban
      'doordash.com': zyte,                               // only Zyte gets through
    },
    residential, // every other site
  ),
});

• rotate([a, b]) picks by attempt: the first try uses a, the retry b, then a again. A fallback after a ban, or when a provider is down. Env: XYTE_PROVIDER=oxylabs,zyte-api.

• route({ host: provider }, fallback) picks by the target's host (subdomains included). Env: XYTE_ROUTES=wolt.com=direct,doordash.com=zyte-api,restaurantguru.com=oxylabs+zyte-api.

• adaptive([direct(), residential, zyte]) learns it instead: per site, the cheapest provider that has been working, moving up after bans and back down when they stop. Env: XYTE_ADAPTIVE=direct,oxylabs,zyte-api.

• byKind({ http, browser }), byCountry({ US: p }, fallback), byIpType({ datacenter, residential }) pick by request kind, geolocation, ipType; weighted([[a, 9], [b, 1]]) splits traffic.

• healthy(p) is a circuit breaker (a provider that keeps failing is skipped for a while instead of costing an attempt per request), budget(p, { max: 500, per: 'hour' }) caps what a provider (say, the Zyte fallback) can spend.

• watch(p, { check }) asks a provider's usage API or status page ahead of requests (ScrapingBee, ZenRows, ScraperAPI, Oxylabs, any Statuspage), so one that ran out of credits or is down is skipped before an attempt is spent on it. Env: XYTE_WATCH=true.

With XYTE_REDIS_URL (or a store), budgets, open circuits and checks are shared by every instance. browserFallback: true retries a banned plain GET in a browser (JavaScript challenges), returning the raw response as Zyte does.

Spending less

• cache: { ttl: 3600 } answers repeats from a cache (per site TTLs, Redis to share it), and identical requests in flight share one run.
• rateLimits: { 'yelp.co.uk': '2/s', '*': '10/s' } paces attempts per site: hammering one site is the fastest way to get banned.
• rates: { oxylabs: { perGB: 6 }, 'zyte-api': { perRequest: 0.0013 } }, your prices, put costUsd on every attempt and report.

Seeing what it did

Every request can tell you what it took, without changing the Zyte response:

new Xyte({ proxy, onRequest: (report) => analytics.insert('scrapes', report) });
// report: outcome, provider that served it, every attempt (provider, route, result, ms,
// bytes on the wire), totals per provider, whether it escalated to a browser.

xyte serve adds X-Xyte-Attempts: oxylabs-residential:ban,zyte-api:ok and X-Xyte-Transfer-Bytes to its responses, and XYTE_LOG_REQUESTS=true logs each report as a JSON line, ready for BigQuery or Datadog. Success rates and MB per provider and site, and what they cost at your rates: observability.

Zyte as a provider. zyteApi({ apiKey }) forwards the request to Zyte unchanged, so every Zyte feature works, AI extraction included, and Zyte's answers (and errors) come back as Zyte sent them. Even with only Zyte behind it, you get the library: typed requests, XyteErrors, retries, a concurrency limit, and the option to move sites to cheaper exits later without touching calling code. zyteProxy({ apiKey }) uses Zyte's proxy mode instead: xyte fetches and renders, exiting through Zyte's IPs (no AI extraction). Env: XYTE_PROVIDER=zyte-api or zyte, with ZYTE_API_KEY.

To find out which exit works for which site, measure it: bun scripts/bench.ts cases.json --providers zyte-api,direct,oxylabs reports, per site and provider, how often the real page came back (a captcha served with a 200 counts as a failure), the median latency and the page size.

Errors and retries

extract() resolves with the Zyte 200 body, or rejects with an XyteError carrying Zyte's status, type, title and detail (error.toJSON() is the problem body the server sends).

| Status | type | When | Retried by xyte | | --- | --- | --- | --- | | 400 | /request/invalid | malformed request (bad url, header syntax), a field xyte doesn't support, an AI field without an extractor | no | | 400 | /request/invalid-json | the body isn't JSON (server) | no | | 401 | /auth/key-not-found | wrong or missing key (server with XYTE_API_KEYS) | no | | 401, 403 | /auth/* | zyteApi() / xyteApi() alone: the upstream's key is wrong or its account suspended, as it answered. In a combinator the next provider gets the request instead | no | | 421 | /website/domain-unreachable | the domain doesn't resolve (direct() only; through a proxy it's a 520) | no | | 422 | /request/unprocessable | incompatible parameters (browserHtml + httpResponseBody, actions on an HTTP request…), or nothing requested | no | | 500 | /server/internal | xyte misconfigured (a provider that can't build its URL, Playwright not installed) | no | | 520 | /download/website-ban | every attempt was a ban | yes, then returned | | 520 | /download/temporary-error | every attempt failed at the proxy or network level | yes, then returned | | 500 | /client/closed | xyte.close() was called while the request was in flight | no |

A target's own 4xx/5xx is not an error: it's a 200 with statusCode set, as on Zyte. Bans are 403, 429, 503, or a known challenge page (Cloudflare, DataDome, PerimeterX, Imperva, Akamai); override with isBan. Each retry goes through a fresh session (a new IP), with exponential backoff; onRetry reports them. With zyteApi(), Zyte's own last problem is returned as Zyte gave it. An account problem from Zyte (a bad key, a suspended account) isn't the request's fault: inside a combinator the next provider gets the request and Zyte isn't tried again for it. error.retryable says whether retrying later makes sense, as with Zyte's own clients (true for 429, 500, 503 and 520, except xyte's own misconfiguration 500s).

How Zyte behaviour maps

| Zyte | xyte | | --- | --- | | geolocation | the provider's country parameter; where the exit can't apply it (direct(), a plain proxy URL), the request runs with an X-Xyte-Warning header / onWarning | | ipType | byIpType() | | session.id | provider sticky session (same exit IP) plus a cookie jar kept across requests | | no session | fresh exit per attempt: redirects and browser sub-requests share it, retries change it | | sessionContext | a pool of sessions per context, initialized by sessionContextParameters.actions, retired after 4 h or 3 bans | | bans | retried on a new exit, then 520 /download/website-ban | | target 4xx/5xx | 200 with statusCode set | | a Cookie custom header | dropped, as Zyte does: cookies go in requestCookies (authenticated pages) | | browser rendering | waits for the network to settle after load, so XHR data lands in browserHtml and networkCapture |

Checked side by side against api.zyte.com with bun run test:live: see migrating from Zyte.

Supported fields

HTTP: httpResponseBody, httpResponseHeaders, httpRequestMethod, httpRequestBody, httpRequestText, customHttpRequestHeaders, requestHeaders.referer, followRedirect, device, requestCookies, responseCookies, cookieManagement, session, geolocation, ipType, echoData, jobId.

Browser (Playwright, or patchright and other forks, or a remote browser): browserHtml, screenshot + screenshotOptions, javascript, viewport, includeIframes, networkCapture (url / resourceType filters, at most 10 captures and 5 MiB), the 60 s browser time budget, and the actions click, doubleClick, hover, hide, evaluate, goto, reload, keyPress, type, select, scrollTo, scrollBottom, waitForNavigation, waitForRequest, waitForResponse, waitForSelector, waitForTimeout (with onError).

Markdown: built in and non-AI like Zyte's, with mainContent / fullPage scopes.

AI extraction (product, article, jobPosting, serp, customAttributes…): answered by zyteApi(), or through any LLM using Zyte's own output schemas, so responses keep Zyte's shape (extraction):

new Xyte({ extractors: llmExtractors({ complete: openAICompatible({ baseURL, apiKey, model }) }) });

Without either, those fields get a 400.

Sessions: session.id (client-managed: sticky exit + cookie jar) and sessionContext + sessionContextParameters (server-managed pool).

Zyte's own actions (searchKeyword, setLocation, interaction) rely on Zyte's per-site knowledge and hosted scripts. With zyteApi() Zyte runs them; elsewhere they run through handlers in actions (a built-in search-box heuristic for searchKeyword, yours for the rest), which always run locally. Without a handler, unsupportedActions decides: skip with a warning (default), 'error', or 'zyte' to hand them to zyteApi() in a combinator. Where every action runs, provider by provider: actions. Beyond /v1/extract: xyte's server also answers Zyte's proxy mode (--proxy-port) and its Stats API (/api/stats, /api/v1/stats); Zyte's Search API isn't implemented. Field-by-field status: coverage.

Status and versioning

xyte is early: it works and is tested (unit, integration with a real browser, Node.js 22/24 smoke tests, live checks against real sites and against Zyte), but it's young and its API can still change. Versions follow ZeroVer: 0.MINOR.PATCH, with no 1.0 planned. A minor (0.2.0) may break the API; a patch (0.1.2) doesn't. Pin with ^0.x.y and read the changelog before moving to a new minor. How releases work: docs/releases.md.

Disclaimer

xyte is a personal, non-commercial project. Its author built it for their own scraping and publishes it in case it's useful to others. It isn't a company or a product, and it isn't meant to compete with or replace Zyte or any other service it works with.

• Not affiliated. Zyte and Zyte API are trademarks of Zyte Group Ltd. Oxylabs, Bright Data, Decodo, IPRoyal, ZenRows, ScrapingBee, ScraperAPI, Browserless, OpenAI and the other names in this repository are trademarks of their respective owners. They appear only to describe compatibility and integrations. xyte is not affiliated with, endorsed by or sponsored by any of them, and links to providers are not affiliate links.
• Compatibility is best effort. It follows Zyte's public documentation and OpenAPI document (see NOTICE), and was checked against the live API, but it can differ from Zyte's service, and Zyte can change. docs/coverage.md lists what is and isn't implemented.
• Provided as is. The software comes "as is", without warranty of any kind, as the MIT licence states. You use it at your own risk, and you're responsible for how you deploy and operate it: security, availability, and costs.
• No commitments. There's no support agreement, no SLA and no guaranteed roadmap. Fixes, features and releases happen on a best-effort basis (SUPPORT.md). Under ZeroVer any minor release may include breaking changes.
• Your scraping is your responsibility. You're responsible for what you fetch and what you do with it: the laws that apply (computer misuse, copyright and database rights, data protection such as the GDPR when pages contain personal data), the websites' terms of service and robots.txt, and not overloading the sites you visit. xyte doesn't check any of that for you. Only access content you're allowed to access.
• Anti-bot features. Stealth browsers, captcha solvers and ban retries exist to match what Zyte offers and to keep a legitimate flow working. Getting past a site's access controls can break its terms or the law where you are. Whether you may use them on a given site is your call and your responsibility.
• Third-party services. Proxy networks, scraping APIs, remote browsers, LLMs and captcha solvers are used with your own accounts, under their own terms and prices. The costs xyte reports are estimates from published prices and your configuration: your provider's bill is the truth.
• Your credentials and data stay with you. xyte runs where you run it and has no telemetry. It contacts only the sites you request and the services you configure, plus ipinfo.io when you run xyte doctor (--url changes it).

Contributing, security and licence

Issues and pull requests are welcome. CONTRIBUTING.md covers the setup, the checks CI runs and the commit style. Report vulnerabilities privately, as SECURITY.md explains, never in a public issue. Questions go where SUPPORT.md says. Everyone taking part follows the Code of Conduct.

The code is MIT licensed (LICENSE). NOTICE credits the third-party material in the repository, including Zyte's OpenAPI document.

Development

bun install
bun test             # unit + integration (local target site, local proxy, real Chromium + patchright)
bun run test:node    # build, then run the dist on Node.js
bun run test:live    # real public sites through a local `xyte serve` (network required)
bun run typecheck
bun run build        # tsdown → dist/
bun run gen:types    # regenerate src/zyte.gen.ts from spec/zyte-openapi.json (and the coverage doc)
bun run docs:coverage  # regenerate docs/coverage.md from scripts/coverage.ts
bun run build && bun run docs:api  # regenerate docs/api.md from the doc comments in src/ (CI checks it)
bun scripts/bench.ts cases.json --providers zyte-api,direct,oxylabs   # compare providers on your sites
bun run release      # bump, CHANGELOG, tag, push → npm (see docs/releases.md)
bun run site:dev     # the docs site (VitePress over docs/), at localhost:5173/xyte/

test:live takes XYTE_LIVE_URL (a deployed xyte, or https://api.zyte.com/v1/extract to run the same checks against Zyte), XYTE_LIVE_PROVIDER and XYTE_LIVE_HTTP_ONLY=true; credentials come from .env.

spec/zyte-openapi.json is Zyte's OpenAPI document, taken from the API reference. One upstream typo is patched: ProductNavigation.required was nested under properties.