Syncables

Give an existing API a local-first interface. Syncables is a TypeScript library that joins the pages of API collections into a local copy, lets your application read and edit that copy, and sends local changes back in the background. Your application can work with already-synced data while the API is slow or unreachable. The provider does not need to install Syncables or change its API.

Use it to build an offline-capable editor, a local dashboard over several services, or a data import that follows every declared page. It runs in Node and browsers, with direct HTTP or a custom transport such as integration-proxy. Supply persistent storage when records and pending writes need to survive a restart; the default storage is in memory.

From an API description to a local copy

Syncables builds on three complementary projects:

| Project | What it contributes | | --- | --- | | OpenAPI Directory | Machine-readable descriptions of existing APIs: URLs, operations, parameters and schemas. You can also use a provider's own OpenAPI document or write one for a private API. | | openapi-extensions | Shared vocabulary for the missing behavior: Pagination Schemes describe how to reach the next page; CRUD Causality describes collections, record identities and operation effects. Syncables implements a subset of these specifications. | | overlays | Reusable additions to API descriptions, so collection and pagination metadata can be maintained separately from the provider's document. Dated catalogs pair documents with their overlay revisions. |

The extension versions this source reads, against the specifications in openapi-extensions/spec/:

| Extension | Version | What Syncables reads | | --- | --- | --- | | Pagination Schemes | 0.10.1 | components.paginationSchemes and x-pagination: the pageNumber (with start and shortPage), pageToken, nextLink and rangeWindow types, an offset field's advance, linkResolution, a scheme's response.envelope (with emptyWhenAbsent) its response.incomplete, and a cursor read from the last item (fromLastItem, with hasMore and a checked order) (below). Auto-detection by query parameters, body fields, and (§6.3, when the scheme turns them on) response headers and response fields as the operation's description declares them. Not incrementalSync. | | CRUD Causality | 0.7.0 | components.crudResources: collections with their fixed reads (listMethod, listQuery, listBody) and envelope, identities (optional for a list-only resource, one that no x-crud operation or Reference Object addresses; it is read-only, §4.1) and bindings, and x-crud operations for writes, with created identities (§4.3.2) and compound creates (followUps, §4.7), Reference Objects resolved to a path (resolveReference, §4.1.3), and 0.7.0's replace and unkeyedLists through exported helpers (§4.8, §4.9; createApiClient does not use them yet). Not its request, patch or mint semantics, nor 0.8.0's committedErrors. | | Throttling | 0.5.1-draft | The root x-throttling (headers, bodyFields, signals with blocked, limits with plan tiers, applies) and an operation's bucket list. Requests are paced under the limits that state requests, or every tier in force. | | Collection Completeness | 0.4.0-draft | x-completeness: absent (only deleted changes what the client does without a read), notFound and gone (what a 404 or 410 of a record's read means, resource-wide; an unrecognised absent or value fails closed to unavailable) and parentAbsent. | | Deletion Feeds | 0.3.0-draft | x-deletion-feed, x-read-tombstone and x-list-tombstone. | | Runtime Schemas | 0.1.0-draft | x-runtime-schema on a CRUD resource, in readCollections (below). Not writes or value conversion. | | Filtering | 0.2.0-draft | x-time-zone on list query parameters, with their x-filter field and operator for the covered spans. Not x-filter or x-collection-scope otherwise. | | Write Preconditions | 0.1.0-draft | x-write-precondition on create, update and delete operations: ifMatch, readVerify and refuseWhen (below). Not on a compound create's follow-up operations. | | Produced Classes | 0.1.1-draft | x-produces on a CRUD resource, as read-only metadata (below). Nothing is fetched or mapped. | | Webhook Deliveries | 0.1.0-draft | The events of x-webhook-deliveries, read by scopedReads only (the webhook inbox consumer). Not signature checks or hook registration, which are the receiver's. | | Webhook Subscriptions | 0.1.0-draft | The consumer routes, through InboxClient and InboxConsumer. The root x-webhook-subscriptions field is not read. |

The other specifications there (the authentication extensions) are not read by Syncables.

These are sources of descriptions and conventions, not three npm packages you must install. Once you have a composed document, Syncables uses it locally; it does not look up a directory or catalog on each request. An API whose document already contains the necessary metadata needs no overlay.

flowchart LR
  Directory[OpenAPI Directory or provider document] --> Document[Composed OpenAPI document]
  Extensions[openapi-extensions specifications] --> Overlays[Provider overlays]
  Overlays --> Document
  Document --> Syncables
  API[Provider API] <--> Syncables
  Syncables <--> Local[Local records and outbox]

To local-firstify an API, describe the collections you want, their stable identities, pagination and supported writes; choose authentication and storage; then point your application at client.list, client.get and the local write methods. Syncables handles collection traversal, page assembly, refreshes and the write queue. "Any API" means an API whose needed operations can be expressed in the supported model: a read-only API gives you a local read-only copy, and custom mutation protocols may need an adapter.

Start with the local-first guide: it includes a runnable offline-edit example, the document/overlay workflow, direct and proxy transports, persistence and write recovery. The API details below serve as a reference. For the CLI/daemon product direction, see the local archive and migration design, which records the PostgreSQL cache, compact collection coverage, durable outbox, device approval and replication plans with their implementation status. This README describes current source, including unreleased changes, which are listed in changes/ until a release folds them into the changelog; the guide explains how to run that source before an npm release.

Usage

For published releases, install with npm install syncables. To build the current source, run these commands from syncables/ with Node 22:

npm ci
npm run build:release

import { loadOpenApiDocument, createMockServer, createApiClient } from 'syncables';

const document = await loadOpenApiDocument('./petstore.yaml');

const server = createMockServer(document);
const { url } = await server.listen();

const client = createApiClient(document, { baseUrl: url, credentialPrefix: false });
await client.sync(); // pulls every discovered resource collection into local storage

const pets = await client.list('/pets'); // a local read, with no API request

The mock server makes it possible to try the workflow without credentials. It uses an in-memory CRUD store and synthetic records generated from the schemas. For a real provider, load its composed document and configure a transport and authentication instead. The mock server uses legacy path-pair discovery and does not implement all metadata-driven behavior of a real provider.

When components.crudResources is present, the client and reader use its collection names, identity bindings and nested collection graph. Without that metadata, the client retains legacy discovery by paired collection and item paths, e.g. /pets and /pets/{petId}; the mock server still uses that legacy model. The browser reader requires CRUD metadata. This is a subset of CRUD Causality, not a complete implementation of its write semantics.

Keeping in sync

sync() can be called on a timer: it conditionally re-fetches using ETag/Last-Modified (or a fallback comparison against the previous sync when a server doesn't support conditional requests) and only touches local storage for items that actually changed.

const handle = client.startPolling({
  intervalMs: 30_000,
  onSync: (result) => console.log('changed:', result.changed),
  onError: (error) => console.error('sync failed:', error),
});

// later
handle.stop();

Wall-clock date-time parameters (x-time-zone)

Some APIs read a date-time query parameter as wall-clock time in a zone the request does not carry (Clockify's time-entry start and end, in the user's profile zone). A parameter that declares x-time-zone (Filtering 0.2.0-draft, openapi-extensions/spec/filtering/) is sent as the spec's client steps say, when a selection gives it a value:

• The value must be an instant with Z or an offset (2026-01-01T00:00:00Z); anything else fails that collection's read.
• The zone is read before the list request: a fixed zone.name, or a GET of zone.operationId with the request's own path values (matched by name), at zone.pointer. One read per zone source and bound path values per readCollections/sync(), through the read's budget (and its storeResponse hook, like any read response). Conversion uses Intl, so it is as accurate as the runtime's time zone data.
• The value sent is the wall-clock digits in that zone, yyyy-MM-ddTHH:mm:ss (whole seconds, truncated), plus suffix.
• The UTC span covered is computed from the digits sent, one span per x-filter field (bounds of different fields are never merged): gte/gt parameters give the lower end, lte/lt the upper one (the tighter one when several bound the same end), with fromInclusive/toInclusive from the operator, and ambiguous picks the offset in a repeated or skipped hour (unspecified: the reading that covers least).
• The rest of the same request's query counts too. A range x-filter parameter without x-time-zone whose value is an instant (Z or an offset) adds its bound to its field's span, merged by the same tighter-bound rule (its instant is in instants). Any other value that narrows the read (an eq x-filter, a parameter without x-filter, a range bound that is not an instant) makes the coverage unknown (otherFilters): the read asked for less than the spans say. Not counted: the pagination scheme's fields that walkPages overwrites or that only size the pages (roles pageSize, offset, page; a pageToken or cursor a selection sets is otherFilters, and so is any of them under a rangeWindow scheme, which sets its window fields instead), and the collection's own fixed listQuery values, which define the collection (a fixed range bound that is an instant still adds to its field's span).
• A zone that cannot be read (an error status, nothing at the pointer, not an IANA name this runtime knows, a zone operation that needs a parameter the request does not have) is not taken as UTC: the UTC digits are sent and each bound covers 14 hours less, so a window of 28 hours or less covers nothing.
• After the read, each operation zone source is read again; if the zone differs or cannot be read, the spans are discarded (zoneChanged). The items are kept. syncables does not read again by itself: the caller decides.
• A rangeWindow read with a range (ranges) whose window fields include x-time-zone query parameters splits the range as instants and sends each window bound to such a parameter as wall-clock digits in its zone (PageWalk.windowValue); the other window fields get the bound as it is. Its coverage is that of the outer range (the range's start in each windowStart parameter, its end in each windowEnd one, with the rest of the query as above): adjacent windows share their bounds, so with an unknown zone only the outer ends narrow by 14 hours. A window field without x-time-zone counts like the rest of the query: with a range x-filter its outer end adds to its field's span, and without one (or as a windowRange field, whose value is not an instant) it gives otherFilters. A windowed read is never complete, so a finished one keeps its spans with reason: 'incomplete'. A window format other than dateTime on such a parameter fails the read. A zoned windowRange parameter (both ends in one value) gives noRangePredicate, with the outer range's value in parameters (the template over both wall-clock ends) and its two instants as an ISO 8601 interval (start/end) in instants.

The result is on CollectionSnapshot.coverage (from readCollections) and on SyncResult.coverage (per collection and bound context):

const { coverage } = await client.sync();
// [{ collection: 'entries', context: {...},
//    parameters: { start: '2026-01-01T01:00:00Z', end: '2026-02-01T01:00:00Z' },
//    instants: { start: '2026-01-01T00:00:00.000Z', end: '2026-02-01T00:00:00.000Z' },
//    zones: { start: 'Europe/Amsterdam', end: 'Europe/Amsterdam' },
//    spans: [{ field: '/timeInterval/start',
//              from: '2026-01-01T00:00:00.000Z', fromInclusive: true,
//              to: '2026-02-01T00:00:00.000Z', toInclusive: false }] }]

parameters holds the values sent (wall-clock digits plus suffix), and instants the instants they were written from. Coverage describes the query, per request: the provider was asked for the collection's items whose fields lie in every span at once. It does not say that every such item is in the local copy (an incomplete or failed read, a record the caller dropped). spans is null when the request covers nothing known; reason is then empty (some field's lower end is not before its upper end), zoneChanged, noRangePredicate (an x-time-zone parameter with a value but no range x-filter), otherFilters (above) or incomplete (the read failed, or finished without being complete, as one ended by a short page whose end is observed or assumed). Only a finished read by range windows over x-time-zone window fields keeps its spans, with reason: 'incomplete'. An open end is left out. A probe read reads no zone and converts nothing. readPlatform reports the same entries, in ReadResult.coverage (CollectionCoverage, as sync() gives them), also for a collection whose read failed, where sync() throws. Only query parameters are read; an x-time-zone in a request body or path is ignored. wallClockParam, instantsOf and coveredSpan are exported (the spec's wall_clock_param, instants_of and covered_span); instantsOf takes the offsets a day either side of the wall-clock time, so two offset changes within two days are not told apart.

Writing

create/update/remove are local-first: they update local storage immediately and return, then apply themselves against the server in the background. Confirmed provider state is separate from pending local intent; refreshes and older write responses replay remaining mutations rather than replacing newer local edits. Updates use the item path's operation that declares CRUD Causality x-crud: { action: update } for the record's resource: when exactly one of its PUT and PATCH does, that one; when both do, the PUT; when neither does (or in a legacy document, without components.crudResources), the PUT, or the PATCH when there is no PUT. The chosen operation's x-crud (§4.3 mode, §4.6 patchFormat) sets the body:

• mode: replace, no x-crud, or a legacy document (no components.crudResources): the last known record with the changes on top, as JSON.
• mode: patch, patchFormat: jsonMergePatch: the changes alone, as an RFC 7396 JSON Merge Patch, with Content-Type: application/merge-patch+json when the operation's requestBody.content declares that media type, else application/json. The local copy follows RFC 7396 too: null removes the field, and a nested object merges into the existing one (on other routes null is a value and a nested object replaces the field). Each queued edit sends only its own changes, in order, also after a restart. mergePatch(target, patch) is exported.
• mode: patch, patchFormat: jsonPatch (RFC 6902): update() throws ("declares its PATCH update with patchFormat jsonPatch (RFC 6902), which this client does not send") before any local change; create and delete are unaffected.
• patchFormat: custom, or mode: patch without patchFormat: the full record, as for replace. Whether the provider accepts that is not checked.

With both declared, the PUT is chosen even when the PATCH is declared as a merge patch, so such a document still sends full records; preferring the merge patch there is a possible follow-up, not done. A PATCH declared with patchFormat: jsonPatch beside an undeclared PUT is followed: update() throws.

A merge-patch edit records the confirmed values of the top-level fields it names, so a remote change anywhere inside a nested object it patches is reported as a WriteConflict on that field, even when the two changes touch different nested members. A refresh is compared with the field as the client's own patches leave it, applied in order (a null matches an absent field), so a refresh that shows the client's own merge patch applied before its response arrived is not a conflict, for that write or a later queued one. A failed merge-patch update loses only the nested members a later settled merge patch sets, and resolveWrite retry strips only those of newer queued merge patches; other routes do this by top-level field.

Each update stores the body format it was made with (updateBody in the outbox). A client restarted on a document that now declares another format for the route does not send or replay the update under the new one: the update, and every write queued before it, becomes failed with a lastError that says so; it stays visible as made, retry throws, and discard drops it. When a create is queued before such an update, the record's writes are kept unrestorable instead (written back unchanged). An update's response is merged over the record it sent (a shallow spread, on merge-patch routes too: a null in a response is stored as a value, not read as a removal), so a provider that answers with only some fields (or only bookkeeping such as updatedAt) does not shrink the confirmed record or revert the edit. The trade-off: a field that the provider removed in that response, rather than omitted, stays in the local copy until the next refresh, and is sent again in later PUT or PATCH bodies until then; a strict provider could reject those, for example after a field rename.

A write the server refuses for good (most 4xx) fails at once, a write whose credentials are refused (401, 403) pauses all writes until the app renews them, and other failures retry with exponential backoff, unlimited by default; set retry.maxAttempts to bound attempts. See Failure classes. Unsettled writes are kept in a durable outbox in the client's storage adapter, so a client built later on the same storage resumes them (see Durable outbox and restarts).

const pet = await client.create('/pets', { name: 'Milo', tag: 'cat' });
// `pet` is already in local storage — the POST to the server is still
// happening (and retrying, if needed) in the background.

client.pendingWrites('/pets'); // writes not yet confirmed by the server

Each entry of pendingWrites() has a state:

| state | Meaning | What the client does | | --- | --- | --- | | pending | Queued, in flight, waiting for a retry, or held for a refresh (awaitingRefresh: true) | Retries automatically; a held update waits for a sync() | | uncertain | A create may or may not have reached the server | Nothing, until resolveWrite | | failed | The server refused it, retries stopped at retry.maxAttempts, or a refresh no longer returned its record (missingRecord, below) | Nothing, until resolveWrite (see below) | | blocked | The server refused the client's credentials for it | Sends no write at all, until authRenewed() |

Entries carry attempts, lastError (for an HTTP failure: the path, the status and up to 200 characters of the response body, whitespace collapsed) and lastStatus (the HTTP status behind lastError, absent when lastError does not describe a response). A failed update whose record a refresh no longer returned also carries missingRecord (deleted, unavailable or unknown).

A new create/update/remove does not drop a failed write. A failed create holds back later writes to the record, like an uncertain one. A failed update or delete does not; later writes go ahead, and once one of them settles, it replaces the fields it set in the failed update. A failed update with no fields left, and a failed delete followed by any settled write, are dropped. Whatever is left stays listed, and visible locally, until resolveWrite.

Failure classes

A write the server answers with a non-2xx status is classified. The defaults (defaultWriteFailureClass, exported) are:

| Response | Class | What the client does | | --- | --- | --- | | 404 or 410 to a delete | satisfied | The record is already gone, so the delete settles as if it had succeeded | | A response that is throttling under the document's x-throttling (below): a matching signal, or any 429 | retry | Sent again no earlier than the earliest retry time the response gives, else after the backoff | | A response a blocked signal matches (Throttling 0.4.0), whatever its status | permanent | Not sent again: the API documents that retrying won't help | | 401 | auth | The write becomes blocked and the client stops sending writes (below); no attempt is counted | | 403 with Retry-After or x-ratelimit-remaining: 0, when the document declares no x-throttling.signals | retry | A rate limit, as GitHub sends it; with signals declared, a 403 is throttling only when a signal matches | | Other 403 | auth | As 401, except for a request sent after authRenewed() before any response showed the renewed credentials accepted (below): then permanent | | 408, 425 | retry | Backoff retry, or the Retry-After delay | | Every other 4xx: 400, 404 or 410 to a create or update, 405, 409, 413, 415, 422, ... | permanent | The write becomes failed at once, after 1 attempt | | Anything else: 5xx, and a 1xx or 3xx the transport passed on | retry | Backoff retry, or the Retry-After delay; a create's 5xx other than 503 becomes uncertain without a usable idempotency key (Uncertain creates) |

Failures without a status keep their earlier handling: a transport that throws (uncertain for a create without a key, otherwise retried), a 2xx with an unusable body (uncertain for a create), and an error before the request was handed to the transport, such as an authenticate adapter that throws (retried with backoff).

retry counts an attempt and sends the write again after the backoff delay (retry.baseDelayMs, doubled per attempt, at most retry.maxDelayMs). A Retry-After header (delay-seconds or an HTTP date) can only lengthen that wait: the delay is the longer of the backoff and the Retry-After, which may exceed retry.maxDelayMs but is cut to retry.maxRetryAfterMs (default 3600000, 1 hour). Retry-After: 0, a date in the past, or a server clock behind the client's therefore waits the backoff, so they cannot cause a tight loop. The cap is a trade-off: a provider that asks for more than an hour is asked again after an hour, which it may answer with another 429; raise the cap to wait as long as such a provider asks, at the cost of a queue that can stay held that long. A value that is neither delay-seconds nor a date (such as 1.5) is ignored. retry.maxAttempts applies to retry failures. The delay is not stored; after a restart the first resend is immediate.

permanent counts an attempt and makes the write failed at once, as if retry.maxAttempts were reached: a failed create holds back the writes queued behind it, a failed update or delete joins the record's failed writes while later writes go ahead, and resolveWrite retries or discards them. Only the write at the head of its record's queue is sent, so a failed write is always older than the record's queued writes.

A 404 or 410 to a delete is the one default that settles a write without a 2xx. If the delete path itself is wrong (a misconfigured document), deletes then settle locally and the next sync() brings the records back; classify such responses as permanent to catch that.

Set classifyWriteFailure to change the classes. It receives the write's type, the HTTP method, status, lower-cased headers, the response body, resource, id, afterRenewal, throttling (the verdict below, when the response is throttling) and signalsDeclared, and returns 'retry', 'permanent', 'auth' or 'satisfied'. A classifier that throws or returns another value gets the default class; satisfied for a create or update counts as permanent.

Throttling

The draft Throttling extension (0.4.0-draft) lets a document say, at the root x-throttling, which response headers report the rate limit (headers: roles limit, remaining, used, reset and retryAfter, the last two with a time unit), which JSON body fields report the same roles (bodyFields, since 0.3.0, keyed by a dot-path such as Todoist's error_extra.retry_after) and which responses are rate-limit refusals (signals: a status list plus a header or JSON-body predicate, with the meaning throttled or quotaExhausted, an optional bucket of limits and minDelaySeconds; since 0.4.0 also blocked, a refusal the API documents as not worth retrying, such as Notion's public_api_request_blocked). A blocked verdict is not throttling: it has no retry time, a read returns the response (and fails on it) without sleeping, and a write is permanent by default and is not sent again. A signal with a meaning this version does not recognise (or none, or one that is not a string) is read as blocked; a blocked signal with a bucket or minDelaySeconds is kept as blocked, with those fields ignored. declaredThrottling reads it and classifyThrottling (both exported) applies it to a response, after the spec's classify(): the first signal that matches decides; without one, a 429 is throttled (with or without a declaration) and nothing else is throttling. The earliest retry time is the later of the retryAfter time and, for quotaExhausted or remaining: 0, the reset time; else the response time plus the signal's minDelaySeconds; else, for quotaExhausted with a bucket whose window is declared, plus that window; else the client's own backoff. An absolute time is measured against both the client's clock and the response's Date header, taking the later, so neither a wrong clock nor a wrong Date makes it earlier; a header value that does not parse in its declared unit is ignored. When no retryAfter header role is declared, the standard Retry-After header is read as RFC 9110 defines it, as before the extension. A body field is read only from a throttling response whose body is JSON: its dot-path's names are matched literally (a name holding a . cannot be addressed; a..b, .a and a. are left out of the declaration), a string is trimmed, a number counts only when it has no fractional part (3.0 reads as 3), and a boolean, null, an array or an object does not parse. For headers and body fields alike, only space, tab, CR and LF are trimmed (a value carrying U+00A0 or another Unicode space does not parse), an integer above 2^53 − 1 does not parse (1e21 in a body, a 400-digit Retry-After), and an HTTP-date parses only in one of RFC 9110's three exact forms (IMF-fixdate, the obsolete RFC 850 form, asctime), never through a lenient date parser (so no ISO date, no UTC for GMT, nothing with another whitespace character or a byte-order mark); the client's own Retry-After reading for other retryable answers, and the Date header, follow the same rules. A role read from both a header and a body field counts once: the later time for retryAfter and reset, and the bucket is exhausted when either remaining is 0. A body retryAfter or reset that gives no wait (0, or a time not after the response) counts as absent, so minDelaySeconds, the bucket window or the backoff still applies; those are fallbacks, not floors.

For a write, a throttling verdict means retry, sent again no earlier than that time and never below the backoff. The time is stored with the queued write (notBefore in the outbox), so a restart or a resolveWrite retry keeps it: a restored or retried write waits it out too. A time further away than retry.maxRetryAfterMs is not cut short: the write becomes failed with lastError naming the time and the cap, since the extension forbids retrying earlier and allows giving up; resolveWrite retry queues it again, and it fails again at once while the time is still that far away. A quotaExhausted answer, to a write or to a read, also holds back every other request counted against the same bucket (the operation's x-throttling selection, else the root applies; every request when the signal names no bucket) until that time, or, when it carries no time and its bucket has no declared window, until the throttled write's own backoff; the paused buckets are stored in the outbox too. A write held that way longer than retry.maxRetryAfterMs fails with lastError "Held until … not sent" rather than waiting unseen. A create answered by a 5xx stays uncertain even when a declared signal matches the response: the extension says such a response may follow partial processing, and that "not applied" is only an inference (Uncertain creates). For a read, the client's transport first waits out a paused bucket of the request's operation (through sleep; a pause longer than limits.timeoutMs stops the read with an error), then the budget waits the earliest retry time of a throttled answer and sends the request again, up to limits.maxRetries times, or stops with "API retry delay exceeds the remaining read time"; a throttled answer without a time is returned to the read as before. Nothing here has been checked against a real provider.

Pacing under announced quotas

Where a Limit Object in x-throttling states requests, the client paces itself, so it does not have to run into the limit first (pieces.md K2). It sends at most paceFraction of requests (default 0.8, rounded down, at least one) in any window of the bucket. Moneybird's 150 per 300 s becomes 120 per 300 s. The buckets a request counts against are its operation's x-throttling selection, else the root applies. An operation whose selection leaves a bucket out (x-throttling: [], say) is not paced by it. Buckets without requests are not paced. Documents without such a Limit Object behave as before.

• Windows. A sliding or unspecified window counts the preceding seconds. A fixed window with an anchor counts from its boundaries. A fixed window without an anchor is paced as sliding. Its phase is unknown, and pacing it as sliding keeps under the quota whatever the phase is.
• Counting. A request counts when it is sent, not when it is answered, so requests in flight count and concurrent write queues cannot overshoot. A request that finds no room waits.
• Reads wait through sleep within their time limit, as for a paused bucket. A wait past limits.timeoutMs, or past what is left of the sync's budget, stops the read with an error before the request is sent.
• Writes are held like a notBefore. Pacing alone never fails a write, however long the hold, and waiting does not count an attempt.
• Only this client's own requests are counted, in memory, so a restart starts the count afresh. A bucket partitioned by sourceIp behind a shared proxy, or shared with other clients of the same user, can still run out. Pacing those across clients is the proxy's job (pieces.md P4). The reaction to a throttled answer described above stays in place.

Turn pacing off with pace: false. paceFraction must be greater than 0 and at most 1.

import { createApiClient, defaultWriteFailureClass } from 'syncables';

const client = createApiClient(doc, {
  classifyWriteFailure: (failure) =>
    // This provider answers 409 while a record is locked for a moment.
    failure.status === 409 ? 'retry' : defaultWriteFailureClass(failure),
});

Plan tiers

A Limit Object may declare tiers instead of requests and window (Throttling 0.5.0): one bucket whose amount and window depend on the account's plan, as Clockify documents (30 requests per hour on FREE, 20 per second on TRIAL, BASIC, STANDARD and PRO, 50 per second on ENTERPRISE), each tier naming its plans. With throttlingPlan set to the account's plan, as the provider names it ('PRO'), the client paces by the tier whose plans include it, as by a plain limit. Without one, or with a plan no tier names, it keeps within every tier at once, each counted separately over the same requests (the spec's limits_in_force): no single tier is safe on its own, since a lower average rate over a longer window (30 per hour) still allows a burst a shorter tier forbids (20 per second). The window a time-less quotaExhausted answer waits out is the plan's tier's, else the longest (fallback_window). A limit whose tiers fail the checks that decide which tier binds (no plans, an empty or repeated plan, a requests or window.seconds that is not a positive integer) is left out, as other entries that do not parse are. The client is more lenient than the spec's validator elsewhere: a window kind it does not know is read as sliding, an anchor that does not parse is left out, and unknown fields, or requests and window beside tiers, are ignored (the tiers are read). throttlingPlan is an option of createApiClient, readCollections, readPlatform and paginate; declaredThrottling(document, plan) gives the tiers in force in tieredQuotas (a plan's tier goes into quotas), and parseTiers, limitsInForce and fallbackWindow are exported. How a client learns its plan is up to the caller. Pacing counts only this client's requests (above).

Refused credentials

An auth failure stops every write of the client, for every record and collection: a client has one server URL and one set of credentials (one transport, credentials and authenticate), so a refusal is taken to apply to all of its writes. The write that met it becomes blocked, without counting an attempt; the other writes stay pending but are not sent, and writes made meanwhile are queued as usual and visible locally. A request already sent completes, and its write is marked blocked too if it is refused. A write whose in-flight mark was still being stored when the block began is not sent and keeps its attempt count. Reads (sync()) are not affected by the block.

const client = createApiClient(doc, {
  authenticate,
  onAuthBlocked: ({ status, lastError, resource, id }) => {
    // ask the user to sign in again, then:
    renewToken().then(() => client.authRenewed());
  },
});
client.authBlocked(); // { status, lastError, resource, id } while blocked

authRenewed() makes the blocked writes pending again and resumes every queue in its order; it does nothing when the client is not blocked. The client cannot tell by itself that credentials were renewed: the authenticate adapter returning a request does not show that the server accepts it. Do not call authRenewed() from onAuthBlocked without renewing: a write refused again blocks the client again, one request per call. After a renewal, a 401 blocks again. A 403 (without rate-limit headers) fails its write instead of blocking, if it answers a request sent after the renewal and before any response showed the renewed credentials accepted, so a permission that new credentials do not grant cannot hold all writes back. Accepted means: a response to a request sent after the latest renewal that is not a refusal, that is a 2xx or a failure classified retry, permanent or satisfied, except a failure that the classifier in use (classifyWriteFailure, or the defaults) calls auth when asked again with afterRenewal: false, such as the 403 that this rule makes permanent. A refusal never counts as acceptance, so several writes the renewed credentials may not make all fail rather than block again. For a failure of a request sent after a renewal that it does not call auth, a custom classifier is therefore called a second time, with afterRenewal: false; one that returns a class other than auth regardless of afterRenewal makes that response count as acceptance. Whether a request counts as sent after the renewal is decided when it is sent, not when its answer arrives. A 401 or 403 for a request sent before the latest renewal is sent again at once, without counting an attempt.

resolveWrite discard drops a blocked write (a blocked create together with the writes queued behind it, as for an uncertain one) and leaves the client blocked; the record's later writes stay queued. retry on a record whose queue starts with a blocked write throws unless the record also has failed or waiting writes: it then retries the failed writes behind the queued ones, or stops the waiting updates from waiting for a refresh; a blocked write itself is resent only by authRenewed().

For an authenticate adapter that renews tokens by itself, set onAuthFailure: 'retry': an auth failure is then retried with backoff like a retry failure (counting attempts, retry.maxAttempts applies), nothing is blocked and onAuthBlocked is not called. A client with 'retry' restored from an outbox stored while blocked drops the stored block and sends the blocked writes as pending ones. The default is 'block'. Under either setting, a create that a classifier calls auth but that may have been applied (a 5xx other than 503) becomes uncertain without a usable idempotency key, instead of being resent.

The block is stored in the durable outbox. A client restored from a blocked outbox is blocked, calls onAuthBlocked once the restore is done, and sends no write before authRenewed(). Whether credentials were just renewed is not stored, so after a restart a 403 blocks again. While blocked, sync() calls do not count towards failing a restored update that waits for a refresh; a complete refresh still releases it, and after authRenewed() it is sent on the newest confirmed record.

Refresh during a pending update

A refresh (sync()) never hides a pending local edit: the visible record is the newest confirmed remote record with the pending changes replayed on top. When a refresh shows that a field with a pending update also changed remotely (its remote value differs from the value the client had confirmed when the edit was made, or when an earlier write to that field settled, and from every value this client has queued, or holds as failed, for that field), the client records a conflict. A refresh that shows an earlier queued edit applied, before or after its response arrives, is not a conflict. For a JSON Merge Patch update, conflicts are per top-level field: a remote change to any nested member of an object the patch touches is reported on the whole field, even when the patch sets other members. Its expected value is the patch applied to its own base, or to what the queued patches before it leave; failed patches are not counted as applied. The local value stays visible, the queued write still sends it (local wins on acknowledgement), and the conflict is observable in two ways:

const client = createApiClient(doc, {
  onConflict: ({ resource, id, field, base, remote, local }) => {
    // called once per newly observed remote value
  },
});
client.pendingWrites('/pets')[0]?.conflicts; // [{ field, base, remote, local, ... }]

The conflict is listed until its write settles or is discarded, and is dropped if a later refresh shows the remote value equal to the local one. To keep the remote value instead, call update again with it. Not covered: conflicts that arrive only in a write's own response, and deletes. A refresh that no longer returns the record at all is handled below.

Records a refresh no longer returns

Presence: tell before prune, and marks. When a complete sync() no longer returns a record that has no unsettled writes, the client either removes it from the local copy or keeps it with a presence mark, by the collection's declaration:

| Collection | The record is | |---|---| | x-completeness: { absent: deleted } | deleted (source: 'declaration'): removed, or kept marked deleted under retainDeleted: true | | x-completeness: { absent: removed }, or wherever a check will be made (missingRecordChecks: 'all', a parent whose nested collection declares parentAbsent) | kept, marked unconfirmed, until a check (a GET of the record) settles it | | no declaration, default checks | removed (no longer returned; whether it still exists is not known): removed |

A check settles an unconfirmed record: a 404/410 under notFound: unavailable marks it unavailable (kept); deleted (a 404/410 under the default notFound, a read tombstone, a feed tombstone) removes it, or keeps it marked deleted under retainDeleted; the record found but not in the list (filtered) removes it from this collection (removed, with the record as that check read it); an error or no check leaves it unconfirmed. The marks persist in the outbox. A retained deleted mark stays until a read returns the record again or the app removes it (a remove() is held and checked like any write to a marked record); the client does not expire or cap them. A read that returns a marked record again drops the mark (present). Checks of records without writes are capped at maxMissingRecordChecks per sync (default 50), one limit shared by all collections, spent in the order the sync reads them (and within a collection, records left unconfirmed by an earlier sync before newly vanished ones); the rest stay unconfirmed, and the next sync checks unconfirmed records first, even when the list did not change. Only a check that makes a request counts. Records with writes are always checked. Under missingRecordChecks: 'none', or without an item GET, nothing is checked, reported or counted, and an absent: removed record stays unconfirmed until a read returns it. client.presence(resource?) lists the current marks (PresenceMark: presence, source, detail, record, and lastSeen: when a read last returned the record, the start of that sync, absent when no read of this client returned it, such as a record kept from storage written before a restart); a record without a mark is present as far as the client knows. The read times are kept in memory for every record read; only a mark stores its lastSeen (in the outbox). A new update() or remove() of a marked record is held for the next refresh, which checks it.

The app is told before anything is removed. onPresence(changes) is called, and awaited, before a record is removed from the local copy, with one PresenceChange per record (collection, context, id, presence, source, record, pruned: true). Marks set or cleared on kept records (pruned: false; present when a mark is dropped) are held and told at the end of the sync. Each record is told once per sync, with its latest change (a record marked unconfirmed and settled in the same sync is told only as settled; a removal replaces a held mark change of the same record). Held changes that are not told (the sync throws, the handler throws at the end, or it threw earlier in the sync, which skips the end call) are kept, the latest per record, and told at the end of a later sync; they are kept in memory only, so a restart loses them though not the marks. A removal the handler refused keeps the record and its mark; an unconfirmed one is checked again by a later sync. A mark dropped because a read returned the record, a write to the record settled, or a check found the record while a write or confirmed copy keeps it, drops its held change too; if the app had been told a mark of the record (or the mark was restored from storage), it is told present instead (not after a delete the app made). The same changes are on SyncResult.presence (absent when there are none). If the handler throws or rejects, the records are not removed (for the read's own removals, that collection's read is not applied: its records do not replace the confirmed copy, though marks the read settles still clear), its held writes do not count a refresh miss, sync() rejects with the error, and a later sync tells again. A write that settles while the handler runs makes the read older than it: its records are then only added or updated and nothing is removed. A record that has a write when the handler returns (queued during the handler, or still being stored) is not removed: its change is listed with pruned: false, although the handler was told pruned: true. A removal after a check, for a record kept earlier (unconfirmed, unavailable) that a check, the deletion feed or its parent now settles, is told the same way, with that source (read, feed, parent). A read that fails or is incomplete removes nothing and tells nothing.

A complete sync() of a collection (every page read, within the read budget) can lack a record that has queued updates. The provider may have deleted it, or the list may just not return it (a default filter, a view that depends on the credentials). The client does not send such an update on the record's last known copy: a PUT built on it could recreate a deleted record on some providers, or write back fields that changed remotely. That applies to PATCH too: unless its operation declares patchFormat: jsonMergePatch, the client's PATCH body also carries the whole last known record with the changes on top, and a PATCH to a deleted record (merge patch included) is a 404 at best and recreates it at worst.

When a complete read lacks the record (and no write to it settled during the read), every queued update of the record up to its first create, apart from one already in flight, is held: it is not sent, and its pendingWrites() entry shows awaitingRefresh: true (a blocked one is held too, shows it once authRenewed() makes it pending, and is checked only then). Updates queued behind a create of the record are not held, since the record is not expected in the list before the create settles. When the first queued write of the record is a held update that is not in flight, the client looks for evidence:

| Evidence | Found by | The held updates | | --- | --- | --- | | deleted | The collection is declared complete with x-completeness: { absent: deleted } (no request is made), a tombstone the collection's deletion feed reported in an earlier sync is stored for it and this sync's feed read does not report it restored (no GET is made), a GET of the record answers 404 or 410 (unless the collection declares notFound: unavailable, next row), or 2xx with the record carrying the resource's read tombstone marker, or the GET did not decide and this sync's feed read has a tombstone for it | Fail, oldest first: state: 'failed', missingRecord: 'deleted', lastError "Record was deleted at the provider (...)", lastStatus the GET's status (none without a GET) | | unavailable | The collection declares x-completeness: { absent: removed, notFound: unavailable } and a GET of the record answers 404 or 410: this caller can no longer read the record, and the API does not say whether it was deleted, moved out of reach, or access was lost (Collection Completeness 0.2.0 §4.3); for a collection with a deletion feed, only once this sync's feed read has no tombstone for it | Fail as for deleted, with missingRecord: 'unavailable' and lastError "Record is unavailable at the provider (...)"; never reported as deleted. Every other write queued for the record fails too, deletes included (a create is parked in place, as a refused create is), so nothing is sent without a decision. The last known values stay, with the edits on top; after a discard the last known values alone stay, marked, until a later read settles the record (see below). A later complete read that returns the record, or a 2xx GET of it when a new update of the record is checked, supersedes the mark for new updates; the failed ones still wait for the decision | | filtered | A GET of the record answers 2xx with a JSON object whose identity field is the record's id, and that is not a read tombstone | Stay pending and are sent on the returned record, which becomes the confirmed copy; a field it changed under an update is a conflict (onConflict), as for any refresh | | unknown | The GET answers any other status, its 2xx body is not that record, it throws, the item path declares no GET, or missingRecordChecks is 'none'; for a collection with a deletion feed, only once this sync's feed read has no tombstone for it | Fail as for deleted, with missingRecord: 'unknown' and lastError "Record is not in the refreshed collection (...)" |

x-completeness is the draft Collection Completeness extension (0.1.0-draft), read from the collection's CRUD Causality definition, which covers its fixed listQuery/listBody (or the older x-list-query/x-list-body), else from its list operation, which counts only for a collection with neither (the operation may serve several collections; this also serves a document without crudResources). A selection that adds or changes a query parameter of the collection makes its reads narrower than the declaration, so its absent is not used for that collection. absent: removed, like no declaration, leads to a GET (unless a stored tombstone of the collection's deletion feed decides first). A wrong deleted declaration makes filtered records count as deleted; no overlay declares one yet, and the behaviour has not been verified against a real provider.

What that GET's 404 or 410 means is the notFound the record's resource states (0.2.0 §4.3), through any of its collections: the one this record was read in, another one, or the operation that lists one (its list path's, or one whose x-crud is a list of that collection). So a record that vanished from a collection with no declaration still follows a notFound another collection of its resource states. When collections state different values, or one the client does not recognise, it is unavailable; when none states it, deleted (the default, as 0.1.0 consumers read it). A notFound stated beside an absent the client does not recognise still counts, though the rest of that declaration is ignored. Since Collection Completeness 0.3.0, a 410 follows the resource's gone instead, when one is stated (notFound: unavailable, gone: deleted for a provider whose 410 means deletion and nothing else), read the same way: resource-wide, conflicting or unrecognised values unavailable; without gone, a 410 means what notFound says. Under an absent the client does not recognise on the record's own collection (§7), the order is the same: a 410 by a stated gone, else a stated notFound; a 404 by a stated notFound; and what neither classifies is unavailable, never the deleted default. unavailable means the record is kept with its last known values and its queued writes fail with missingRecord: 'unavailable', so that nothing is sent, and nothing is reported as deleted, until the app decides (resolveWrite). The record is kept, not pruned, also without writes (a vanished record read under missingRecordChecks: 'all') and after its writes are discarded: its last known values stay visible, marked by the onMissingRecord report and stored in the outbox (unavailable, added within version 1), until a later read settles it (a complete read that returns it, a GET that answers 2xx, or evidence of its deletion, after which the record is removed again; a write to it that the provider accepts also ends the mark). A kept record without writes is read again by later syncs wherever the collection makes checks (the same rule as for a vanished record: missingRecordChecks: 'all', absent: removed, or a parent of a parentAbsent collection; never under 'none'). A kept record with queued or failed writes is not: its writes say what became of it, and a failed one is resolved with resolveWrite. The re-reads come in a second pass, after every collection's unconfirmed records and the records its read newly lacks (those go first within each collection), and share what is left of maxMissingRecordChecks, a limit per sync across all collections. Within that pass the record checked longest ago goes first (one never checked before any other; the time is stored with the mark), so a small limit reads each kept record in turn rather than the same one every sync. The time moves only when a check ran: a record that the request budget (limits.maxRequests, the timeout) or a 429 left unread keeps its place and goes first next time. Each re-read is reported to onMissingRecord again, whatever it finds. A 404 under notFound: unavailable keeps it kept, and tells onPresence nothing again; a 2xx shows it exists outside the collection, so it leaves the copy (told removed); an answer the document concludes as deletion removes it (told deleted). A new update() or remove() of a kept record is held for the next refresh, which checks the record again. A queued delete of a record a complete read no longer returns is held like an update and checked first: released and sent when the read says deleted, present (filtered) or, for a record not kept as unavailable, unknown; failed (held for a decision) when it says unavailable; still held when the record is kept as unavailable and the read is unknown. It still applies when a selection narrows the read, since it says what the record's own read means. A notFound on the list operation counts even for a collection with fixed reads (the safe reading). With absent: deleted, where the spec does not allow it, no GET is made, so it has no effect. A tombstone in the collection's deletion feed is positive evidence of deletion and stands over an unavailable answer. No overlay declares notFound yet; not verified against a real provider.

A nested collection (one whose URL has a path variable that another resource's identity.bindings binds, /lists/{listId}/tasks under task lists) can declare parentAbsent on its Collection Object (0.2.0 §4.4; unavailable is its one value, and any value is read as unavailable; ignored on the list operation, on a collection whose absent the client does not recognise, for a variable a constants entry fixes, and on a collection whose path variables two or more other resources bind, which the spec does not describe). Once a record of the parent collection is concluded gone, by its declaration (absent: deleted), by the GET (404/410 per notFound, or a read tombstone) or by a feed tombstone, the records last read under it in that nested collection are concluded unavailable, never deleted, without a request: deleting a member needs evidence about the member itself (0.2.0 has no deleted cascade). A member is exempt only when this sync's read returned it under the same identity, its parent variable included; the same bare id under another parent is still marked (when in doubt, mark). Each member is reported to onMissingRecord with source: 'parent', its context and its last known values, and its held updates fail with that missingRecord, as above (under unavailable, every write queued for it, as above); a parent whose GET shows it still exists (filtered), or whose fate is unknown, concludes nothing. Nothing is pruned: the records keep their last known values until the parent returns in a complete read and the nested collection is read again. Until then the nested scope is marked: a new update() or remove() of a member is concluded at once like the earlier writes were (failed, not sent on a record the caller cannot read), and a new create() into the scope is parked the same way. The mark is in memory only: after a restart, a member's failed writes still carry missingRecord and hold a new edit, but a member without writes is not marked until the parent is concluded again. So that a queued edit under a vanished parent is not sent blindly, the default missingRecordChecks: 'pending' also checks a vanished parent record without writes of its own when a nested collection declaring parentAbsent has records with unsettled writes under it. A record concluded through parentAbsent is not itself taken as a gone parent for collections nested under it. No overlay declares parentAbsent yet; not verified against a real provider.

The GET uses the record's item path, the client's transport and authentication, the conditional-request cache and storeResponse, and counts against the same budget as the sync's read (limits: requests, time and 429 retries). It runs after all collections are read, so it gets what the read left. Records the budget does not cover are not checked in that sync; their updates stay held and the next sync checks them. A 429 counts as the budget being spent too: one whose Retry-After reaches past the read's deadline, and any 429 the budget hands back (it waits out a usable Retry-After up to limits.maxRetries times, and returns a 429 without one at once). A held update that is the first queued write of its record fails after three syncs that did not release it, with lastError "Waiting for a complete refresh", as a restored update does (below).

A record whose first queued write is a delete, with no update ahead of it, is not checked: the delete is sent, and a 404 or 410 settles it (satisfied, see Failure classes).

Only the first queued write of a record is failed, and only when no earlier write of the record is in flight. An update already in flight when the read shows its record missing is left to its response (a 404 makes it failed, see Failure classes). If that response leaves it queued (a retry answer such as a 503, or blocked), it is held too and is not resent before a sync checks the record. The held updates behind it wait for that response and then for the next sync, which checks the record if it is still missing; they are not sent when the in-flight one settles. Updates held behind another queued write (a delete, say) likewise wait until they are first. This keeps the order the client relies on: a failed write is never newer than a queued write of the same record. (A settled write takes its fields out of older failed writes, so a failed write newer than it would lose its edit; an earlier attempt at this feature did that.)

Nothing is deleted locally because of the evidence: the failed updates stay visible, as failed updates do, and records without writes are pruned from the visible copy by a complete read exactly as before. What a deleted record means for the app is its decision. resolveWrite retry sends the failed updates on the record's last confirmed copy (a PUT may then recreate a deleted record; that is the caller's choice), and discard drops them. A new update() of such a record, while its failed writes carry missingRecord and no refresh, GET or write response has confirmed the record since, is held too and checked by the next sync.

const client = createApiClient(doc, {
  onMissingRecord: ({ resource, id, evidence, source, status, record }) => {
    // evidence: 'deleted' | 'filtered' | 'unavailable' | 'unknown'
    // source: 'declaration' | 'feed' | 'read' | 'parent' | 'none'
  },
  missingRecordChecks: 'pending', // default; or 'all', or 'none'
});

onMissingRecord is called for every record checked, with the GET's status (when a GET answered) and, for filtered, the returned record; for unavailable, record holds the last known values when the client has them (the copy confirmed before the read, or the last known copy of the record's writes). missingRecordChecks: 'all' also checks records without unsettled writes: those the client had confirmed and that a read drops from its confirmed copy, once, in that sync (a record the budget did not cover is not checked later); the report changes nothing in the client. 'none' makes no GET and reads no deletion feed: without an x-completeness declaration the evidence is unknown, so the updates fail. (Before this, an update made in the running client was sent on the last known copy.)

Deletion feeds

A collection can declare the operation that reports its deletions, with the draft Deletion Feeds extension (0.2.0-draft): x-deletion-feed on its CRUD Causality definition, else on its list operation.

collections:
  pets:
    urlTemplate: /pets
    x-deletion-feed:
      operationId: listPetChanges # a GET operation of the document
      envelope: { itemsField: changes } # default: the body is the array
      cursor:
        parameter: since # the query parameter the cursor is sent in
        responseField: next # where the last page holds the next cursor
        expiredStatuses: [410]
      tombstone: { field: state, values: [deleted] } # default: every item is one
      idField: id # default: the collection's identity field

After a complete read of a collection, sync() reads its feed once per bound context (a collection under /owners/{ownerId}/pets has one feed read per owner). It does so in every such sync, whether or not a record is missing, so that the cursor stays recent. Every feed is read at the end of the sync, after all collection reads and all GETs of missing records, from the budget (limits) they left, so a feed read never takes a request those GETs need. The feed is not read when the collection's x-completeness: { absent: deleted } applies, which settles every missing record without a request, or with missingRecordChecks: 'none'.

The read binds the feed operation's path parameters from the collection's bound context, sends cursor.parameter with the stored cursor if there is one, and follows every page as for a collection read (Pagination Schemes). The items are the array at envelope.itemsField. Each feed read counts its own items against limits.maxRecords, from 0: more makes it incomplete (below). A sync therefore reads at most limits.maxRecords collection records plus limits.maxRecords items per feed read. An item is about the record whose identity is the value at idField; items that are not objects, or have no such value, are skipped. For each record, its last item in the read decides: it is a tombstone when the value at tombstone.field equals one of tombstone.values (same JSON type).

For a record the complete read lacked, in a collection with a feed, the client looks in this order:

1. a tombstone stored from an earlier sync's feed read (below): no GET; the record is deleted (source: 'feed') at the end of the sync, unless that sync's feed read completes with a later item about it that is not a tombstone (restored),