supafetch

A lightweight fetch(...) wrapper that integrates optional persistent caching, dynamic rate-limiting, and automatic retries with exponential backoff.

All the features are configurable and optional.

Install

npm i @ozymandiasthegreat/supafetch

If you want persistent caching, there are some additional dependencies depending on your environment.

To enable rocksdb persistent cache for node/bare:

npm i rocksdb-native

Usage

Simple

import supafetch from "@ozymandiasthegreat/supafetch"

const response = await supafetch("https://api.example.com/data")

Full

import supafetch from "@ozymandiasthegreat/supafetch"
import RocksDBProvider from "@ozymandiasthegreat/cache/rocksdb.js"

const provider = new RockDBProvider("<path>")

await provider.ready()

const controller = new AbortController()
const response = await supafetch(
  "https://api.example.com/data",
  { signal: controller.signal },
  {
    cache: {
      provider,
      duration: "1m",
    },
    limits: [
      { requests: 1, duration: "1s" },
      { requests: 5, duration: "1m" },
    ],
    retry: {
      attempts: 5,
      delay: "100ms",
      status: [
        [429, 429],
        [500, 599],
      ],
    },
    debug: true,
    fetch: globalThis.fetch,
  },
)

API

const response = await supafetch(uri, init, options)

response is any JSON compatible object, or if decoding fails, utf-8 string. In case request is aborted, response will be null.

uri is either a string or any object with a toString() method, including URL.

init is RequestInit passed through to underlying fetch implementation. If you pass AbortSignal in init, the request will terminate on abort() immediately, regardless of waiting due to rate-limits or retries.

options: { cache, limits, retry, debug, fetch }

All options members are optional, but their members aren't unless otherwise noted.

All durations/delays can be either positive integers or string like 100ms or 1m.

cache: { provider, duration }

cache is has two-fold use: it caches requests for a given duration if it's provided, and it persists requests timings for persistent rate-limit enforcement. To disable it completely pass cache: { provider: null } to supafetch(...).

provider is an instance of an object implementing CacheProvider interface. @ozymandiasthegreat/supafetch/cache/* contains 3 implementations:

• MemoryProvider which is a simple Map wrapper. It is used by default if no provider is specified, scoped to the module.
• RocksDBProvider utilizing rocksdb-native to provide persistent caching/rate-limiting on node/bare. You must await it's ready() method before passing it to supafetch(...).
• StorageProvider based on Storage interface, it uses LocalStorage by default, but you can pass in SessionStorage in the constructor. Only works in browser-based environments.

You can implement your own CacheProvider as long as it conforms to the following interface:

interface CacheProvider<T extends JsonValue> {
  ready(): Promise<void>
  get(key: string): Promise<T | null>
  set(key: string, value: T): Promise<void>
  delete(key: string): Promise<void>
  clear(): Promise<void>
}

All methods except get and set are optional.

limits: { requests, duration } | { requests, duration }[]

Rate limiting rule or rule array to enforce. Underlying fetch(...) calls will be capped at requests/duration. requests must be a positive integer. If limits is an array, rules will be applied sequentially.

retry: { attempts, delay, status }

Retry failed requests for max attempts. attempts must be a positive integer > 0. delay is the base delay for exponential backoff, set it to 0 to retry immediately.

status is a range or array of ranges of HTTP status codes to retry. Network errors are always retried if attempts is set.

debug

Boolean to enable some additional logging to the console.

fetch

Provide a custom fetch(...) implementation, e.g. bare-fetch. Defaults to globalThis.fetch.

License

Apache-2.0