isotropic-request

One function, one config object, one HTTP request.

Why Use This?

• Single Function Interface: Execute one function with one config object and get a response with the parsed body already on it. No method shortcuts to memorize, no chained format calls, no double await.
• Instance Defaults: Create requester instances with a default config (base headers, formats, retries, timeouts, even a default URL) merged into every request
• Built-In Throttling: Define rate limits once and every matching request automatically waits for an available slot, powered by isotropic-throttle
• Retry With Backoff: Opt into automatic retries of failed requests with configurable escalating delays, powered by isotropic-backoff
• Observable Lifecycle: Requests publish request, response, and error events through isotropic-pubsub, with full before/on/after stages and the ability to prevent requests
• Symmetric Format Config: requestFormat serializes the request body and responseFormat parses the response body
• Invalid Certificates: Set rejectUnauthorized: false to communicate with that one third-party API that will never fix its TLS certificate (Node.js only)
• Cancelable: The returned promise has a cancel() method, and AbortSignal and timeout are supported throughout
• Isomorphic Core: The baseline functionality uses the standard fetch available in Node.js and browsers; Node-specific modules are only loaded when their features are used
• Extensible: Create instances, subclasses, and custom formats through isotropic-make and isotropic-property-chainer

Installation

npm install isotropic-request

Usage

import _request from 'isotropic-request';

{
    const response = await _request({
        url: 'https://api.example.com/things'
    });

    // The body is already parsed based upon the response content type.
    console.log(response.status, response.result);
}

Everything is a config property. There are no method shortcuts and no format methods.

import _request from 'isotropic-request';

{
    const response = await _request({
        body: {
            name: 'Widget'
        },
        headers: {
            authorization: 'Bearer abc123'
        },
        method: 'POST',
        query: {
            confirm: 'true'
        },
        url: 'https://api.example.com/things'
    });

    console.log(response.result);
}

The Response Object

The resolved response is a web standard Response object. You usually don't need to await response.json() as a separate step. The formatted response value is available on the result property.

• headers (Headers): The response headers.
• ok (Boolean): Whether the status code indicates success.
• redirected (Boolean): Whether the request was redirected.
• result (Any): The parsed response body, according to responseFormat.
• status (Number): The response status code.
• statusText (String): The response status text.
• url (String): The final url after any redirects.

Throttling

Many APIs impose strict rate limits. Define the limit once and the requester automatically waits until the appropriate time to send each request. Throttle definitions accept the same count, duration, and durationBuffer options as isotropic-throttle, plus an optional match.

import _request from 'isotropic-request';

// At most 30 requests per minute to api.example.com, applied automatically
// to every request with a matching url.
_request.defineThrottle('exampleApi', {
    count: 30,
    duration: 60000,
    match: 'https://api.example.com/'
});

{
    // No extra config needed. If the limit has been reached, this waits
    // for an available slot before sending.
    const response = await _request({
        url: 'https://api.example.com/things'
    });
}

match may be:

• a string: applied when the request url begins with the string
• a regular expression: applied when the request url matches
• a function: called with the normalized request config, where config.url is a URL instance; applied when it returns a truthy value

A throttle defined without match only applies when a request names it with the throttles config property. This supports APIs with different rate limit buckets for different groups of endpoints:

import _request from 'isotropic-request';

_request.defineThrottle('searchBucket', {
    count: 10,
    duration: 60000
});
_request.defineThrottle('generalBucket', {
    count: 100,
    duration: 60000
});

{
    // This endpoint consumes from both buckets.
    const response = await _request({
        throttles: [
            'generalBucket',
            'searchBucket'
        ],
        url: 'https://api.example.com/search?q=widgets'
    });
}

Multiple applicable throttles are deduplicated and awaited sequentially. Throttles are awaited before every attempt, so retried requests consume a rate limit slot for each attempt. _request.getThrottle(name) returns the underlying isotropic-throttle instance for advanced usage such as mark()ing consumption that happened elsewhere. _request.removeThrottle(name) removes a definition.

Retry With Backoff

Set the retry config property to automatically retry failed requests using isotropic-backoff.

import _request from 'isotropic-request';

{
    const response = await _request({
        retry: true, // Use the default backoff levels.
        url: 'https://api.example.com/things'
    });
}

retry may be:

• true: retry with the default isotropic-backoff levels
• a positive integer n: retry up to n times with escalating delays
• an object with optional properties:
  • levels (Array): isotropic-backoff levels controlling the number of retries and the delays between them
  • methods (Array): The request methods eligible for retry. Defaults to ['DELETE', 'GET', 'HEAD', 'OPTIONS', 'PUT', 'TRACE'].
  • statusCodes (Array): The response status codes that trigger a retry. Defaults to [408, 413, 429, 500, 502, 503, 504].

Network errors and attempt timeouts are also retried when the method is eligible. When retries are exhausted, the request settles with its final outcome: the last error response is processed normally (rejecting with HttpError by default, or resolving when throwHttpErrors is false), or the last network error is rethrown. Cancellation, abort, total timeout, and configuration errors are never retried. ReadableStream bodies can only be consumed once, so they are not retryable.

Events

Instead of hooks, requests publish events through isotropic-pubsub. Subscribe on the requester to observe every request, or subscribe through the config object to observe a single request.

| Event | When | Data | |---|---|---| | request | Before each attempt is sent | attemptNumber, config, previousError, previousResponse, requestSignal | | response | After each attempt receives a response | attemptNumber, config, response | | error | After the request settles with an error | attemptNumber, config, error |

The config data property is the normalized request config object. config.symbol is a unique value across all events for a given request.

import _request from 'isotropic-request';

// Observe every request made through the default requester.
_request.requester.on('response', event => {
    console.log(`${event.data.response.status} ${event.data.response.url}`);
});

// Modify requests before they are sent. Before-stage subscribers may mutate
// event.data.config, or call event.prevent() to reject the request with a
// PreventedError.
_request.requester.before('request', event => {
    event.data.config.headers.set('x-request-time', `${Date.now()}`);
});

// Prevent retries conditionally. Preventing the request event rejects the
// request with a PreventedError.
_request.requester.before('request', event => {
    if (event.data.attemptNumber > 1 && event.data.previousResponse?.status === 500) {
        event.prevent();
    }
});

Per-request subscriptions are passed through the config object and only observe events belonging to that request:

import _request from 'isotropic-request';

{
    const response = await _request({
        subscribe: {
            request: {
                callbackFunction (event) {
                    console.log(`Attempt ${event.data.attemptNumber}`);
                },
                stageName: 'before' // Defaults to 'on'. Also accepts once: true.
            }
        },
        url: 'https://api.example.com/things'
    });
}

subscribe is a key/value object where the keys are event names and the values are subscription configs. The subscription config can be a callback function or it can be an object with the following properties:

• callbackFunction (Function, required): The function to execute when the event is published
• once (Boolean, optional): Defaults to false
• stageName (String, optional): Defaults to 'on'

Request Formats

The requestFormat config property controls how the body is serialized before sending:

• 'auto' (default): Selects a request format based upon the request body. Plain objects and arrays switch to 'json' and everything else switches to 'none'.
• 'json': The body is stringified with JSON.stringify and the content type defaults to application/json.
• 'none': The body is passed through unchanged. Use for strings, Blob, FormData, URLSearchParams, typed arrays, and ReadableStream bodies.
• 'text': The body is converted to a string. (Not necessary when the body is already a string.)
• 'urlencoded': The body is converted to URLSearchParams.

Response Formats

The responseFormat config property controls how response.result is parsed:

• 'arrayBuffer': The corresponding web standard Response body method.
• 'auto' (default): Selects a format based upon the response. Bodiless responses (204, 205, 304, or an explicit zero content length) yield null. JSON content types use 'json', XML content types use 'xml', other text content types use 'text', and everything else uses 'bytes'.
• 'blob': The corresponding web standard Response body method.
• 'bytes': The corresponding web standard Response body method.
• 'formData': The corresponding web standard Response body method.
• 'json': The corresponding web standard Response body method.
• 'response': The raw Response object without consuming the body.
• 'stream': The response body as a ReadableStream.
• 'text': The corresponding web standard Response body method.
• 'xml': The response text parsed into an XmlDocument by @rgrove/parse-xml, a small, strict, dependency-free XML parser. The parser module is loaded the first time the format is used.

import _request from 'isotropic-request';

{
    const response = await _request({
        responseFormat: 'xml',
        url: 'https://api.example.com/feed'
    });

    console.log(response.result.root.name);
    console.log(response.result.root.attributes);
    console.log(response.result.root.children);
}

When a successful response body can not be parsed, the request rejects with a ParseError.

Invalid TLS Certificates

Sometimes a required API is hosted with a self-signed or otherwise invalid TLS certificate and the provider is never going to fix it. The standard fetch interface always fails in this situation. Set rejectUnauthorized: false to proceed anyway.

import _request from 'isotropic-request';

{
    const response = await _request({
        rejectUnauthorized: false,
        url: 'https://badcert.example.com/api/things'
    });
}

This works in Node.js only. When it applies, the request is transported by a Node-specific module (built upon node:https) that is dynamically imported on first use, so it costs nothing in the browser or when unused. The Node transport follows redirects and decompresses br, gzip, deflate, and zstd response bodies to match fetch behavior. In a browser, the option has no effect and certificate validation is enforced by the browser itself.

Cancellation, Abort, and Timeout

The returned promise has a cancel() method like isotropic-later:

import _request from 'isotropic-request';

{
    const promise = _request({
        url: 'https://api.example.com/things'
    });

    // Cancel directly. The promise rejects with a CanceledError, or the
    // given reason.
    promise.cancel();

    // Or defer cancellation to an AbortSignal.
    promise.cancel({
        signal: someAbortController.signal
    });
}

The config object also accepts:

• attemptTimeout (Number | Temporal.Duration): Time allowed per attempt. A number is interpreted as milliseconds. The timer restarts with each attempt. When combined with retry, timed out attempts are retried. The promise rejects with a TimeoutError when the final attempt times out.
• signal (AbortSignal): Aborts the request when the signal aborts. The promise rejects with an AbortError.
• timeout (Number | Temporal.Duration): Time allowed for the entire request process, including throttle waits, every attempt, and the delays between retries. A number is interpreted as milliseconds. The promise rejects with a TimeoutError.

The two timeouts can be combined; for example, allow each attempt 5 seconds while capping the entire retried process at 30 seconds:

import _request from 'isotropic-request';

{
    const response = await _request({
        attemptTimeout: 5000,
        retry: true,
        timeout: 30000,
        url: 'https://api.example.com/things'
    });
}

Configuration Reference

The default export is a function that accepts a single config object:

• attemptTimeout (Number | Temporal.Duration, optional): Time allowed per attempt.
• body (Any, optional): The request body, serialized according to requestFormat.
• fetchOptions (Object, optional): Additional options passed through to the underlying Request, such as cache, credentials, or mode.
• headers (Object | Headers | Array, optional): The request headers.
• method (String, optional): The request method. Defaults to 'GET'.
• query (Object | URLSearchParams | Array, optional): Parameters appended to the url search params.
• redirect (String, optional): 'error', 'follow', or 'manual'. Defaults to 'follow'.
• rejectUnauthorized (Boolean, optional): Set false to ignore invalid TLS certificates in Node.js. Defaults to true.
• requestFormat (String, optional): The request body serialization format. Defaults to 'auto'.
• responseFormat (String, optional): The response body parse format. Defaults to 'auto'.
• retry (Boolean | Number | Object, optional): The retry configuration.
• signal (AbortSignal, optional): An abort signal.
• subscribe (Object, optional): Per-request event subscriptions.
• throttle (String | Symbol, optional): Named throttle to wait upon.
• throttles (Iterable, optional): Same as throttle but allows more than one.
• throwHttpErrors (Boolean, optional): Set false to resolve error status responses instead of rejecting with an HttpError. Defaults to true.
• timeout (Number | Temporal.Duration, optional): Time allowed for the entire request process.
• url (String | URL, required): The request url.

The default export function also has these properties:

• defineThrottle(name, config): Defines a named throttle on the default requester. Chainable.
• getThrottle(name): Returns the named isotropic-throttle instance or null.
• removeThrottle(name): Removes a named throttle. Chainable.
• requester: The default Requester instance backing the function.
• Requester: The Requester constructor for advanced usage.

Errors

All rejections are isotropic-error instances with descriptive names:

• AbortError: The config signal aborted. The signal reason is nested as the inner error.
• ArgumentError: The config was invalid.
• CanceledError: The promise's cancel() method was called.
• DestroyedError: The requester was destroyed while the request was pending.
• HttpError: The response status indicated an error and throwHttpErrors was not disabled. The error has a response property with the full response object, including the parsed result when the body was parseable.
• ParseError: A successful response body could not be parsed as the requested format.
• PreventedError: A subscriber prevented the request event.
• RequestError: The request failed at the network level.
• TimeoutError: The request exceeded the configured timeout or the final attempt exceeded the configured attemptTimeout.

Advanced Usage

Requester Instances

The default export uses a shared default Requester instance. Create separate instances for isolated throttles, subscriptions, and default configs. The constructor optionally accepts initial throttle definitions and a default config.

import _request from 'isotropic-request';

{
    const requester = _request.Requester({
            defaultConfig: {
                headers: {
                    authorization: `Bearer ${apiToken}`
                },
                responseFormat: 'json',
                retry: true,
                throttle: 'exampleApi',
                timeout: 30000
            },
            throttleByName: {
                exampleApi: {
                    count: 30,
                    duration: 60000,
                    match: 'https://api.example.com/'
                }
            }
        }),
        response = await requester.request({
            url: 'https://api.example.com/things'
        });
}

• defaultConfig (Object, optional): A config merged into every request config. Any property from the configuration reference is allowed, including url (a request may then omit its own url entirely). The merge is shallow: a property given at request time completely replaces the same default property, with one exception - headers are merged per header name, with the request's headers winning on conflicts.
• throttleByName (Object, optional): Initial throttle definitions, equivalent to calling defineThrottle.

The Requester constructor is also directly importable from isotropic-request/lib/requester.js.

A requester is an isotropic-pubsub instance, so it supports the full event lifecycle, distributors, and destroy(). Destroying a requester cancels its pending requests and throttle waits.

Custom Formats

Request and response formats are defined as static properties chained with isotropic-property-chainer, so subclasses can add formats while inheriting all of the built-in ones. Format functions are called with the requester as this.

import _make from 'isotropic-make';
import _Requester from 'isotropic-request/lib/requester.js';

const _CsvRequester = _make('CsvRequester', _Requester, {}, {
    _requestFormatByName: {
        csv ({
            body,
            headers
        }) {
            if (!headers.has('content-type')) {
                headers.set('content-type', 'text/csv');
            }

            return body.map(row => row.join(',')).join('\n');
        }
    },
    _responseFormatByName: {
        async csv ({
            response
        }) {
            return (await response.text()).split('\n').map(line => line.split(','));
        }
    }
});

{
    const requester = _CsvRequester(),
        response = await requester.request({
            responseFormat: 'csv',
            url: 'https://api.example.com/report.csv'
        });

    console.log(response.result); // An array of arrays.
}

A response format function receives { response } with the raw Response object and returns the parsed result. A request format function receives { body, headers }, may set headers, and returns the serialized body. The built-in xml response format is implemented exactly this way, wrapping @rgrove/parse-xml.

External Rate Limit Accounting

When something else consumes from the same rate limit bucket, keep the local throttle in sync with mark():

import _request from 'isotropic-request';

_request.defineThrottle('exampleApi', {
    count: 30,
    duration: 60000,
    match: 'https://api.example.com/'
});

// Some other system made a request against the same limit.
_request.getThrottle('exampleApi').mark();

Tests

npm test

Contributing

Please refer to CONTRIBUTING.md for contribution guidelines.

Issues

If you encounter any issues, please file them at https://github.com/ibi-group/isotropic-request/issues