isotropic-backoff

npm version License

An escalating retry delay policy that tracks failures, waits the appropriate amount of time between attempts, and knows when to give up.

Why Use This?

• Escalating Delays: Retry immediately a few times, then wait progressively longer between attempts
• Configurable Levels: Describe an entire retry strategy as a simple array of levels, each with its own count, delay, growth, and cap
• Knows When To Quit: When every level is exhausted, the backoff reports that it has given up instead of retrying forever
• Explicit Outcomes: Report succeeded() or failed() from your own event flow; the backoff never has to guess whether an operation worked
• Promise Based: await attempt() to continue at the right time, or let run() manage the whole retry loop
• Cancelable: Abort a pending attempt with an AbortSignal, a timeout, or cancel()
• Observable: Publishes preventable delayChange and giveUp events through isotropic-pubsub
• Optional Jitter: Randomize waits per level to avoid thundering herds
• Extendable: Subclass and override a single method to implement a custom delay strategy
• Clean Shutdown: Disposable via using, settling any pending attempts
• Minimal Dependencies: Builds on isotropic-error, isotropic-later, isotropic-make, and isotropic-pubsub

Installation

npm install isotropic-backoff

How It Works

A backoff instance maintains a current delay and a count of consecutive failures. The delay starts at 0, meaning an attempt may proceed immediately.

The retry strategy is described by an array of levels. Each level covers a number of consecutive failures (count) and describes how the delay behaves while the failures are within that level:

• When a failure enters a level, the level's delay becomes the current delay. If the level does not define a delay, the current delay carries over unchanged.
• Each additional failure within the same level multiplies the current delay by factor and adds increment, clamped to the level's maximumDelay.
• When a level's count is spent, the next failure moves to the next level. A failure beyond the final level causes the backoff to give up: it publishes a giveUp event and becomes exhausted.

Every failure is reported with failed(), which advances through the levels and updates the delay. Every success is reported with succeeded(), which resets the backoff to its initial state, ready for future failures. attempt() returns a promise that resolves after the current delay, so callers wait exactly as long as the strategy prescribes.

The default levels allow one immediate retry, then eight escalating retries with Fibonacci backoff, and then give up:

[{
    count: 1
}, {
    count: 8,
    delay: 987,
    factor: (1 + Math.sqrt(5)) / 2
}]

This produces delays of approximately 0, 987, 1597, 2584, 4181, 6765, 10946, 17711, 28657 allowing for nine consecutive failures within around 74 seconds, and gives up before the tenth attempt.

Usage

Running a Retry Loop

When one function both performs the operation and observes its outcome, run() manages the entire loop: it waits out the delay before each call, reports the outcome for you, and resolves with the callback function's return value once it succeeds.

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff();

    try {
        const response = await backoff.run(async () => {
            const response = await fetch('https://example.com/api/resource');

            if (!response.ok) {
                throw new Error(`HTTP ${response.status}`);
            }

            return response;
        });

        // Success within the allowed attempts.
    } catch (error) {
        // error.name === 'ExhaustedError'
        // error.error is the error from the final attempt.
    }
}

Running a Supervision Loop

In more complex setups, the failed/succeeded outcome of an attempt may be reported from elsewhere.

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff();

    subscribeToAFailureEvent(() => {
        backoff.failed();

        if (backoff.exhausted) {
            // Too many consecutive failures; stop retrying.
        } else {
            // Wait for the current delay.
            await backoff.attempt();

            restartTheThing();
        }
    });

    subscribeToASuccessEvent(() => {
        backoff.succeeded();
    });

    startTheThing();
}

Defining Custom Levels

Levels describe the whole strategy declaratively. This configuration retries once immediately, then five times starting at 100ms and adding 250ms per failure, then indefinitely at a steady thirty seconds:

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff({
        levels: [{
            count: 1
        }, {
            count: 5,
            delay: 100,
            increment: 250
        }, {
            count: Infinity,
            delay: 30000
        }]
    });

    // Delays per consecutive failure:
    // 0, 100, 350, 600, 850, 1100, 30000, 30000, 30000, ...
}

A final level with count: Infinity never gives up. A level without a delay carries the current delay into the level, which allows a level to continue growing (or holding) whatever delay the previous level reached:

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff({
        levels: [{
            count: 3,
            delay: 1000,
            factor: 2
        }, {
            count: Infinity,
            factor: 1.5,
            maximumDelay: 120000
        }]
    });

    // 1000, 2000, 4000 in the first level, then the second level continues
    // from 4000: 6000, 9000, 13500, ... capped at 120000, forever.
}

Adding Jitter

When many clients share a failing resource, synchronized retries arrive in waves. A level's jitter randomizes each wait downward by up to the given fraction of the current delay. The stored delay stays deterministic; only the actual wait is randomized.

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff({
        levels: [{
            count: 10,
            delay: 1000,
            factor: 2,
            jitter: .5,
            maximumDelay: 60000
        }]
    });

    backoff.failed();

    // The delay is 1000, but this resolves after somewhere between 500ms
    // and 1000ms.
    await backoff.attempt();
}

Canceling an Attempt

A pending attempt() can be canceled three ways: an AbortSignal passed to attempt(), a timeout, or the cancel() method on the backoff. A canceled attempt rejects rather than resolving.

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff();

    backoff.failed().failed().failed();

    // Give up on this particular attempt if the delay outlasts 500ms.
    try {
        await backoff.attempt({
            timeout: 500
        });

        retryTheOperation();
    } catch (error) {
        // error.name === 'TimeoutError'
    }
}

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff(),
        controller = new AbortController();

    backoff.failed().failed().failed();

    const promise = backoff.attempt({
        signal: controller.signal
    });

    // Later, abort the attempt from anywhere that holds the controller.
    controller.abort();

    try {
        await promise;
    } catch (error) {
        // error.name === 'AbortError'
    }
}

Observing the Backoff

A backoff instance publishes attempt, cancel, delayChange, failed, giveUp, reset, and succeeded events. All are standard isotropic-pubsub events, so they can be observed and even prevented.

import _Backoff from 'isotropic-backoff';

{
    const backoff = _Backoff();

    backoff.on('delayChange', ({
        data: {
            failureCount,
            newValue,
            oldValue
        }
    }) => {
        console.log(`Delay ${oldValue} -> ${newValue} after ${failureCount} failures`);
    });

    backoff.on('giveUp', ({
        data: {
            failureCount
        }
    }) => {
        console.log(`Giving up after ${failureCount} failures`);
    });
}

Releasing a Backoff

A backoff with pending attempts keeps internal timers running. Disposing the backoff cancels those timers and rejects any pending attempts with a DisposedError. The using declaration disposes automatically at the end of the scope:

import _Backoff from 'isotropic-backoff';

{
    using backoff = _Backoff();

    backoff.failed();

    await backoff.attempt();

    retryTheOperation();

    // backoff is disposed automatically here.
}

API

backoff(options)

Creates a backoff instance. Invalid options throw an error whose name is 'ArgumentError'.

Parameters

• options (Object, optional): Configuration object.
  • levels (Array, optional): A non-empty array of level objects describing the retry strategy. Defaults to two immediate retries followed by eight retries doubling from one second and capped at one minute. Each level accepts:
    • count (Number, optional): The number of consecutive failures this level covers. Must be a positive integer or Infinity. Defaults to 1.
    • delay (Number, optional): The delay in milliseconds applied when a failure enters this level. Must be a finite non-negative number. When omitted, the current delay carries over unchanged.
    • factor (Number, optional): The multiplier applied to the delay for each additional failure within this level. Must be a finite non-negative number. Defaults to 1.
    • increment (Number, optional): The amount in milliseconds added to the delay for each additional failure within this level, after factor is applied. Must be a finite number; a negative increment shrinks the delay, clamped at 0. Defaults to 0.
    • jitter (Number, optional): The fraction of the delay that may be randomly subtracted from each individual wait. Must be a number between 0 and 1. Defaults to 0.
    • maximumDelay (Number, optional): The maximum delay in milliseconds for this level. Must be a non-negative number or Infinity. Defaults to Infinity.

Returns

• (Object): A backoff instance.

backoff.failed()

Records a failure. Advances through the levels, updating the current delay by publishing a delayChange event, or publishes a giveUp event when the failure count exceeds the final level. Does nothing when the backoff is already exhausted.

Returns

• (Object): The backoff instance, so calls can be chained.

backoff.succeeded()

Records a success. Resets the failure count and delay to their initial state and clears the exhausted state, publishing a delayChange event when the delay actually changes. An alias of reset().

Returns

• (Object): The backoff instance, so calls can be chained.

backoff.reset()

Resets the failure count and delay to their initial state and clears the exhausted state, publishing a delayChange event when the delay actually changes.

Returns

• (Object): The backoff instance, so calls can be chained.

backoff.attempt(options)

Returns a promise that resolves with the backoff instance after the current delay (with the current level's jitter applied). Resolves immediately when the effective delay is 0.

Rejects with an ExhaustedError when the backoff has given up, an AbortError when the given signal aborts, a TimeoutError when the timeout elapses first, a CanceledError when cancel() is called without a reason, or a DisposedError when the backoff is disposed while the attempt is pending.

Parameters

• options (Object, optional): Configuration object.
  • signal (AbortSignal, optional): A signal that cancels the pending attempt when aborted.
  • timeout (Number | Temporal.Duration, optional): The maximum time to wait before rejecting the pending attempt. A number is interpreted as milliseconds.

Returns

• (Promise): Resolves with the backoff instance when the delay has elapsed.

backoff.run(config, callbackFunction)

Executes callbackFunction repeatedly until it succeeds or the backoff gives up. Before each execution, waits out the current delay via attempt(). A resolved callback function reports succeeded() and resolves the run with the callback function's return value. A rejected callback function reports failed() and either retries or, once exhausted, rejects with an ExhaustedError whose error property is the error from the final attempt.

Parameters

• config (Object, optional): Configuration object.
  • signal (AbortSignal, optional): A signal that cancels a pending delay when aborted.
  • timeout (Number | Temporal.Duration, optional): The maximum time to wait for each individual delay. A number is interpreted as milliseconds.
• callbackFunction (Function): A function executed for each attempt. Receives an object with:
  • attemptNumber (Number): The current attempt number, starting at 1.
  • signal (AbortSignal): The signal given to run, if any.

Returns

• (Promise): Resolves with the callback function's return value upon success.

backoff.cancel(options)

Cancels all pending attempts.

Parameters

• options (Object, optional): Configuration object.
  • reason (Error, optional): The rejection reason. Defaults to an error whose name is 'CanceledError'.
  • signal (AbortSignal, optional): When given, the pending attempts are canceled when the signal aborts instead of immediately.

Returns

• (Object): The backoff instance, so calls can be chained.

backoff.ref()

Requires the process to stay alive while attempts are pending. This is the default. Applies to pending timers immediately.

Returns

• (Object): The backoff instance, so calls can be chained.

backoff.unref()

Allows the process to exit while attempts are pending. Applies to pending timers immediately.

Returns

• (Object): The backoff instance, so calls can be chained.

backoff.hasRef()

Returns

• (Boolean): true when pending attempts keep the process alive.

backoff.delay

• (Number): The current delay in milliseconds, before jitter.

backoff.exhausted

• (Boolean): true when the backoff has given up.

backoff.failureCount

• (Number): The number of consecutive failures recorded since the last reset.

backoff.level

• (Number): The index of the current level.

backoff.levelFailureCount

• (Number): The number of consecutive failures recorded within the current level.

backoff.levels

• (Array): The normalized, frozen levels configuration.

Events

Both events flow through the standard isotropic-pubsub stages, so before handlers can observe and prevent them. Preventing an event prevents the corresponding state change.

delayChange

Published when the current delay changes, either from a failure or from a reset. The state change is applied by the event's complete function.

Data

• delay (Number): The new delay.
• failureCount (Number): The new consecutive failure count.
• level (Number): The new level index.
• levelFailureCount (Number): The new failure count within the level.
• previousDelay (Number): The delay before the change.

giveUp

Published when a failure exceeds the final level. The exhausted state is applied by the event's complete function.

Data

• failureCount (Number): The consecutive failure count including the failure that caused the give up.

Examples

Supervising Worker Restarts

A process supervisor replaces crashed workers. A worker that crashes on startup would otherwise restart in a tight loop forever; a backoff paces the restarts and eventually gives up. The outcomes arrive as separate events, which is exactly what explicit succeeded() and failed() reporting is for.

import _Backoff from 'isotropic-backoff';

const _restartBackoff = _Backoff();

_supervisor.on('workerReady', () => {
    // The worker proved viable; future crashes start fresh.
    _restartBackoff.succeeded();
});

_supervisor.on('workerCrash', async () => {
    _restartBackoff.failed();

    if (_restartBackoff.exhausted) {
        _supervisor.shutDown();

        return;
    }

    await _restartBackoff.attempt();

    _supervisor.fork();
});

Retrying an Unreliable Request

import _Backoff from 'isotropic-backoff';

const _fetchWithRetry = async url => await _Backoff({
    levels: [{
        count: 5,
        delay: 250,
        factor: 2,
        jitter: .5
    }]
}).run(async () => {
    const response = await fetch(url);

    if (response.status === 503) {
        throw new Error('Service unavailable');
    }

    return response;
});

Pairing With a Throttle

A backoff and a throttle answer different questions: the throttle limits how often operations may start, and the backoff decides how long to wait after failures. They compose naturally at the call site.

import _Backoff from 'isotropic-backoff';
import _throttle from 'isotropic-throttle';

{
    const backoff = _Backoff(),
        throttle = _throttle({
            count: 10,
            duration: 1000
        });

    const result = await backoff.run(async () => {
        await throttle.wait();

        return await _doUnreliableRateLimitedWork();
    });
}

Implementing a Custom Strategy

Subclasses can replace the levels mechanism entirely by overriding _getNextConfig(). It returns the next state after a failure, or null to give up.

import _Backoff from 'isotropic-backoff';
import _make from 'isotropic-make';

const _FibonacciBackoff = _make(_Backoff, {
    _getNextConfig () {
        const failureCount = this._failureCount + 1;

        if (failureCount > 12) {
            return null;
        }

        let a = 0,
            b = 1000;

        for (let index = 1; index < failureCount; index += 1) {
            [
                a,
                b
            ] = [
                b,
                a + b
            ];
        }

        return {
            delay: b,
            failureCount,
            level: 0,
            levelFailureCount: failureCount
        };
    }
});

Contributing

Please refer to CONTRIBUTING.md for contribution guidelines.

Issues

If you encounter any issues, please file them at https://github.com/ibi-group/isotropic-backoff/issues