xresilient

Build regenerative, resumable NodeJS streams

Installing

Via NPM:

npm install xresilient

Usage

// Node CommonJS
const xresilient = require('xresilient');
// Or ES6
import xresilient from 'xresilient';

Examples

const retriableStream = xresilient(
  function returnStream({bytesRead, retryCount}) {
    console.log(`Count = ${retryCount}`);
    return request.get('https://website.com/image.png', {
      headers: {
        Range: `bytes=${bytesRead}-`,
      },
    });
  },
  {retries: 5},
);

retriableStream.pipe(fs.createWriteStream('image.png'));

The above snippet of code would request an image file from the URL and pipe the result into the image.png file

Based on the configuration to retry 5 times, it would recall the function specified with the number of bytes read before the error so you can resume the stream if need be

For instance, get requests interrupted by network errors can be resumed without breaking the active pipe.

API

xresilient(fn[, options])

• fn: <GenFn>
• options extends stream.ReadableOptions: Object
  • retries: <number> Number of times to retry the stream. Default: 5.
  • destroyer: <function> Method with which to destroy the underlying stream.
    • source: <NodeJS.ReadableStream> Readable stream returned by the specified fn function.
• Returns: <ResilientStream>

Return a regenerative, persistent, resuming, resilient stream wrapped That swaps underlying stream source without data loss.

The fn argument must be a function or async function taking two arguments returning a ResilientStream.

Returning an ended readable stream forces the resilient stream to be ended. Returning a destroyed stream forces the resilient stream to retry the generator once until the limit is reached.

Event: 'data'

• chunk: <Buffer>

The 'data' event is emitted whenever the active underlying stream pushes relinquishes ownership of a chunk to the resilient stream which in turn pushes the chunk out to a consumer.

[See stream.Readable (Event: 'data')]

Event: 'end'

The 'end' event is emitted whenever there is no more data within the resilient stream's underlying resources to be consumed.

[See stream.Readable (Event: 'end')]

Event: 'retry'

• retrySlice: <RetrySlice>

The 'retry' event is emitted after an error is caught either within the active stream source or the generative function and the stream hasn't used up all of it's retries.

This event is emitted right before the genFn is called.

Event: 'error'

• err: <Error>

The 'error' event is emitted either when the underlying readable stream encounters an 'error' event or the generative function throws an error while the resilient stream has maxed-out all possible retries. i.e

self.getRetries() === self.getRetryCount()

At this point, the resilient stream is destroyed and the specified GenFn isn't called. This, ends the resilient iteration.

Event: 'readable'

The 'readable' event is emitted when there is data available to be read from the stream or the end of the stream has been reached.

Either way, the retriableStream.read() method would return the available data and reset the flow state so all buffered data flushes.

[See stream.Readable (Event: 'readable')]

GenFn: Function|AsyncFunction

• storeSlice: <ResilientStore>
• Returns: <NodeJS.ReadableStream>

Generator function returning a readable stream. Errors / Promise rejections caught during this function's execution would be emitted through the error event.

This function can either be an async function or a normal function.

ResilientStream extends stream.Readable

The Core resilient stream whose data is streamed off of the underlying streams gotten from the GenFn.

RetrySlice: object

• retryCount: <number> The number of retry iterations so far. (starting from 0)
• maxRetries: <number> The maximum number of retries possible.
• bytesRead: <number> The number of bytes previously read (if any).
• lastErr: <Error> The error emitted either by the last generator function execution or previous stream.
• oldStream: <NodeJS.ReadableStream> The old stream that error-ed out (if any).

ResilientStore extends RetrySlice: object

• trialCount: <number> The active execution iteration count. (starting from 1)

ResilientStream.setRetries(retries)

• retries: <number> The retry value to update to.
• Returns: <boolean>

This method sets the number of possible retries. If the retryCount is less than this value, returns false. else, returns true on successful setting.

ResilientStream.getRetries()

• Returns: <number>

Returns the value of max possible retries before error emit.

ResilientStream.getRetryCount()

• Returns: <number>

Returns the number of the retries so-far.

Development

Building

Feel free to clone, use in adherance to the license and perhaps send pull requests

git clone https://github.com/miraclx/xresilient.git
cd xresilient
npm install
# hack on code
npm run build

License

Apache 2.0 © Miraculous Owonubi (@miraclx) <[email protected]>