gearman-stream

gearman-stream is a stream that serves as a wrapper around a gearman client, allowing you to stream the payload and any worker data.

Install

npm install gearman-stream

Usage

gearman-stream supports taking the payload in two ways:

1. As a constructor option, in which case it's a Readable stream

2. From streaming data, in which case it's a Transform stream

Examples

Readable

var GearmanStream = require('gearman-stream');

var gearmanOpts = {host: 'localhost', port: 4730};
var streamOpts = {highWaterMark: 10};
var payload = JSON.stringify([0, 1, 2, 3, 4, 5]);

// Stream that writes 10, 11, 12, 13, 14, 15
var add10Worker = new GearmanStream(streamOpts, gearmanOpts, 'add10', payload);

Transform

Note: The implementation or ArrayStream is left as an exercise to the reader.

var GearmanStream = require('gearman-stream');
var JSONStream = require('JSONStream');

var gearmanOpts = {host: 'localhost', port: 4730};
var streamOpts = {highWaterMark: 10};

var add10Worker = new GearmanStream(streamOpts, gearmanOpts, 'add10');
var readable = new ArrayStream([0, 1, 2, 3, 4, 5]);
var stringifyArray = JSONStream.stringify('[', ',', ']');
readable.pipe(stringifyArray).pipe(add10Worker); // 10, 11, 12, 13, 14, 15

All together

var GearmanStream = require('gearman-stream');
var JSONStream = require('JSONStream');

var gearmanOpts = {host: 'localhost', port: 4730};
var streamOpts = {highWaterMark: 10};
var payload = JSON.stringify([0, 1, 2, 3, 4, 5]);

var add10Worker = new GearmanStream(streamOpts, gearmanOpts, 'add10', payload);
var sumWorker = new GearmanStream(streamOpts, gearmanOpts, 'sum');
var stringifyArray = JSONStream.stringify('[', ',', ']');
add10Worker.pipe(stringifyArray).pipe(sumWorker) // 75

Testing

You must have the Gearman daemon running.

gearmand -d
make test