Burstable

Simple and powerful task queue for Node.js on top of Beanstalkd

Features

• Wait for job completion
• Child jobs (keeping parent alive till done)

How-To

Setup

import Burstable from 'burstable';

const burstable = new Burstable(
  host, // beanstalkd host
  port, // beanstalkd port
  {
    // Will receive job errors and job log calls
    log: function(level, err|message, meta) {
      winston.log(level, message.message || message, meta); // meta will be enhanced with tube and jobId
    }
  }
);

Spawning jobs

burstable.spawn(tube, {
  delay: 0,
  priority: 1000,
  timeout: 10 * 60 * 1000,
  payload: {
    // job payload/values
  },
  wait: Job | [Job...] | [{tube: String, id: Number}] // Wait for jobs before handling this job - Logic on consumer side
});

Handling jobs

burstable.handle(tube, function (payload) {
  // Complete job
  return Promise.resolve();

  // Job error
  return Promise.reject();

  // Spawn a job
  return this.spawn(someTub);

  // Spawn child job and wait for completion before completing this job
  return this.child(anotherTube);

  // Puts current job back in queue with delay, does not affect retries counter
  return this.delay(5000); // ms, default: original timeout

  // Keep someone updated
  this.log('info', 'doing the thing');
}, {
  maxTries: 3, // Total amount of tries including the first one
  backoff: {
    initial: 60 * 1000, // ms
    exponential: 1.5 // multiple backup by N each try
  },
  labels: function(payload) {
    // Will be added to log.meta
    return {
      accountId: payload.accountId
    };
  },
  // Will receive job errors and job log calls
  log: function(level, err|message, meta) {
    winston.log(level, message.message || message, meta); /// meta will be enhanced with jobId
  }
});

burstable.start(); // Enable handlers and start processing jobs, make sure handlers are setup before calling start

Keep in mind that burstable will spawn a connection equal to width * amount of tubes. You'll want to make sure that your server is configured to handle that amount of connections (ulimit).

Debugging

Use DEBUG=burstable* to enable verbose debugging.