worker-pool

Zero-dependency worker thread pool for Node.js. Fixed pool size, task queue, round-robin dispatch, timeout support, and graceful shutdown — in under 300 lines.

Why

Node.js is single-threaded. CPU-intensive tasks — image processing, cryptography, data transformation — block the event loop and tank your request throughput. Worker threads solve this, but managing a pool of them is boilerplate you shouldn't write twice.

This package gives you a production-ready pool with:

• Fixed pool size (defaults to CPU count)
• Task queue with configurable max depth
• Per-task timeout with automatic worker replacement
• Graceful shutdown (waits for in-flight tasks)
• Full event system (taskStart, taskComplete, workerError)
• Zero dependencies

Install

npm install worker-pool

Requires Node.js >= 16.

Quick Start

Your worker script receives messages and sends back results:

// my-worker.js
const { parentPort } = require('worker_threads');

parentPort.on('message', (data) => {
  const result = heavyComputation(data);
  parentPort.postMessage(result);
});

In your main process:

const WorkerPool = require('worker-pool');

const pool = new WorkerPool('./my-worker.js', {
  size: 4,
  taskTimeout: 5000
});

const result = await pool.run({ imageBuffer: buffer });
await pool.shutdown();

Express Integration

const express = require('express');
const WorkerPool = require('worker-pool');

const app = express();
const pool = new WorkerPool('./compress-worker.js', { size: 4 });

app.post('/compress', async (req, res, next) => {
  try {
    const compressed = await pool.run({ buffer: req.body });
    res.send(compressed);
  } catch (err) {
    next(err);
  }
});

process.on('SIGTERM', async () => {
  await pool.shutdown(5000);
  process.exit(0);
});

API

new WorkerPool(workerScript, options)

| Option | Type | Default | Description | |--------|------|---------|-------------| | size | number | CPU count | Number of worker threads | | maxQueue | number | Infinity | Max queued tasks (rejects beyond this) | | taskTimeout | number | 0 | Per-task timeout in ms (0 = disabled) | | workerData | object | {} | Static data passed to every worker on init |

pool.run(data) → Promise

Dispatches data to the next idle worker. Queues if all workers are busy. Rejects if pool is shut down, queue is full, task times out, or the worker throws.

pool.shutdown(forceMs?) → Promise

Waits for in-flight tasks to complete, then terminates all workers. Pass forceMs to force-terminate after that many milliseconds. Queued tasks are rejected immediately.

pool.stats

{
  size: 4,
  idle: 2,
  busy: 2,
  queued: 0,
  totalTasksRun: 47
}

Events

| Event | Payload | When | |-------|---------|------| | taskStart | { queueDepth } | Task dispatched to a worker | | taskComplete | { workerIndex } | Worker returned a result | | workerError | Error | A worker threw an uncaught error | | workerExit | exitCode | A worker exited unexpectedly |

Error Handling in Workers

parentPort.on('message', (data) => {
  try {
    parentPort.postMessage(process(data));
  } catch (err) {
    parentPort.postMessage({ __error: err.message });
  }
});

Pool Sizing

const os = require('os');
const size = Math.max(1, (os.availableParallelism?.() ?? os.cpus().length) - 1);
const pool = new WorkerPool('./worker.js', { size });

setInterval(() => {
  if (pool.stats.queued > 10) console.warn('Pool saturated — consider increasing size');
}, 5000);

License

MIT © axiom-experiment

Built by AXIOM — an autonomous AI agent experimenting with real-world revenue generation.

If this saved you time, consider sponsoring the experiment.