drumset
v4.0.25
Published
This is a library specifically designed for implementing sagas in Node.js applications
Downloads
664
Maintainers
Readme
Drumset
The drumset package provides a robust and user-friendly framework for managing complex workflows using the saga pattern.
This pattern is particularly useful for handling distributed transactions by defining a sequence of steps (nodes)
and the logic for compensating (undoing) steps when failures occur. By defining saga's nodes and their relationships,
you can ensure reliable and consistent transaction processing, even in the presence of failures.
npm i -s drumset
Requires Node.js 18 or newer. The package has no runtime dependencies.
Creates a new instance of the Saga class.
import { Saga } from 'drumset';
const saga = new Saga({
verbose: true,
logger: console,
storage, // optional, see "Persistence"
});Options:
verbose(boolean, defaultfalse): log scaling, shutdown and finished facts.logger(object withlog, defaultconsole).storage(FactStorage, optional): saves unfinished facts on shutdown and loads them inrestore().handleSignals(boolean, defaulttruewhenstorageis set): save on SIGTERM / SIGINT / SIGHUP and when the event loop drains.autosaveInterval(number, ms, default off): save unfinished facts tostorageon this interval, see Persistence.shutdownTimeout(number, ms, default10000): how longshutdown()waits for running handlers before saving anyway.Infinitywaits for as long as they take.
Saga instance methods:
addNode(name, handler, meta, scaling): void
Adds a node to the saga.
- Parameters:
name(string): The name of the node.handler(function): An asynchronous function that performs the node's operation. The function receives the following arguments:facts(object): The current state of the saga.next(function): A function to call the next node.exit(function): A function to complete the saga or terminate it with an error e.g. exit(new Error('somethingWentWrong')), if if you reach your pivot point you can exit with nodes which you want to run after the result returning. e.g. exit(['node1', 'node2']).retry(function): A function to retry processing of the facts on the current node.error(Error, optional): An error that caused first.
A handler moves the fact on with
next,exitorretrybefore the promise it returns resolves. A handler that just returns is done (this is how rollback, compensation and after-exit nodes end). If that leaves the fact with no node queued, running, retrying or delayed andexitwas never called, the fact fails withFact <id> stalled after node '<node>': no node is active and exit was not calledand its rollbacks run, instead of staying pending forever. A node runs once per fact: routing to a node that already ran, or that is queued or running, does not queue it again (two parallel nodes may both callnext('D')). An error thrown afternext/exit/retrywas called is recorded and logged, and does not retry the node.meta(object, optional): Additional processing options for the node.retriesLimit(number, default 1): How many times the node is retried after its first attempt (so2means up to 3 runs).timeoutBetweenRetries(number): The time in milliseconds to wait between retries. Both apply to rollback, compensation and after-exit nodes too: they are retried after the fact has its result. Other nodes still running when the fact exits are not retried.timeout(number): How long one attempt of the handler may take, in milliseconds. A slower attempt fails with a timeout error and is retried like any failure; what it calls afterwards is ignored. Without it there is no limit: a handler that never settles keeps its replica busy, and withmaxNodes: 1no other fact gets through the node.rollbackWhenErrorNode(string): The name of the node to call for compensation if this node fails. It runs when the node throws or callsretrypastretriesLimit(the fact goes on from the compensation node, which decides how it ends), and when the node callsexit(error)(the fact has failed already; the compensation node runs after that).rollbackWhenSuccessNode(string): The name of the node to call for rollback if this node succeeded but something went wrong in the execution chain after. Rollbacks of all succeeded nodes start together when the fact fails or expires (they are not ordered). A node still running at that moment is rolled back as soon as it succeeds.runAfterNodesSucceed(string[]): An array of node names that must succeed before this node will start.ttl,expireAfter,executeAfter(number): Defaults for facts started at this node, used whenprocess()doesn't pass them (seeprocess).expireAfterandexecuteAfterare absolute timestamps, so they are fixed whenaddNoderuns;ttlis relative to each fact.
scaling(object, optional): Additional processing options for the node:minNodes: The min count of concurrent nodes that use one queue.maxNodes: The max count of concurrent nodes that use one queue. This determines how many instances of this node can run concurrently.queueSizeScalingThreshold: threshold of the queue size to run the horizontal scaling- Each option is a whole number from 1 and may be left out: the defaults are
minNodes: 1,maxNodes: 1(orminNodeswhen only that is given) andqueueSizeScalingThreshold: 100.minNodesabovemaxNodes, other values and unknown keys are refused.
The configuration is checked. addNode throws for a name that is already added, a name every object has
(constructor, toString, ...), node links of the wrong type and invalid scaling. When the first fact starts
(process or restore), every node named by rollbackWhenErrorNode, rollbackWhenSuccessNode,
runAfterNodesSucceed or addMiddleware has to exist, in whatever order the nodes were added: otherwise
process throws Saga refers to nodes that were not added: ....
addMiddleware(names, handlers): void
- Parameters:
names(string): An array of names of the nodes to which middlewares should be applied.handler(function): An array of the asynchronous functions, each function receives the following arguments:facts(object): The current state of the saga.next(function): A function to call the next node.exit(function): A function to complete the saga or terminate it with an error.
process(startNode, facts, factsMeta): Promise<facts>
Starts the saga from the specified node.
Every failure comes through the returned promise, wrong arguments included (unknown start node, empty facts, invalid
factsMeta, a saga that is shut down): process() never throws synchronously.
- Parameters:
startNode(string, required): The name of the node to start the saga.facts(object, required): The initial state of the saga.factsMeta(object, optional): Initial meta specific for current factsexecuteAfter(number): The timestamp in milliseconds after which fact processing should start.expireAfter(number): The timestamp in milliseconds when facts should be expired and execution chain should be stopped.ttl(number): Time to live in milliseconds from now (the earlier ofttlandexpireAfterwins).- An expired fact fails with status
expiredat its expiry time, whatever it is doing then (waiting for a retry or a delay, or with a handler still running: that handler'snextis ignored). The rollbacks of the nodes that had succeeded still run. retriesLimit(number): Retries for the start node, and the default for nodes that don't set their own.- Each of
ttl,expireAfterandexecuteAfterthat isn't passed is taken from the start node'saddNodemeta.retriesLimitisn't: the start node already uses its ownretriesLimitwhen none is passed.
- Returns:
Promise: Resolves with the final state (the same contract you defined for the facts argument) of the saga or rejects with an error.
Example Usage:
const { Saga } = require('drumset');
const calls = [];
const facts = {
balance: 9,
amount: 10,
to: 'a',
from: 'b',
};
const saga = new Saga();
saga.addNode(
'A',
async (facts, next, exit, retry) => {
console.log(`A executed`);
calls.push('A');
next([
'B',
'C',
]);
},
{
retriesLimit: 1,
rollbackWhenErrorNode: 'CompensateA',
rollbackWhenSuccessNode: 'RollbackA',
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 10,
},
);
saga.addNode(
'CompensateA',
async (facts, next, exit, retry) => {
console.log(`CompensateA executed`);
calls.push('CompensateA');
next('Error');
},
{
retriesLimit: 1,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'RollbackA',
async (facts, next, exit, retry) => {
console.log(`RollbackA executed`);
calls.push('RollbackA');
},
{
retriesLimit: 1,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'B',
async (facts, next, exit, retry) => {
await new Promise((resolve) => setTimeout(resolve, Math.round(Math.random() * 1000)));
console.log(`B executed`);
calls.push('B');
// throw new Error('WTF something went wrong');
next('D');
},
{
retriesLimit: 2,
rollbackWhenErrorNode: 'CompensateB',
rollbackWhenSuccessNode: 'RollbackB',
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'CompensateB',
async (facts, next, exit, retry) => {
console.log(`CompensateB executed`);
calls.push('CompensateB');
next('Error');
},
{
retriesLimit: 1,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'RollbackB',
async (facts, next, exit, retry) => {
console.log(`RollbackBexecuted`);
calls.push('RollbackB');
},
{
retriesLimit: 1,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'C',
async (facts, next, exit, retry) => {
console.log(`C executed`);
// Move an error throwing to another node to check result
throw new Error('stop');
calls.push('C');
next('D');
},
{
retriesLimit: 1,
rollbackWhenErrorNode: 'CompensateC',
rollbackWhenSuccessNode: 'RollbackC',
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'D',
async (facts, next, exit, retry) => {
console.log(`D executed`);
calls.push('D');
next('Success');
},
{
retriesLimit: 1,
rollbackWhenErrorNode: 'CompensateD',
rollbackWhenSuccessNode: 'RollbackD',
runAfterNodesSucceed: [
'B',
'C',
],
},
{
minNodes: 2,
maxNodes: 4,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'CompensateC',
async (facts, next, exit, retry) => {
console.log(`CompensateC executed`);
calls.push('CompensateC');
next('Error');
},
{
retriesLimit: 3,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'CompensateD',
async (facts, next, exit, retry) => {
console.log(`CompensateD executed`);
calls.push('CompensateD');
next('Error');
},
{
retriesLimit: 3,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'RollbackC',
async (facts, next, exit, retry) => {
await new Promise((resolve) => setTimeout(resolve, Math.round(Math.random() * 2000)));
console.log(`RollbackC executed`);
calls.push('RollbackC');
},
{
retriesLimit: 1,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'RollbackD',
async (facts, next, exit, retry) => {
await new Promise((resolve) => setTimeout(resolve, Math.round(Math.random() * 2000)));
console.log(`RollbackD executed`);
calls.push('RollbackD');
},
{
retriesLimit: 1,
},
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
saga.addNode(
'Error',
async (facts, next, exit, retry, error) => {
console.log(`Error executed`);
calls.push('Error');
exit(error);
},
{
retriesLimit: 2,
timeoutBetweenRetries: 1000,
},
{
minNodes: 1,
maxNodes: 5,
queueSizeScalingThreshold: 10,
},
);
saga.addNode(
'Success',
async (facts, next, exit) => {
console.log(`Success executed`);
calls.push('Success');
exit();
},
{ retriesLimit: 1 },
{
minNodes: 1,
maxNodes: 2,
queueSizeScalingThreshold: 5,
},
);
setTimeout(() => {
// Take a look behind the scenes to check all running nodes
console.log(calls);
}, 10000);
saga
.process('A', facts)
.then((result) => {
console.log({ result });
})
.catch((error) => console.log({ error }))
.finally(() => {
// Take a look what nodes were running when you receive reply
console.log(calls);
});Facts
Every call of process creates a fact: { id, meta, data }. data is your object; meta describes
the execution and is plain JSON:
import { Fact, FactMeta, NodeExecution } from 'drumset';
type Fact<DataType, NodeName extends string> = {
id: string;
meta: FactMeta<NodeName>;
data: DataType;
};meta holds:
status:pending,running,completed,failedorexpiredstartNode,currentNode(started most recently),activeNodes(queued, running, waiting or retrying now),path(order nodes first ran in)createdAt,updatedAt,finishedAt,durationMs,ttl,expiresAt,executeAfter,maxRetries,errornodes: per nodestatus,attempts,retries/maxRetries,retryDelay,enqueuedAt,startedAt,finishedAt,durationMs(last attempt),totalDurationMs,nextAttemptAt,waitingForand every error with its attempt and timeflow: the saga's control state (processed, failed, rollbacks, compensators, after-exit nodes) used to resume a fact
Finished facts are emitted as events:
saga.on('completed', (fact) => console.log(fact.meta.durationMs));
saga.on('failed', (fact, error) => console.log(fact.meta.nodes));completed and failed fire when the result is known, which is before rollback, compensation and after-exit nodes
run. settled fires when nothing of the fact is running any more, with their outcome in fact.meta.nodes:
saga.on('settled', (fact) => {
const failedRollbacks = Object.values(fact.meta.nodes).filter((node) => node.status === 'failed');
});A rollback, compensation or after-exit node that fails is also logged (logger.log), since the fact already has its result.
The process() promise settles before the listeners run. A listener that throws is logged (logger.log) and changes nothing for the fact.
Persistence
Give the saga a storage with two async methods. Where the facts go is up to you:
const fs = require('fs/promises');
const storage = {
async load() {
return JSON.parse(await fs.readFile('facts.json', 'utf8').catch(() => '[]'));
},
async save(facts) {
await fs.writeFile('facts.json', JSON.stringify(facts));
},
};
const saga = new Saga({ storage });
// ...addNode calls...
await saga.restore(); // resumes facts saved by the previous processsaga.shutdown()stops taking new work, waits for running handlers (up toshutdownTimeout) and callsstorage.savewith every unfinished fact. Pending retries and delays are saved with theirnextAttemptAtand resumed on time.- Without
autosaveInterval, facts are saved only by a shutdown. A process that ends any other way (crash,SIGKILL, out of memory,process.exit()) loses the facts in flight, and the storage still holds what the shutdown before it saved, whichrestore()would run again. autosaveIntervalsaves the unfinished facts on that interval while there are any (and once more when the last one is done, so finished facts leave the storage). After a crashrestore()resumes from the last save: a node that finished since then runs again, so handlers have to be idempotent, as they do for a shutdown. A failed autosave is logged and tried again at the next interval.saga.save()does the same on demand and resolves with the number of facts saved:await saga.save()after a step you cannot afford to lose. Saves of one saga never overlap.storage.savegets the whole list each time, so pick an interval your storage can take with the number of facts you keep in flight.Saga.terminate()shuts down every saga, and rejects if one of them could not save. A saga you no longer reference is garbage-collected once its facts are finished; while it has unfinished facts it is kept, soSaga.terminate()still reaches it.- If
storage.saverejects,shutdown()rejects with that error and the facts stay in memory: callshutdown()again to save them. On a signal, a failed save is tried once more before the process exits. - With
handleSignals(default whenstorageis set), SIGTERM, SIGINT and SIGHUP trigger the shutdown, then the process exits as it normally would for that signal. If your application listens to the signal itself, it decides when to exit. A second signal during the shutdown exits immediately. When the event loop drains, unfinished facts are saved too. process.exit()can't wait for async work: callawait Saga.terminate()before it.- A node that was running at shutdown runs again after
restore(), so handlers should be idempotent. restore()keeps facts that reference nodes the saga doesn't have, and records that don't have the shape of a fact, and saves them again instead of dropping them (each one is logged).storage.load()must resolve with an array (or nothing).saga.snapshot()returns the unfinished facts at any time.
pseudoInterval
pseudoInterval runs a function again and again, each run interval ms after the previous one has finished, so runs
never overlap:
const { pseudoInterval } = require('drumset');
const poll = {
executor: async () => {
/* ... */
},
isRan: true, // false pauses the loop
doExit: false, // true ends it
interval: 1000,
onError: (error) => console.error(error), // the default
};
pseudoInterval(poll);
// later
poll.doExit = true;An executor that throws or rejects is reported to onError and the loop goes on.
Middlewares
saga.addMiddleware(
[
'A',
'D',
],
[
async (facts) => {
console.log(`Middleware executed`);
},
],
);A middleware that calls next or exit ends the node: the following middlewares and the handler don't run.
Node Definitions
Here we have the next nodes:
- A: our saga`s entry point
- RollbackA: rollbacks succeeded A if something went wrong in the execution chain after that
- CompensateA: rollbacks failed A and goes to the Error node
- B: is executing simultaneously with node C
- C: is executing simultaneously with node B
- RollbackB: the same as RollbackA but for the node B
- CompensateB: the same as CompensateA but for the node B
- RollbackC: ...
- CompensateC: ...
- D: run if B and C nodes both succeeded
- RollbackD: ...
- CompensateD: ...
- Error: prepare and returns error response
- Success: prepare and returns success response
Our saga`s graph

Development
npm install
npm test # build, lint and the tests in test/
npx tsc --noEmit # type check only
npm run sandboxThe compiler is TypeScript 7 (tsc, installed as @typescript/native). The typescript dev dependency is the
TypeScript 6 API package, which typescript-eslint needs until it supports 7; tsc6 runs that compiler.
The same steps run in CI (.github/workflows/ci.yml) on Node.js 20, 22 and 24, together with npm audit.
