idempotent-saga

Reliability primitives for money-movement systems — idempotency, sagas, and exactly-once event handling — over one pluggable storage adapter.

I build payment systems that don't double-charge. This is the toolkit behind that — idempotent sagas, compensations, and exactly-once event handling for fintech & commerce.

Contents

• The problem
• Why the obvious fixes fail
• Quickstart
• The three primitives
• How it works (architecture)
• Correctness & testing
• Run the demo (verifiable proof)
• NestJS usage
• Storage adapters
• When to use this — and when not to
• Production considerations
• Status & roadmap
• Development
• Further reading

The problem

A single logical action — "check out this cart" — fans out across systems you don't control: a payment gateway, a loyalty service, a voucher service, your own database. None of them share a transaction. The network between them is unreliable. And that unreliability shows up as money bugs:

• A client retries a charge that actually succeeded → the customer is billed twice.
• A gateway redelivers its success webhook three times → you ship three orders.
• A checkout fails after step 3 of 4 → loyalty points are spent but no order exists.
• The process crashes mid-charge → money moved, but your system has no record of it.

These are not edge cases. At ~4,000 orders/day, "rare" races happen many times a week. The cost of getting them wrong is refunds, chargeback fees, support load, and eroded trust in checkout — the one flow that has to be trustworthy.

This library packages the three primitives that make those cases safe, with the hard-won design rules baked in:

• Persist intent before acting. A pending record exists before the side effect runs, so a crash mid-flight is recoverable — never "money moved but unrecorded."
• Atomic claim (compare-and-set) kills duplicate-callback races. Exactly one concurrent caller wins a key; the rest wait for and return its result.
• Compensations are themselves idempotent saga steps. Undo is replay-safe, because undo can fail and be retried too.
• Storage is pluggable behind one interface. In-memory for tests; Postgres or DynamoDB in production — same code.

Why the obvious fixes fail

The instinct is to wrap everything in a transaction. Every option in that family breaks for payments — which is why this library exists:

| Approach | Why it fails for money-movement | | --- | --- | | Two-phase commit (2PC) | Gateways expose no prepare/commit protocol. Even if they did, holding locks across minutes-long async callbacks under bursty load wedges the system; a coordinator crash blocks participants. | | Naive retry on timeout | A late success and a real failure are indistinguishable. Blind retry is a coin-flip between "correct" and "charged twice." | | One big DB transaction | A DB transaction ends at your service boundary; it cannot span an outbound gateway call. Crash after capture but before commit = money gone, no record. | | Saga + idempotency (this library) | Accept eventual consistency. Make every step replay-safe and reversible. Survive partial failure and reconcile, instead of pretending it can't happen. |

A deeper treatment of these trade-offs is in DESIGN.md.

Quickstart

npm install idempotent-saga

import { idempotent } from 'idempotent-saga';

// Wrap any risky operation. Same key = runs once, even under retries/concurrency.
const safeCharge = idempotent(
  async (orderId: string, amount: number) => gateway.charge(amount),
  { key: (orderId) => 'charge:' + orderId }
);

await safeCharge('order-1', 2000); // charges
await safeCharge('order-1', 2000); // returns stored result — does NOT charge again

Storage backends and the NestJS decorator are optional peer dependencies — install only what you use:

npm install pg                                              # Postgres adapter
npm install @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb  # DynamoDB adapter
npm install @nestjs/common reflect-metadata                 # @Idempotent decorator

The three primitives

1. Idempotency — a retry with the same key runs once, returns the stored result

const safeCharge = idempotent(
  async (orderId, amount) => gateway.charge(amount),
  { key: (orderId) => 'charge:' + orderId }
);

Call safeCharge('order-1', 100) as many times as you like — the gateway is hit once. Concurrent duplicates collapse to a single execution; the losers wait and return the same result.

2. Saga — multi-step flow; each step has an action + compensate (undo). On failure, compensations run in reverse

const checkout = saga('checkout')
  .step('applyLoyalty',  { action: ctx => loyalty.apply(ctx.userId, ctx.points), compensate: ctx => loyalty.refund(ctx.userId, ctx.points) })
  .step('redeemVoucher', { action: ctx => vouchers.redeem(ctx.voucherId),        compensate: ctx => vouchers.release(ctx.voucherId) })
  .step('chargePayment', { action: ctx => gateway.charge(ctx.orderId, ctx.amount), compensate: ctx => gateway.refund(ctx.orderId) })
  .step('saveOrder',     { action: ctx => orders.create(ctx) });

await checkout.run(ctx);

If chargePayment fails, the orchestrator runs release voucher then refund loyalty — in reverse — and throws a SagaFailure describing exactly what was undone.

3. Exactly-once event / webhook handling

await onceOnly(eventId, async () => { await orders.markPaid(orderId); });

The first delivery runs the handler; every redelivery returns the stored result without re-running it.

How it works (architecture)

All three primitives sit on one StorageAdapter. The entire correctness story rests on a single atomic operation — acquire, a compare-and-set claim:

   idempotent()      saga()        onceOnly()      @Idempotent
        |               |              |               |
        +---------------+------+-------+---------------+
                               |
                     ┌─────────▼──────────┐
                     │  acquire(key)  CAS │  ← exactly one concurrent caller wins
                     │  get / save / release │
                     └─────────┬──────────┘
                               |
         ┌──────────────┬──────┴───────┬──────────────┐
   MemoryAdapter   PostgresAdapter  DynamoDBAdapter  (your adapter)

The lifecycle of a single guarded call:

get(key) ── found? ──► return stored result        (no re-run)
   │ not found
acquire(key) ── lost the race? ──► wait, return winner's result
   │ won
run work() ──► save(key, result) ──► return

Full design rationale — persist-intent-before-acting, why acquire must be atomic, how compensations stay idempotent — is in DESIGN.md. A plain-English version for non-engineers and product stakeholders is in EXPLAINER.md.

Correctness & testing

The interesting bugs here are concurrency and crash bugs, so the tests target exactly those:

| Property under test | What it proves | | --- | --- | | Replay safety | The same key re-run returns the stored result, never re-executes the side effect. | | Concurrent-duplicate collapse (CAS) | N simultaneous callers of one key → exactly one execution; the rest return the same result. | | Failure is not cached | A thrown action releases the claim, so a later legitimate retry can succeed. | | Webhook redelivery | onceOnly runs the handler once across repeated deliveries of the same event. | | Reverse compensation | A saga that fails at step k compensates steps k-1 … 1 in reverse order. |

npm test        # 10/10 passing
npm run typecheck

Run the demo (verifiable proof)

examples/demo-checkout is a runnable checkout app that lets you try to break it — toggle delayed, duplicated, dropped, or crashing gateway callbacks — and watch a live counter prove real charges never exceed real orders.

# build the toolkit, then run the demo
npm install && npm run build
cd examples/demo-checkout && npm install && npm run dev   # http://localhost:3000

Command-line proof (duplicate webhooks on, same order submitted twice):

curl -X POST localhost:3000/toggle/duplicate
curl -X POST localhost:3000/checkout -H 'Content-Type: application/json' -d '{"orderId":"ord_A"}'
curl -X POST localhost:3000/checkout -H 'Content-Type: application/json' -d '{"orderId":"ord_A"}'
curl localhost:3000/stats
# → { distinctOrders: 1, realCharges: 1, zeroDoubleCharges: true }

NestJS usage

import { Idempotent, configureIdempotentStorage, PostgresAdapter } from 'idempotent-saga';

// once at bootstrap:
configureIdempotentStorage(new PostgresAdapter(pgPool));

class PaymentsController {
  @Idempotent({ key: (dto) => 'charge:' + dto.orderId })
  async charge(dto: ChargeDto) {
    return this.gateway.charge(dto.orderId, dto.amount);
  }
}

The decorator wraps the method with idempotent(), deriving the key from the method arguments (typically the request DTO).

Storage adapters

All primitives accept a storage option implementing one interface. The whole correctness story rests on a single atomic operation, acquire (compare-and-set claim).

| Adapter | Atomic claim mechanism | Use for | | --- | --- | --- | | MemoryAdapter | Map guarded by the single-threaded event loop | tests, local dev | | PostgresAdapter | INSERT ... ON CONFLICT DO NOTHING on a unique key | most production apps | | DynamoDBAdapter | PutItem with attribute_not_exists(...) condition | serverless / high-scale |

import { idempotent, PostgresAdapter } from 'idempotent-saga';

const storage = new PostgresAdapter(pgPool); // pgPool: any { query() } client

const safeCharge = idempotent(
  async (orderId, amount) => gateway.charge(amount),
  { key: (orderId) => 'charge:' + orderId, storage }
);

Postgres schema (run once):

CREATE TABLE idempotency_keys (
  key         TEXT PRIMARY KEY,
  state       TEXT NOT NULL,            -- 'pending' | 'completed'
  value       JSONB,
  intent      JSONB,
  created_at  BIGINT NOT NULL,
  updated_at  BIGINT NOT NULL
);

DynamoDB: a table with a string partition key (default attribute pk). Pass the SDK command classes in so the SDK stays optional:

import { DynamoDBAdapter } from 'idempotent-saga';
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, GetCommand, PutCommand, UpdateCommand, DeleteCommand } from '@aws-sdk/lib-dynamodb';

const doc = DynamoDBDocumentClient.from(new DynamoDBClient({}));
const storage = new DynamoDBAdapter(doc, {
  table: 'idempotency_keys',
  commands: { GetCommand, PutCommand, UpdateCommand, DeleteCommand },
});

Writing your own adapter

Implement StorageAdapter: get, acquire (atomic CAS), save, release. The only hard requirement is that acquire is atomic — exactly one concurrent caller may win a given key.

When to use this — and when not to

Use it when a single logical operation crosses systems that don't share a transaction (payment gateways, third-party APIs, separate services), retries or redeliveries are possible, and double-execution moves money or has real-world cost.

You probably don't need it when your operation is a single ACID database transaction (the DB already gives you atomicity), the operation is naturally idempotent (a pure PUT of a full resource), or duplicate execution is harmless.

Being explicit about this matters: idempotency and sagas add a storage round-trip and operational surface. They earn their keep on money-movement and cross-system workflows — not on every endpoint.

Production considerations

• Key design is the whole game. Derive the key from the logical operation (order + step), never from the request envelope. Same logical action → same key, always.
• TTL / cleanup. Idempotency records accumulate. A sweeper to expire old keys is on the roadmap; until then, schedule a periodic delete on created_at.
• Observability. Wrap claims, actions, and compensations in spans so you can see collapsed duplicates and compensation paths in production (OpenTelemetry support is on the roadmap).
• Compensation gaps. A saga can only undo what is reversible. Design irreversible steps (e.g. an email already sent) to be last, or model them as best-effort outside the saga.

Status & roadmap

Status: v0.1 — API is stable; the three primitives plus all three adapters are implemented and tested.

• [ ] TTL / sweeper for expiring old idempotency keys
• [ ] Saga step-level retry policies with backoff
• [ ] Redis adapter (claim via SET NX)
• [ ] Durable saga resumption from a stored run log (resume after process crash)
• [ ] OpenTelemetry spans around claims, actions, and compensations

Development

npm install
npm run typecheck
npm test
npm run build

Further reading

• DESIGN.md — the high-level design: principles, trade-offs, the storage contract, the lifecycle of a guarded call.
• EXPLAINER.md — the same ideas for three audiences (product/business, non-technical, senior dev).
• examples/demo-checkout — the runnable, break-it-yourself proof.

License

MIT © Prateek Kumar