@krizaka/outbox
v0.1.3
Published
The transactional outbox of every Krizaka product: an event written in the caller's transaction, delivered after the commit at least once by leased workers, with doubling retries, dead letters and an idempotent-consumer helper. No broker, no database clie
Maintainers
Readme
@krizaka/outbox
The transactional outbox of every Krizaka product. An event is written in the caller's transaction, so it exists exactly when the work committed, and is delivered after the commit, at least once, by workers that lease a batch. No broker: PostgreSQL and FOR UPDATE SKIP LOCKED share the work between instances. It replaces Akinaza's mail_outbox (whose flushOutbox nobody called) and the analytics outboxes the products carried; it is the TypeScript twin of krizaka-messaging's krizaka_outbox (the envelope is id, type, occurredAt, payload).
The guarantee is at least once. Consumers must be idempotent on the message id. A claim is exclusive while its lease is valid. A worker that delivers and then crashes before
completelets the lease expire, and the message is delivered again. The outbox never promises « exactly once ».
import { emitMigration, transaction } from "@krizaka/db";
import { consumeOnce, createOutbox, outboxSchema, runWorker } from "@krizaka/outbox";
import { readOutboxSettings } from "@krizaka/outbox/settings";
emitMigration(outboxSchema, 1, { tables: {} }); // commit the SQL once, in your migrations: outbox.message and outbox.processed
const outbox = createOutbox({ clock: { now: () => new Date() }, settings: () => readOutboxSettings((key) => settings.get(key)) });
// 1. in the use case: the message commits or rolls back with the work
await transaction(connect, async (tx) => {
const redemption = await coupons.redeem(tx, { code, accountId });
await outbox.append(tx, { topic: "credits", type: "evt.coupon.redeemed.v1", payload: { redemptionId: redemption.id }, id: redemption.eventId });
});
// 2. a worker the product starts (Next.js instrumentation.ts); the package has no timer, you pass `sleep`
const stop = new AbortController();
void runWorker({
outbox, query: pool.query.bind(pool) /* a pool-level Query */, owner: `${hostname}:${process.pid}`, topic: "credits", signal: stop.signal,
sleep: (seconds) => new Promise((resolve) => setTimeout(resolve, seconds * 1000)),
deliver: async (message) => { /* throw to retry */ },
});
// 3. a consumer with an effect in the database applies it once per message id, in the effect's transaction
await transaction(connect, (tx) => consumeOnce(tx, "credits", message.id, clock.now(), () => credits.grant(tx, …)));append(tx, { topic, type, payload, id?, availableAt? })takes the transaction'sQueryfirst. A givenidmakes the append idempotent (the same fact twice is one message).typeis a routing key,evt.<module>.<fact>.v1like Java's.claim(query, { owner, topic? })isUPDATE … WHERE id IN (SELECT … FOR UPDATE SKIP LOCKED)and sets the lease.completeandfail(error)act only for the lease's owner (a worker that lost its lease cannot overwrite the new holder).failbacks off —availableAt = now + backoffSeconds × 2^(attempts−1), capped — and aftermaxAttemptsthe message is dead (kept, with its last error;retryDeadgives it another life). A message whose last allowed attempt's lease expired with no answer is dead too, not delivered a ninth time.runWorkerloopsdrainOnce(claim a batch, deliver in order, record each outcome), sleeps through thesleepyou pass when a round found nothing, and ends when the signal is aborted. RundrainOncefrom a scheduler instead if you prefer cron to a loop.- Settings (
@krizaka/outbox/settings, needs@krizaka/settingsandzod):outbox.batchSize(50, 1–500),outbox.leaseSeconds(60, 10–3600: longer than your slowest delivery),outbox.maxAttempts(8, 1–50),outbox.backoffSeconds(30, 1–3600),outbox.backoffMaxSeconds(3600),outbox.retentionDays(7).purgeremoves delivered and dead messages older than the retention: call it from the product's job. - Tables (
outboxSchema, Postgres schemaoutbox):message(id, topic, event_type, payload, created_at, available_at, attempts, leased_until, lease_owner, delivered_at, dead_at, last_error) andprocessed(consumer, message_id), whichconsumeOnceuses: the marker and the effect commit in the same transaction, so a crash leaves neither and a redelivery finds the marker. outboxStoreContract(from@krizaka/outbox/testkit) is the contract every store runs;memoryOutboxStore()is for unit tests.
