@millstream/redis-stream
v0.5.0
Published
Redis Streams Publisher/Subscriber for @millstream on ioredis: XREADGROUP consumer groups, XACK on ack, XAUTOCLAIM recovery
Maintainers
Readme
@millstream/redis-stream
Redis Streams Publisher / Subscriber for
millstream, built on ioredis. Peer dependency:
ioredis@^5 || ^6; Redis ≥ 6.2 (XAUTOCLAIM).
import { Redis } from 'ioredis';
import { RedisStreamPublisher, RedisStreamSubscriber } from '@millstream/redis-stream';
const client = new Redis('redis://localhost:6379');
const publisher = new RedisStreamPublisher({ client, maxLen: 100_000 });
router.addConsumerHandler({
name: 'billing',
subscribeTopic: 'orders.created',
subscriber: new RedisStreamSubscriber({ client, consumerGroup: 'billing' }),
handler: async (msg) => {
/* … */
},
});The client belongs to you. Each subscription opens a duplicate() of it for blocking reads.
Semantics
- Topic = stream key (
streamNameto change).publish()is oneXADDper message inside aMULTI, with fieldsuuid,payload(binary-safe) andmetadata(JSON).maxLentrims the stream (MAXLEN ~ n) — trimmed entries are gone even for groups that haven't read them. consumerGroup→XREADGROUP: each group gets every message, subscribers of one group compete for them, and progress lives in Redis. New groups start atstartId('0': everything, default;'$': only new entries).- No group →
XREADfromstartIdwith the position in memory: every subscription sees the whole stream. - Ack →
XACK. Nack → in-place redelivery of a fresh copy afternackResendMs(the entry stays pending), so a poison message blocks its subscription: addpoisonQueue/retrymiddlewares on the Router. - Ordering: strict with
concurrency: 1and one consumer per group.concurrency: nhandles up tonentries in parallel. - Crashes: entries pending on a dead consumer for
claimIdleMs(30s) are claimed withXAUTOCLAIMby the group's other subscribers (checked everyclaimIntervalMs), which can reorder them. Entries being handled have their idle time reset, so slow handlers aren't claimed. Abort while holding an entry makes it claimable at once. - A subscription's consumer (
millstream-<uuid>, orconsumerName) is removed from the group when it stops with nothing pending. redisStreamInfo(msg)returns stream / entry id / group. Entries added by other clients get the uuidredis:<stream>:<id>, and their fields (as JSON) as the payload.
