@thadeu/ndb
v1.24.0
Published
TypeScript client for the ndb daemon
Readme
@thadeu/ndb
TypeScript client for ndb, an NDJSON query daemon. Typed collections, a chainable query builder, aggregate, watch and blobs. The client builds JSON. The daemon decides what every query means.
Node 24 or newer, or Bun. ESM and CJS. One runtime dependency (undici, used on Node; Bun uses its fetch).
Docs: client guide, query language, HTTP API.
Install
pnpm add @thadeu/ndb
# or
npm install @thadeu/ndb
bun add @thadeu/ndbYou also need a running ndb daemon. See the repository for how to run one.
Connect
import { connect } from '@thadeu/ndb'
// Unix socket.
const ndb = connect('unix:///run/ndb/ndb.sock')
// TCP with a bearer token.
const remote = connect('tcp://10.0.0.5:7070', { token: 'secret' })
// Address and token from the environment: NDB_URL and NDB_TOKEN.
const fromEnv = connect()
// Retry for up to 5 s while the daemon starts or recovers.
const patient = connect('unix:///run/ndb/ndb.sock', { readyTimeout: 5000 })
await ndb.health() // resolves when the daemon is ready (GET /_health)
await ndb.close()connectdoes not connect. The first call does.- The address is
unix:///pathortcp://host:port. - The token goes only in the
Authorizationheader. A unix socket does not need it. readyTimeoutis in milliseconds. It sets how long a call retries when the daemon is not ready (not_ready, or the socket is missing or refuses). Default 10000. Zero turns the retry off. Writes are retried only when the daemon certainly did not process them. Reads are also retried after a dropped connection.- The pool has up to 16 connections. There is no setting for it.
new Ndb(addr, opts)does the same asconnect.
Define a collection
const calls = ndb.collection('calls', {
time: 'at',
fields: {
status: { type: 'string', index: true },
caller: { type: 'string', unique: true },
duration: { type: 'number', optional: true },
at: { type: 'date' },
meta: { type: 'json', optional: true },
},
})- Field types:
string,number,boolean,date(aDate),json(any value). - Field options:
index,unique,optional. _idis the primary key of every document. It is not a schema field: leave it out on insert and the daemon makes a UUIDv7 (or usenewId()). For a natural key (email, SKU), keep_idand mark that fieldunique.timeis the collection time field. Used by zone maps and$dateTrunc.- The collection is declared on first use (
PUT /calls). You can callawait calls.ready()to do it at once. - A name that ends with
*is a collection pattern:ndb.collection('logs-2026-*', schema)reads every collection whose name starts withlogs-2026-. It declares nothing, and a write throws with codepattern_read_only. Each row has_coll, the name of its collection. - Types come from the schema. Rows have the schema fields,
_idand_v, the document version. Adatefield is aDatein rows. Input takesDatevalues and ISO strings in filters. expiresis thedatefield that holds when a document expires. It turns onexpireIn,expire,persistandttl(below).- The client has no
ttlcatalog option. Use the HTTP API for it.
Insert, get, update, delete
// One document (POST /calls/_doc): { _id, _v }.
const one = await calls.insert({ _id: 'c1', status: 'answered', caller: 'a', duration: 30, at: new Date() })
// Many in one transaction, all or nothing (POST /calls/_bulk): an array.
const results = await calls.insert([
{ _id: 'c2', status: 'failed', caller: 'b', at: new Date() },
{ _id: 'c3', status: 'failed', caller: 'c', at: new Date() },
])
// results: [{ _id: 'c2', _v: 1 }, { _id: 'c3', _v: 1 }]
// Upsert: create by _id, or merge the fields into the stored document (the
// others stay), like doc_as_upsert in Elasticsearch. One transaction.
await calls.upsert({ _id: 'c2', status: 'answered' })
await calls.upsertAll([{ _id: 'c2', status: 'answered' }, { _id: 'c4', status: 'new', at: new Date() }])
// Get by _id (GET /calls/_doc/c1). Returns null when missing.
const row = await calls.get('c1')
// Atomic update by _id (POST /calls/_update/c1). guard adds conditions on
// the same document: a _v guard is an optimistic lock.
const res = await calls.update('c1', { $set: { status: 'done' }, $unset: { duration: true } }, { guard: { _v: row!._v } })
// res: { matched, modified, doc } (doc is the new row with _v; absent for many)
// A guarded decrement: exactly one of two racing calls wins.
await stock.update('a', { $inc: { qty: -1 } }, { guard: { qty: { $gte: 1 } } })
// Create the document with this _id when it is missing.
await calls.update('c9', { $inc: { duration: 1 } }, { upsert: true })
// Atomic update by query (POST /calls/_update_by_query): the first match, or
// every match with many (the query needs _id or an indexed field).
await calls.updateBy({ status: 'failed' }, { $set: { status: 'retry' } }, { many: true })
// Delete by _id (DELETE /calls/_doc/c2), or by query: { deleted }.
await calls.delete('c2')
await calls.deleteBy({ status: 'failed' }, { many: true })- Operators:
$set,$setOnInsert(like$set, but only when an upsert creates the document),$unset,$inc,$push(with{ $each: [...] }),$pull,$min,$max. Field names can be dot paths. - Nothing matched is not an error:
matchedis 0 (for example when_vis stale, so read again and retry).modifiedis 0 when the operators leave the document as it is. updatetakesguard,upsert(creates the document with that_id) andexpireIn.updateBytakesmany,upsert(creates the document from the equality fields of the query) andexpireIn.deleteBytakesmany.manyneeds_idor an indexed field in the query. An empty query is refused (empty_filter).- A key of the operators without
$throwsInvalidUpdateErrorbefore anything is sent. The daemon contract errors (unknown_operator,no_update,conflicting_update,invalid_field,unindexed_many,empty_filter) areInvalidUpdateErrortoo.
An insert with a duplicate _id throws DuplicateIdError (a _bulk insert then writes nothing). A duplicate unique value throws UniqueViolationError. An empty or non-string _id fails with code invalid_id.
Truncate and drop
await calls.truncate() // every document goes; the definition and indexes stay
await calls.drop() // the collection, its definition and its watch consumers go- Both take the same time for 10 or 10 million documents. Watch consumers get one event with
op: 'truncate'(_idis''), then the new changes. - After
drop(), the next call on the collection declares it again. A collection declared inndb.ymlcomes back, empty, when the daemon restarts.
Expiry (TTL like Redis)
const sessions = ndb.collection('sessions', {
expires: 'exp',
fields: { exp: { type: 'date', optional: true } },
})
await sessions.insert({ _id: 's1' }, { expireIn: '1h' }) // or 3600; all docs of the call
await sessions.update('s1', { $set: { user: 'u' } }, { expireIn: '30m' })
const { _v, expiresAt } = await sessions.expire('s1', '30m') // EXPIRE
await sessions.expireAt('s1', new Date(Date.now() + 3_600_000))
await sessions.ttl('s1') // seconds left, -1 = never expires, null = missing or expired
await sessions.persist('s1') // PERSIST- A duration is seconds or a string such as
"90m","1h30m","1d".expireInwins over a value of theexpiresfield in the same call. expireandpersistthrowNotFoundErrorfor a missing or expired document.TtlError(ttl_not_enabled,ttl_derived) means the schema has noexpiresfield, or the expiry comes fromttl+time.- Disk space of expired documents returns after compaction, and fully only with
wal_retention; see Freeing disk.
Query
A query is lazy and immutable (POST /{coll}/_search). It is also thenable: await runs it.
const rows = await calls
.query({ status: 'answered', duration: { $gte: 30 } })
.query({ caller: { $in: ['a', 'b'] } })
.not({ caller: 'zzz' })
.or({ duration: { $lt: 5 } }, { 'meta.agent.id': 7 })
.sort({ at: -1, _id: 1 })
.fields('_id', 'status')
.limit(50)
.skip(100)- Operators:
$eq,$ne,$gt,$gte,$lt,$lte,$in,$nin,$exists. Severalquerycalls are ANDed. Dot paths work onjsonfields. $regex(with$optionsi,m,s,x) uses Go RE2: no backreferences, no look-around. Pass a string, or aRegExp({ name: /^al/i }or{ name: { $regex: /^al/i } }). Only thei,mandsflags are accepted; any other flag (g,y,u, ...) throwsInvalidQueryErrorbefore the request. An anchored prefix such as^abccan use an index.- The order of the
sortkeys is the order of the sort. The id is always the last key. fields(...)returns only these fields. The rows arePartial.page(n, size)setsskipandlimitfor pagen(from 1).first()returns the first row, ornull.- The collection has the same shortcuts:
calls.query(...),calls.sort(...),calls.all(),calls.first(),calls.count(...),calls.each().
A query built elsewhere
A query from a search form or a saved search is not type-checked. rawQuery sends it as is (the daemon validates it), ANDed with the other clauses:
const rows = await calls.rawQuery({ status: 'answered', at: { $gte: 'now-7d' } }).limit(50)For a whole search body (query, sort, limit, after, count), use search. It answers one page:
const page = await calls.search({ query: { status: 'answered' }, sort: { at: -1 }, limit: 50, count: true })
page.rows // decoded like the rows of query()
page.next // cursor for the next page ('' on the last one)
page.total // with count: trueCount
const answered = await calls.count({ status: 'answered' }) // POST /calls/_count
const failed = await calls.query({ status: 'failed' }).count()
const total = await calls.count()Cursor pagination
Offset pages get slower as skip grows. Use a cursor for deep pages.
const q = calls.query({ status: 'answered' }).sort({ _id: 1 }).limit(100)
const page = await q
const token = q.nextCursor // '' on the last page
const next = await calls.query({ status: 'answered' }).sort({ _id: 1 }).limit(100).after(token)A token works only with the same query and sort. Another query shape throws CursorError.
To walk every row, use each(). It follows the cursor, with limit as the batch size:
for await (const row of calls.query({ status: 'answered' }).sort({ _id: 1 }).limit(500).each()) {
console.log(row._id)
}Explain
There is no explain method. Send explain: true in a raw request and read the header:
const res = await ndb.stream('POST', calls.path('_search'), { query: { status: 'failed' }, explain: true, limit: 1 })
console.log(res.header('x-ndb-plan')) // 'index:status'
console.log(res.header('x-ndb-explain')) // JSON string with the counters
await res.body.dump()Aggregate
The pipeline needs exactly one $group. Before it: $match and one $sort. After it: $match, $sort, $limit, $skip, $project. Accumulators: $count, $sum, $avg, $min, $max, $first, $last.
const res = await calls.aggregate(
[
{ $match: { status: { $in: ['answered', 'failed'] } } },
{
$group: {
_id: { status: '$status', hour: { $dateTrunc: { date: '$at', unit: 'hour' } } },
n: { $count: {} },
avg: { $avg: '$duration' },
},
},
{ $sort: { n: -1 } },
],
{ limit: 100, count: true },
)
res.rows // the group rows
res.next // cursor for the next page ('' on the last one)
res.total // with count: trueOptions: limit, skip, after, count. Pass a type to type the rows: calls.aggregate<{ _id: string; n: number }>(...). $dateTrunc units are minute, hour, day, week and month.
Watch
A watch streams the changes of a collection to a named consumer. First it sends the events after the last ack. Then it sends new events as they happen.
import { HistoryTruncatedError, WatchLaggedError } from '@thadeu/ndb'
const controller = new AbortController()
try {
for await (const ev of calls.watch('relay', { query: { status: 'answered' }, signal: controller.signal })) {
// ev: { consumer, pos, op: 'put' | 'del' | 'truncate', _id, _v, doc }
await handle(ev)
await calls.ack(ev)
}
} catch (err) {
if (err instanceof WatchLaggedError) {
// The consumer was too slow. Call watch again. It continues after the last ack.
} else if (err instanceof HistoryTruncatedError) {
// The saved position is too old. Start over.
} else {
throw err
}
}- Options:
from: 'now'(a new consumer starts at the end),limit(stop after n events),query(same filters as a query),signal(anAbortSignalthat stops the stream). - A delete event has
doc: undefined. Withquery, a delete is sent only if the same stream saw an earlier matching put of that id. - Delivery is at least once. After a restart an event can come again, so make the work safe to repeat.
ack(ev)stores the position (POST /{coll}/_ack). The nextwatchwith the same consumer name starts after it.HistoryTruncatedErrormeans the WAL files for the position were deleted (wal_retention).
Streams
A stream is an append-only log with consumer groups, with the behavior of Redis Streams (see Streams). T is the type of the entry. The entry ID and time are in Entry, not in T.
const orders = db.stream<{ order: number; total: number }>('orders')
const id = await orders.append({ order: 42, total: 99.9 }) // "1731164400000-0"
const ids = await orders.appendMany([{ order: 43, total: 1 }, { order: 44, total: 2 }]) // one fsync
await orders.append(o, { trim: { maxLen: 10_000, approx: true } })
const entries = await orders.range('-', '+', 100) // XRANGE; entries[0].id, .time, .doc
const info = await orders.info() // length, first and last ID, groupsA worker in a group. work reads its own pending entries first, then new ones (waiting up to blockMs), acks the entries the handler did without an error, and claims the entries other workers left idle. An entry delivered maxDeliveries times goes to the dead-letter stream:
import { PoisonError, startNew } from '@thadeu/ndb'
const g = orders.group('billing')
await g.ensure(startNew) // or startAll
const controller = new AbortController()
await g.work('worker-1', async (e, signal) => {
await charge(e.doc, signal) // resolves: ack; throws: stays pending
}, {
count: 100,
claimIdleMs: 60_000,
maxDeliveries: 5,
deadLetter: 'orders_dead',
onError: (err) => console.error(err), // a PoisonError comes after its ack
signal: controller.signal,
})work resolves when the signal aborts; the abort also ends a blocked read. Delivery is at least once: make the work safe to repeat. Run many workers with different consumer names to share the work. A deleted entry is acked without the handler. Defaults: count 10, blockMs 5000, claimIdleMs 60000 (negative turns the claim off), claimEveryMs claimIdleMs / 2 (at least 1000). A handler that throws after the abort leaves its entry pending: it is not reported and not buried. A failed read goes to onError and is tried again after a backoff (100 ms, doubled up to 5 s), also when the group or the stream does not exist yet: a worker that starts before its group waits for it. Only the abort ends work. An error that onError throws is ignored.
Delayed retry (ndb only). With retry, an entry whose handler fails is nacked: it comes back after the wait of its try, with no owner while it waits (a restarted worker does not take it early; any worker of the group takes it when it is due, within about dueEveryMs, default 1000). retry is (deliveries) => ms or an array of waits (the last one repeats); tiers builds the function, with steps { waitMs, upTo }. An entry older than maxAgeMs (by its ID time) goes to the dead letter with reason: 'max_age' and is acked, with no retry. An entry that fails on its maxDeliveries-th delivery still goes to the dead letter (reason: 'max_deliveries'), not to a nack. The fsw-esl schedule (15 s, 1 min, 10 min, 1 h, 6 h, for 15 days):
import { tiers } from '@thadeu/ndb'
const minute = 60_000
const hour = 60 * minute
await g.work('worker-1', handler, {
retry: tiers(
{ waitMs: 15_000, upTo: 1 },
{ waitMs: minute, upTo: 5 },
{ waitMs: 10 * minute, upTo: 11 },
{ waitMs: hour, upTo: 23 },
{ waitMs: 6 * hour },
),
maxAgeMs: 15 * 24 * hour,
deadLetter: 'orders_dead',
signal: controller.signal,
})The calls one by one, when you need your own loop:
| Redis | TypeScript |
|---|---|
| XGROUP CREATE | g.create(startNew) / g.ensure(startNew) |
| XREADGROUP ... > | g.read('w1', { count: 100, blockMs: 5000, signal }) |
| XREADGROUP ... 0 | g.read('w1', { pending: true }) |
| XACK | g.ack(...ids) |
| XPENDING | g.pending() / g.pendingList({ idleMs: 60_000 }); ndb only: waiting, retryAt, { due: true } |
| XCLAIM / XAUTOCLAIM | g.claim('w2', 60_000, ids) / g.autoClaim('w2', 60_000, '0-0', 100) |
| ndb only (no Redis command) | g.nack('w1', ids, 60_000) (back in 1 min) / g.claimDue('w2', { cursor: '0-0', count: 100 }) |
| XGROUP SETID / DESTROY | g.setId(id) / g.destroy() |
| XGROUP CREATECONSUMER / DELCONSUMER | g.createConsumer('w1') / g.deleteConsumer('w1') |
| XREAD (no group) | orders.read({ after: id, count: 100, blockMs: 5000, signal }); after: '$' gives only later entries |
| XTRIM | orders.trim({ maxLen: 1000 }) or { minId, approx, limit } |
| XDEL | orders.delete(id) |
| XINFO GROUPS / CONSUMERS | orders.groups() / g.consumers() |
A blocked read has no request timeout: it waits as long as blockMs. Stream errors have their own classes: StreamIdError (invalid_stream_id, stream_id_too_small), UnknownGroupError, GroupExistsError and NotAStreamError. db.stream(name) with one argument is a stream; with a method and a path it is a raw request.
Redis commands (ioredis and node-redis)
An app that uses Redis Streams can change to ndb and keep its calls. Two subpaths give the stream commands of the two Redis libraries on top of ndb streams, with the same arguments and the same answers. The app selects the backend in one place:
import { connect } from '@thadeu/ndb'
import { ioredis } from '@thadeu/ndb/ioredis'
import { Redis } from 'ioredis'
const streams = process.env.STREAMS_BACKEND === 'ndb' ? ioredis(connect()) : new Redis()
await streams.xadd('orders', 'MAXLEN', '~', 1000, '*', 'order_id', '42')
await streams.xgroup('CREATE', 'orders', 'billing', '$', 'MKSTREAM')
const got = await streams.xreadgroup('GROUP', 'billing', 'w1', 'COUNT', 10, 'BLOCK', 5000, 'STREAMS', 'orders', '>')
// [['orders', [['1731164400000-0', ['order_id', '42']]]]], or null
await streams.xack('orders', 'billing', '1731164400000-0')The adapters have no dependency: the main entry does not load them, and they do not import ioredis or redis. Their types are structural. A type test in this package checks that each adapter is assignable to the stream commands of its library (Pick<Redis, ...> of ioredis 5, Pick<RedisClientType, ...> of node-redis 5), so the two sides of the ?: above have the same methods.
@thadeu/ndb/ioredis
ioredis(db) gives the methods of ioredis 5: lowercase names, the arguments of the Redis command in the same order (option words in any case), an optional callback as the last argument, and the raw arrays of Redis as the answer.
| Method | Arguments | Answer |
|---|---|---|
| xadd | key [NOMKSTREAM] [MAXLEN\|MINID [=\|~] n [LIMIT n]] id field value ... | the ID; null with NOMKSTREAM and no stream |
| xlen | key | a number |
| xrange / xrevrange | key start end [COUNT n] ((id leaves the ID out) | [[id, [field, value, ...]], ...] |
| xread | [COUNT n] [BLOCK ms] STREAMS key ... id ... | [[key, [[id, [field, value]]]]] or null |
| xdel | key id ... | a number |
| xtrim | key MAXLEN\|MINID [=\|~] n [LIMIT n] | a number |
| xgroup | CREATE key group id [MKSTREAM] [ENTRIESREAD n], SETID key group id [ENTRIESREAD n], DESTROY key group, CREATECONSUMER key group consumer, DELCONSUMER key group consumer | 'OK', 'OK', 1/0, 1/0, the pending count |
| xreadgroup | GROUP group consumer [COUNT n] [BLOCK ms] [NOACK] STREAMS key ... id ... | as xread; with an ID that is not >, [[key, []]] when there is no entry, and [id, null] for a deleted entry |
| xack | key group id ... | a number |
| xpending | key group | [count, lowestId, highestId, [[consumer, '2']]], or [0, null, null, null] |
| xpending | key group [IDLE ms] start end count [consumer] | [[id, consumer, idleMs, deliveries], ...] |
| xclaim | key group consumer minIdle id ... [IDLE ms] [RETRYCOUNT n] [FORCE] [JUSTID] [LASTID id] | entries as xrange; IDs with JUSTID |
| xautoclaim | key group consumer minIdle start [COUNT n] [JUSTID] | [nextStart, entries or IDs, deletedIds] |
| xinfo | STREAM key, GROUPS key, CONSUMERS key group | flat [name, value, ...] arrays, with the field names of Redis |
The return types are those of ioredis (unknown[] for xreadgroup, xpending, xclaim and xautoclaim; unknown for xgroup and xinfo). The subpath exports EntryReply, StreamsReply and GroupStreamsReply for a cast.
@thadeu/ndb/node-redis
nodeRedis(db) gives the methods of node-redis 5 (the redis package): camel case names, options as an object, and the answers that node-redis makes from the arrays.
import { nodeRedis } from '@thadeu/ndb/node-redis'
import { createClient } from 'redis'
const streams = process.env.STREAMS_BACKEND === 'ndb' ? nodeRedis(connect()) : await createClient().connect()
await streams.xAdd('orders', '*', { order_id: '42' }, { TRIM: { strategy: 'MAXLEN', strategyModifier: '~', threshold: 1000 } })
await streams.xGroupCreate('orders', 'billing', '$', { MKSTREAM: true })
const got = await streams.xReadGroup('billing', 'w1', { key: 'orders', id: '>' }, { COUNT: 10, BLOCK: 5000 })
// [{ name: 'orders', messages: [{ id: '1731164400000-0', message: { order_id: '42' } }] }], or null| Method | Answer |
|---|---|
| xAdd(key, id, message, { TRIM }?) | the ID |
| xAddNoMkStream(key, id, message, { TRIM }?) | the ID, or null when there is no stream |
| xLen(key) | a number |
| xRange(key, start, end, { COUNT }?) / xRevRange(key, end, start, { COUNT }?) | [{ id, message }] |
| xRead(streams, { COUNT, BLOCK }?) | [{ name, messages: [{ id, message }] }] or null |
| xDel(key, id \| ids) | a number |
| xTrim(key, 'MAXLEN' \| 'MINID', threshold, { strategyModifier, LIMIT }?) | a number |
| xGroupCreate(key, group, id, { MKSTREAM, ENTRIESREAD }?) / xGroupSetId(key, group, id, { ENTRIESREAD }?) | 'OK' |
| xGroupDestroy(key, group) / xGroupCreateConsumer(key, group, consumer) | 1 or 0 |
| xGroupDelConsumer(key, group, consumer) | the pending count |
| xReadGroup(group, consumer, streams, { COUNT, BLOCK, NOACK }?) | as xRead |
| xAck(key, group, id \| ids) | a number |
| xPending(key, group) | { pending, firstId, lastId, consumers: [{ name, deliveriesCounter }] \| null } |
| xPendingRange(key, group, start, end, count, { IDLE, consumer }?) | [{ id, consumer, millisecondsSinceLastDelivery, deliveriesCounter }] |
| xClaim(key, group, consumer, minIdle, id \| ids, { IDLE, RETRYCOUNT, FORCE, LASTID }?) | [{ id, message }] |
| xClaimJustId(...) | IDs |
| xAutoClaim(key, group, consumer, minIdle, start, { COUNT }?) | { nextId, messages, deletedMessages } |
| xAutoClaimJustId(...) | the same, with IDs in messages |
| xInfoStream(key) | { length, 'last-generated-id', 'entries-added', 'first-entry', ... } |
| xInfoGroups(key) | [{ name, consumers, pending, 'last-delivered-id', 'entries-read', lag }] |
| xInfoConsumers(key, group) | [{ name, pending, idle, inactive }] |
The answers are those of the default client (RESP 2, no type mapping). A TRIM needs its strategy, as with a real Redis.
What is different from Redis
- Values are text. Redis keeps strings, so the adapters give every value as a string, and they store the values of
xaddas strings. An entry that other ndb code appended with JSON types comes back as text:42is'42',trueis'true', andnull, an array or an object are JSON text. - Field order. The daemon gives the fields of an entry in name order, not in the order of the
xadd. A field name that starts with_is reserved, and a field name given two times is kept one time. - One stream for each request.
xreadandxreadgroupwith many streams send one request for each stream. WithBLOCKand more than one stream they throw (not_supported). Before anxreadgroupof>on many streams, the adapters check the group of each stream, so aNOGROUPon one stream gives nothing from the others, as in Redis. - No
COUNT, orCOUNT 0or less. Redis gives every entry. The adapters give at most 10 000 in one read (xrangehas no limit). ACOUNTover 10 000 is read as 10 000. BLOCK 0waits with no limit, as in Redis: the adapter repeats a 30 s read. A blockedxreadon a stream that does not exist looks for it each 500 ms.- Not supported (the call throws):
XREADwith the ID+,XREADGROUP ... CLAIM,XCLAIM ... TIME,XINFO STREAM ... FULL,XGROUP HELPandXINFO HELP, the Redis 8.2 optionsDELREFandACKED, and the idempotentXADDoptions.LASTIDandKEEPREFare read and have no effect. - Not there:
XSETID,XACKDEL,XDELEX, theBuffervariants of ioredis (xreadBuffer, ...), pipelines andMULTI. The object has only the methods of the tables. xinfo:radix-tree-keysandradix-tree-nodesare0.- A range of
xpendingdoes not take(id. - Errors in the arguments are those of Redis: a negative
BLOCK,LIMITwithout~, an argument after the end ofxtrim, and a number that is not decimal digits (0x10,5000.0). The option words go in any order where Redis takes them. (idwithout a seq inxrangeis read as in Redis:(1000as a start leaves out only1000-0, and(2000as an end leaves out only2000-<max seq>.- Errors are
ReplyError(exported by the two subpaths), a subclass ofNdbError. Itsmessageis the text of Redis, so a check such aserr.message.startsWith('BUSYGROUP')works on the two backends.codeandstatusare those of ndb, andcauseis the ndb error.
| Case | message starts with | code |
|---|---|---|
| The group exists | BUSYGROUP | group_exists |
| No such group, or no stream in a group command | NOGROUP | unknown_group, unknown_collection |
| XGROUP without MKSTREAM, and no stream | ERR The XGROUP subcommand requires the key to exist | unknown_collection |
| XINFO and no stream | ERR no such key | unknown_collection |
| An ID that is not greater than the last one | ERR The ID specified in XADD is equal or smaller | stream_id_too_small |
| A bad ID | ERR Invalid stream ID | invalid_stream_id |
| The name is a collection that is not a stream | WRONGTYPE | not_a_stream |
| A bad argument list | ERR | invalid_request |
| Something ndb does not have | ERR ndb: ... is not supported | not_supported |
An error that Redis does not have (unauthorized, not_ready, a network error) stays as it is. As in Redis, these give a value and not an error when the stream does not exist: xlen (0), xrange ([]), xread (null), xdel, xtrim and xack (0).
Blobs
Large values live in the blob store. A document keeps only a reference. The same content always gives the same reference.
const blob = await ndb.putBlob(bytes, 'audio/wav') // POST /_blobs: { $blob, size, type }
await calls.update('c1', { $set: { recording: blob } })
const data = await ndb.getBlob(blob) // GET /_blobs/{ref}: a Uint8ArrayAdmin
await calls.stats() // rows, indexes, files, bytes (GET /calls/_stats)
await calls.compact() // merge every segment now (POST /calls/_compact)
await calls.ensureIndex('caller') // built in the background (PUT /calls/_indexes/caller); the indexed fields
await calls.dropIndex('caller')
await ndb.collections() // one definition per collection (GET /_collections)
await ndb.stats() // every collection, totals, version, uptime (GET /_stats)
await ndb.size() // disk use per collection and part (GET /_size)Errors
Daemon errors are NdbError or a subclass. They have code (the daemon code) and status (the HTTP status; 0 for an error line inside a stream).
| Class | Codes |
|---|---|
| UnauthorizedError | unauthorized |
| NotFoundError | not_found, unknown_collection |
| DuplicateIdError | duplicate_id |
| UniqueViolationError | unique_violation |
| InvalidQueryError | invalid_query, invalid_pipeline |
| CursorError | cursor_invalid, cursor_mismatch, cursor_expired |
| WatchLaggedError | watch_lagged |
| HistoryTruncatedError | history_truncated |
| TtlError | ttl_not_enabled, ttl_derived |
| InvalidUpdateError | unknown_operator, no_update, conflicting_update, invalid_field, unindexed_many, empty_filter, invalid_update |
| NdbError | every other code: not_ready, catalog_conflict, catalog_invalid, direct_collection, invalid_request, group_limit_exceeded, request_too_large, line_too_large, closed, invalid_position, confirm_required, unknown_route, pattern_read_only, http_status, ... |
import { NdbError } from '@thadeu/ndb'
try {
await calls.get('c1')
} catch (err) {
if (err instanceof NdbError && err.code === 'not_ready') {
// The daemon is still recovering, and readyTimeout ran out.
}
}ErrorCode is the union of the known codes.
Testing
The tests of this package (vitest) build the daemon from the repository (go build ./cmd/ndb) in a global setup. They start it on a temp unix socket and a free TCP port, wait for GET /_health to return 200, and stop it at the end. To do the same by hand:
ndb -config /nonexistent.yml -data-dir "$(mktemp -d)" -unix /tmp/ndb-test.sock -tcp 127.0.0.1:7070Add -token-file <path> to ask for a bearer token on TCP. Use a short socket path: macOS allows 104 bytes. Use a new collection name per test.
License
MIT.
