@rivyn/db-server

Self-hosted NoSQL database server with a MongoDB-like document model. No cloud, no accounts — you run it on your own machine or VPS, and connect to it with @rivyn/db.

Features

• Document collections with MongoDB-style queries ($gt, $in, $regex, $or, $and, $nor, $not, $exists, $elemMatch, $all, $size, $type, sort/limit/skip)
• Dotted paths reach through arrays: punishments.active matches any element
• Update operators: $set, $setOnInsert, $unset, $inc, $mul, $min, $max, $rename, $push, $pull, $pop, $addToSet, with $each
• Aggregation stages: $match, $group, $project, $unwind, $sort, $skip, $limit, $count
• Positional updates: $, $[] and $[identifier] with arrayFilters
• upsert — create the document from the filter when nothing matches
• Durable storage: append-only journal (CRC-checked) + atomic snapshots, crash recovery via replay
• Configurable fsync policy: always, interval (default), or off
• Secondary indexes, including unique and multikey indexes over arrays; sorted indexes accelerate range queries ($gt/$gte/$lt/$lte)
• Aggregation pipelines: $match, $group ($sum, $avg, $min, $max), $sort, $limit, $skip
• Optional per-collection schema validation enforced on every write (types, required, enum, min/max, pattern, nested fields, strict mode)
• Mandatory key authentication (timing-safe), brute-force IP lockout, per-IP rate limiting and connection caps, optional IP allowlist, optional TLS
• Data directory lockfile — prevents two servers from corrupting the same data
• Rivyn Studio — a built-in web UI (Prisma Studio-style) for browsing, filtering and editing your data
• Zero runtime dependencies

Quick start

npm i -g @rivyn/db-server
rivyn-server init     # creates rivyn.config.json + prints your auth key
rivyn-server start

The server listens on tcp://0.0.0.0:7223 by default. Keep the auth key safe — clients need it to connect.

Rivyn Studio (web UI)

Studio starts automatically with the server at http://127.0.0.1:7224. Log in with your auth key to browse collections, run filter queries, insert/edit/delete documents and manage indexes — no extra install, zero dependencies, a single embedded page.

Studio is bound to localhost by default and is protected by the same auth key with brute-force lockout. To reach a remote server's Studio, use an SSH tunnel (ssh -L 7224:127.0.0.1:7224 user@server) instead of exposing the port. Configure via rivyn.config.json:

"studio": { "enabled": true, "host": "127.0.0.1", "port": 7224 }

CLI

rivyn-server init                       create config + auth key
rivyn-server start                      start the server
rivyn-server backup <target-dir>        snapshot everything and copy data to target
rivyn-server restore <backup-dir> --force
                                         replace the data directory with a backup
rivyn-server version                    print version

backup works while the server is running: it detects the lock, asks the live server to flush snapshots over the wire, then copies the data directory. restore still requires the server to be stopped.

Configuration (rivyn.config.json)

| Key | Default | Description | | --- | --- | --- | | host | 0.0.0.0 | Bind address | | port | 7223 | TCP port | | dataDir | ./data | Where collections are stored | | authKey | generated | Shared secret; min 32 chars | | allowlist | [] | If non-empty, only these IPs may connect (loopback always allowed) | | maxOpsPerSecond | 2000 | Per-IP operation rate limit | | maxAuthFailures | 5 | Wrong keys before the IP is blocked | | blockMinutes | 15 | Block duration after too many auth failures | | maxConnectionsPerIp | 32 | Concurrent connection cap per IP | | fsync | interval | always (safest), interval, or off (fastest) | | fsyncIntervalMs | 100 | Flush interval when fsync is interval | | snapshotOps | 1000 | Compact the journal after this many writes | | snapshotBytes | 4194304 | Compact once the journal passes this size (4 MiB) | | tls | null | { "certFile": "...", "keyFile": "..." } to enable TLS | | studio | { enabled: true, host: "127.0.0.1", port: 7224 } | Built-in web UI; set enabled: false to turn off |

Indexes

Indexes are declared per field and may use dotted paths. A path that crosses an array builds a multikey index: the document is indexed under every value the path yields, so one document with three punishments contributes three entries and is still returned once.

createIndex { "col": "users", "field": "level.xp" }            // range queries
createIndex { "col": "users", "field": "punishments.active" }  // multikey
createIndex { "col": "users", "field": "id", "unique": true }

The query planner uses an index only when the filter names exactly that field. Anything else falls back to a full scan of the collection, which is in memory and therefore fast, but linear. Clients using @rivyn/db can declare indexes in the schema with index: true and the model creates them on first write.

Rate limiting bites before indexing does. maxOpsPerSecond defaults to 2000 per IP — a busy application will hit that and start seeing rate limit exceeded errors. Raise it in rivyn.config.json before going to production.

Wire protocol

Length-prefixed JSON frames over TCP (or TLS): 4-byte big-endian body length, then a UTF-8 JSON body. Max frame size 16 MB.

Requests carry a numeric id; the response echoes it, so requests can be pipelined on one socket:

→ { "id": 0, "op": "auth", "key": "..." }
← { "id": 0, "ok": true, "data": { "server": "rivyn", "version": "0.2.1" } }

→ { "id": 1, "op": "insert", "col": "users", "docs": [{ "name": "arel" }] }
← { "id": 1, "ok": true, "data": { "insertedIds": ["..."] } }

The first message on every connection must be auth. Unauthenticated sockets are dropped after 10 seconds.

Operations

| Op | Params | Returns | | --- | --- | --- | | auth | key | server info | | ping | — | "pong" | | insert | col, docs[] | { insertedIds } | | find | col, filter?, opts? (sort, limit, skip) | Doc[] | | findOne | col, filter? | Doc \| null | | update | col, filter?, update, multi?, arrayFilters?, upsert? | { modifiedCount, upsertedId? } | | delete | col, filter?, multi? | { deletedCount } | | count | col, filter? | number | | aggregate | col, pipeline ($match, $group with $sum/$avg/$min/$max, $sort, $limit, $skip) | Row[] | | snapshotAll | — | true (flushes every collection to a snapshot) | | findOneAndUpdate | col, filter?, update, returnNew?, arrayFilters?, upsert? | Doc \| null | | findOneAndDelete | col, filter? | Doc \| null | | stats | col? | server stats, or collection stats when col is given | | setSchema | col, schema (or null to clear) | true | | getSchema | col | SchemaSpec \| null | | createIndex | col, field (dotted paths allowed), unique? | { field, unique } | | dropIndex | col, field | true | | indexes | col | IndexDef[] | | drop | col | boolean | | collections | — | { name, count }[] |

Capacity

Every document lives in memory; disk is for durability, not for paging. That makes reads fast and the ceiling firm — a collection has to fit in RAM.

Measured at roughly 1.5 KB of heap per document for a small document with a nested object and a two-element array, plus about 120 bytes per index entry:

| Documents | Rough footprint | | --- | --- | | 50 000 | ~75 MB | | 250 000 | ~375 MB | | 1 000 000 | ~1.5 GB |

stats reports estimatedBytes per collection, sampled from the documents themselves, so the real figure for your shape is one call away rather than a guess from this table.

Sorting

sort with a limit is answered from the index when the sort is on a single indexed field — the query walks the index in order and stops once the page is full, instead of sorting the whole collection and discarding the rest. At 50 000 documents that is 0.5 ms instead of 166 ms.

The index is skipped, and the ordinary sort runs, when the sort spans several fields, the field has no index, the index holds mixed value types, or the walk cannot fill the requested page. Results are identical either way, ties included.

find { "col": "users", "sort": { "level.xp": -1 }, "limit": 10 }   // index-backed
find { "col": "users", "sort": { "level.xp": -1 } }                // full sort

Storage format

Each collection lives in data/<name>/:

• journal.log — append-only op log; one CRC32-prefixed JSON entry per line
• snapshot.riv — JSONL snapshot (header line + one doc per line), written atomically
• meta.json — index definitions

Writes append physical entries (put full docs / del ids) to the journal. A snapshot is written in the background and the journal rotated once either snapshotOps writes or snapshotBytes of journal have accumulated, and again on a clean shutdown — so a graceful restart replays nothing. On startup the server loads the snapshot and replays the journal; a torn tail (power loss) is detected by CRC and truncated safely.

No-op writes are dropped. An update whose result is byte-identical to the stored document is not written to the journal at all, and its updatedAt is left untouched. A client that re-writes an unchanged dataset on a schedule therefore costs nothing on disk — only real changes append. modifiedCount still reports how many documents matched, so this is invisible to existing callers.

Development

npm install
npm run dev     # run from source with tsx
npm test        # run the test suite
npm run bench   # run the TCP loopback benchmark
npm run build   # compile to dist/

License

MIT