notedb

A Datomic- and SQLite-inspired embedded document database. Immutable, append-only, file-backed. Rust core, JavaScript API.

• 🗃️ Document store — JSON documents in named collections, no schema.
• 📜 Append-only log — every write is a fact appended to one file. The file is the history.
• ⏳ Time travel — read any note (or list a whole collection) as it was at a past point in time.
• 🧱 Crash safe — a torn write from a crash is discarded on reopen. No corruption, no recovery step.
• 🦀 Native speed — Rust core via napi-rs, exposed as a plain JS module.
• 📦 Zero infrastructure — no server, no daemon. One file is one database; copy it anywhere.

It fills the gap SQLite leaves for document data: SQLite is relational-first, MongoDB needs a running server, and the JS-only embedded stores are abandoned.

Install

npm install @fippli/notedb

Quick start

import notedb from "@fippli/notedb"

const db = notedb.use("/data/db.log")

const cars = db.collections.add("cars")

// remember(doc) inserts (id generated); remember(id, doc) updates
const note = cars.remember({ make: "Volvo", year: 1992 })
cars.remember(note.id, { make: "Volvo", year: 1993 })

cars.read(note.id)              // { id, make: "Volvo", year: 1993 }
cars.list()                     // [{ id, ... }, ...]
cars.find((d) => d.year > 1990) // filter with a predicate
cars.map((d) => d.year)         // map over notes

// time travel
cars.read(note.id, { at: 1 })   // the note as it was at logical time t=1

// full version history of a note
cars.history(note.id)
// [ { t, op: "remember", doc }, ... ]

cars.forget(note.id)            // tombstone (stays in history)

Concepts

• The database is a value, not a mutable blob. Writes never overwrite; they append.
• Two primitives: remember (insert/update) and forget (tombstone).
• Logical time t is a monotonic counter stamped on every write — that's what { at: t } refers to.
• Single writer. Designed for one process (e.g. one container, one backend). No concurrent-writer locking.

See SDK.md for the full API specification.

API

| Operation | Description | | --- | --- | | notedb.use(path) | Open (or create) a database; replays the log into memory | | db.collections.add(name) | Create a collection, return its handle | | db.collections.use(name) | Get a collection handle | | db.collections.drop(name) | Drop a collection (history preserved) | | db.collections.list() | List collection names | | col.remember(doc) | Insert a new note (id generated) | | col.remember(id, doc) | Update an existing note | | col.forget(id) | Tombstone a note | | col.read(id, [opts]) | Read a note (opts.at for time travel) | | col.list([opts]) | List notes (opts.at for time travel) | | col.find(fn, [opts]) | Filter notes with a predicate | | col.map(fn) | Map a function over notes | | col.history(id) | Full chronological version history of a note | | db.compact([opts]) | Compact the log to a snapshot (opts.archive to keep the old log) |

Log format

The database is newline-delimited JSON — one fact per line, human readable:

{ "t": 1, "op": "remember", "collection": "cars", "id": "abc", "doc": { "make": "Volvo", "year": 1992 } }
{ "t": 2, "op": "remember", "collection": "cars", "id": "abc", "doc": { "make": "Volvo", "year": 1993 } }
{ "t": 3, "op": "forget",   "collection": "cars", "id": "abc" }

(Collection lifecycle uses two additional structural ops, mkcol/dropcol, so empty collections and drops survive a reopen.)

Compaction

The log grows with every write and the full history is held in memory. When it gets large, compact it to a snapshot of current state:

db.compact()                                  // discard prior history
db.compact({ archive: "/data/db.archive.log" }) // keep the old log

Building from source

npm install
npm run build   # compiles the Rust core to a native .node addon
npm test

Status & limitations

notedb is young. Known constraints to be aware of:

• Single process / single writer — opening the same file from two processes can corrupt it.
• Synchronous, fsync-per-write — durable, but writes block; best for moderate write rates.
• In-memory history — memory and the log grow until you compact().

License

MIT © Filip Johansson