NEDB

Content-addressed Merkle DAG · Hash-chained · Time-traveling · Bi-temporal · Causally-provable embedded database.

Replay-protected · idempotent · relational · filterable · sortable · searchable · concurrent. One Rust core → ships to PyPI and npm from a single source.

Studio → studio.interchained.org · nedb.aiassist.net

NEDB v2.0.36 — Production Stable

Current stable: 2.0.36 — Cross-platform native wheels shipping nedbd-v2 binary inside pip install nedb-engine. Linux + Windows wheels built on GitHub Actions; macOS arm64 + x86_64 wheels built on Codemagic (M2 Mac Minis). All four platforms publish from one v* tag.

NEDB v2 replaces the append-only log (AOF) with a content-addressed Merkle DAG. Every document version is an immutable, BLAKE2b-verified object. Nothing is ever overwritten. As of v2.0.36, restarts after the first open are O(1) warm starts (driven by a MANIFEST of seq + Merkle head), the cold scan is deferred so the daemon accepts connections immediately, and a new GET /events SSE endpoint streams scan progress + per-write events live.

# Run the v2 DAG engine — ships inside pip install nedb-engine
nedbd --dag --data ./data
# or
NEDBD_DAG=1 NEDB_TMK=<32-byte-hex> nedbd --data ./data

curl http://127.0.0.1:7070/health
# {"ok":true,"version":"2.0.36","service":"nedbd","engine":"dag","startup_ready":true,"encrypted":true}

# Tail the live event stream (new in v2.0.36)
curl http://127.0.0.1:7070/events
# event: scan   data: {"objects":730000,"of":1310703,"rate":21043,"eta_s":28}
# event: ready  data: {"seq":1310703,"head":"b2:9c14e07a…"}
# event: write  data: {"seq":1310704,"coll":"beliefs","head":"b2:7af3c11e…"}

| Property | v2 DAG | v1 AOF | |---|:---:|:---:| | Uncorruptable (atomic writes, hash-verified reads) | ✅ | ⚠️ | | O(1) warm start via MANIFEST (no scan, no replay) | ✅ | ❌ | | Deferred cold scan (socket open immediately) | ✅ | ❌ | | O(1) incremental Merkle head (never recomputed) | ✅ | ❌ | | Parallel writes (no global lock) | ✅ | ❌ | | BLAKE2b Merkle head on every response | ✅ | ❌ | | IdIndex sharded across 256 subdirectories | ✅ | ❌ | | TCP_NODELAY (no 40–200 ms loopback Nagle delay) | ✅ | ❌ | | GET /events SSE log stream | ✅ | ❌ | | Tombstone deletes (history preserved) | ✅ | ✅ | | Auto-migrates v1 AOF → v2 DAG on startup | ✅ | — | | Same HTTP API — Vision, Studio, all clients unchanged | ✅ | ✅ |

v1 AOF engine is still shipped and unchanged — nedbd (no flag) runs v1.

Production status: vision.interchained.org is live on v2.0.36 — 1,310,703 sequences indexed in the Vision database, AES-256-GCM encrypted at rest, at block height 620,989.

What makes NEDB different

Every database stores what. NEDB stores what, when, when it was true, and why — all sealed in a cryptographic hash chain that proves none of it was tampered with.

| Capability | NEDB | SQLite | Redis | MongoDB | |---|:---:|:---:|:---:|:---:| | Hash-chained tamper evidence | ✅ | ❌ | ❌ | ❌ | | Time-travel reads (AS OF seq) | ✅ | ❌ | ❌ | ❌ | | Bi-temporal (VALID AS OF date) | ✅ | ❌ | ❌ | ❌ | | Causal Write Provenance | ✅ | ❌ | ❌ | ❌ | | Replay-protected idempotent writes | ✅ | ❌ | ❌ | ❌ | | SQL + Redis + MongoDB adapters | ✅ | — | — | — | | Concurrent group-commit daemon | ✅ | ❌ | ✅ | ✅ | | At-rest AES-256-GCM encryption | ✅ | ❌ | ❌ | — |

Install

pip install nedb-engine      # Python ≥ 3.8 — pure-Python + optional Rust native wheel
npm install nedb-engine       # Node ≥ 16   — napi-rs prebuilt binaries

Python — 5-minute tour

from nedb import NEDB

db = NEDB("./mydata")          # durable: every op is AOF-logged, fsync'd, and hash-chained
# db = NEDB()                  # or in-memory

db.create_index("users", "status", "eq")
db.create_index("users", "bio",    "search")

db.put("users", "alice", {"name": "Alice", "age": 31, "status": "active", "bio": "rust hacker"})
db.put("users", "bob",   {"name": "Bob",   "age": 24, "status": "active", "bio": "python dev"})

# NQL: WHERE + ORDER BY + LIMIT + SEARCH + TRAVERSE + GROUP BY
db.query('FROM users WHERE status = "active" ORDER BY age ASC')
db.query('FROM users SEARCH "rust"')
db.query('FROM users GROUP BY status COUNT')

# Time-travel — AS OF any past sequence
snap = db.seq
db.put("users", "alice", {"name": "Alice", "age": 32, "status": "retired"})
db.get("users", "alice", as_of=snap)          # → age 31, status active

# Bi-temporal — VALID AS OF any past date
db.put("policy", "rate_2024", {"pct": 5.0}, valid_from="2024-01-01", valid_to="2024-12-31")
db.put("policy", "rate_2025", {"pct": 6.0}, valid_from="2025-01-01")
db.query('FROM policy VALID AS OF "2024-06-15"')   # → rate 5.0

# Causal Write Provenance — why did this write happen?
db.put("inputs", "msg_1", {"text": "user prefers dark mode"})
seq_msg = db.seq
db.put("beliefs", "dark_mode", {"value": True},
       caused_by=[seq_msg], evidence="user_message", confidence=0.95)
db.query('FROM beliefs WHERE _id = "dark_mode" TRACE caused_by')   # → msg_1
db.query('FROM inputs WHERE _id = "msg_1" TRACE caused_by REVERSE') # → dark_mode

# Relations + graph traversal
db.link("users:alice", "follows", "users:bob")
db.query('FROM users WHERE _id = "alice" TRAVERSE follows')

# Hash-chain integrity
assert db.verify()             # cryptographic proof — no tampering

# SQL, Redis, MongoDB compatibility adapters
from nedb import sql_exec, RedisCompat, MongoClient
sql_exec(db, "SELECT * FROM users WHERE status = 'active' ORDER BY age DESC")
r = RedisCompat(db); r.execute("HSET", "user:1", "name", "Alice")
MongoClient(db)["users"].find({"status": "active"}).sort("age", -1).to_list()

Redis layer-2 — wrap_redis()

Already running on Redis? Wrap your connection in one line and gain NEDB features alongside your existing Redis app — no migration required.

import redis, json
from nedb import wrap_redis

r = wrap_redis(redis.Redis("localhost", 6379), db_name="rideshare")

# Step 1 — register: map Redis key globs to NEDB collections (chainable)
(r.nedb
 .register("driver:*", collection="driver", value_parser=json.loads)
 .register("trip:*",   collection="trip",   value_type="hash")
)

# Step 2 — backfill: import all existing Redis data into NEDB in one pass
imported = r.nedb.backfill()           # → int (keys imported)

# Step 3 — shadow: all future r.set/hset/... auto-chain into NEDB
r.nedb.shadow_writes = True

# ─── Alice's app keeps running — zero changes ───────────────────────────
r.set("driver:d1", json.dumps({"name": "Bob", "status": "active"}))   # ← shadowed
r.hset("trip:t1", mapping={"status": "en_route", "driver_id": "d1"})  # ← shadowed

# ─── New features available on the same connection ──────────────────────
r.nedb.query('FROM driver WHERE status = "active" ORDER BY lat ASC')
r.nedb.verify()       # → True  (every write chain-verified)
r.nedb.head()         # → 64-char BLAKE2b commitment hash

Isolation guarantee: NEDB never writes to Alice's namespace. It owns only:

| Key | Type | Purpose | |-----|------|---------| | nedb:{db_name}:oplog | Redis Stream | append-only op log | | nedb:{db_name}:snapshot | Redis Hash | checkpoint | | nedb:{db_name}:meta | Redis Hash | index config |

See examples/fakeredis_demo.py for a full local demo (no Redis server needed).

Node.js

import { NedbCore } from "nedb-engine";

const db = new NedbCore();               // in-memory
// const db = NedbCore.open("./data");   // durable

db.createIndex("users", "status", "eq");
db.put("users", "alice", JSON.stringify({ name: "Alice", age: 31, status: "active" }));

// Time-travel
const snap = db.seq();                   // BigInt
db.put("users", "alice", JSON.stringify({ name: "Alice", age: 32, status: "retired" }));
JSON.parse(db.getAsOf("users", "alice", snap)).age;  // → 31

// Full NQL
const rows = db.query('FROM users WHERE status = "active" ORDER BY age ASC');
rows.map(r => JSON.parse(r));

// Tamper evidence
db.verify();   // → true
db.head();     // → 64-char BLAKE2b commitment hash
db.seq();      // → BigInt

nedbd — the concurrent server daemon

nedbd runs NEDB as a long-lived process with an HTTP/JSON API and an optional RESP2 wire protocol. Built on a single-writer group-commit sequencer — parallel reads, batched durable writes, one hash-chain per database, zero write-write races.

nedbd                                     # :7070, data ./nedb-data (v1 AOF engine)
nedbd --dag --data ./data                 # v2 DAG engine (or NEDBD_DAG=1)
NEDBD_RESP2_PORT=6380 nedbd               # also speak RESP2 (redis-cli compatible)
nedbd --log-level 2                       # 0=errors 1=requests 2=deploy 3=verbose

# Live event stream (new in v2.0.36) — SSE: scan progress, ready, per-write head
curl http://127.0.0.1:7070/events

Startup modes (v2.0.36)

• Warm start — every restart after the first open reads the MANIFEST file and restores seq + Merkle head in O(1). No scan, no replay, independent of dataset size. Boots in milliseconds.
• Cold start — first open of an existing dataset spawns the integrity scan in a background thread and accepts connections immediately. Reads serve instantly from the content-addressed DAG; writes return HTTP 503 startup in progress until the startup_ready gate flips. Progress (objects, rate, ETA) streams over GET /events.

Environment variables

| Variable | Default | Description | |---|---|---| | NEDBD_DAG | 0 | Set 1 to launch the v2 DAG engine (nedbd-v2). Same as --dag. | | NEDBD_HOST | 127.0.0.1 | Bind address. v2.0.36 defaults to loopback (was 0.0.0.0) — security hardening fix. Set explicitly to 0.0.0.0 to expose. | | NEDBD_PORT | 7070 | HTTP bind port. | | NEDBD_TOKEN | unset | Optional bearer token; required on every /v1/* request when set. | | NEDB_TMK | unset | 32-byte hex AES-256-GCM at-rest encryption key. | | NEDBD_DATA | ./nedb-data | Root directory. v2 creates dag/, IdIndex sharded across 256 subdirectories, and a small MANIFEST file. |

# Create a database with seed data and relations
curl -X POST :7070/v1/databases -d '{
  "name": "shop",
  "init": {
    "indexes": [["users","status","eq"]],
    "seed": {"users": [{"_id":"u1","name":"Alice","status":"active"}]},
    "links": [["users:u1","buys","orders:o1"]]
  }}'

# Query (full NQL including time-travel and bi-temporal)
curl -X POST :7070/v1/databases/shop/query \
  -d '{"nql":"FROM users WHERE status = \"active\" ORDER BY name ASC"}'

# Verify the hash chain
curl :7070/v1/databases/shop/verify

# MongoDB-compatible endpoint
curl -X POST :7070/v1/databases/shop/mongo \
  -d '{"collection":"users","op":"find","filter":{"status":"active"},"limit":10}'

From redis-cli — no Redis installation needed:

redis-cli -p 6380 SELECT shop
redis-cli -p 6380 SELECT shop EVAL 'FROM users SEARCH "alice"' 0
redis-cli -p 6380 SELECT shop EVAL 'FROM users AS OF 10 WHERE status = "active"' 0
redis-cli -p 6380 SELECT shop EVAL 'FROM beliefs TRACE caused_by' 0

NQL — the NEDB Query Language

FROM <collection>
  [ AS OF <seq> ]                            transaction time (when was it written?)
  [ VALID AS OF "<date>" ]                   valid time (when was it true in the world?)
  [ WHERE <field> <op> <value> (AND ...) ]   op: = != < <= > >=
  [ SEARCH "<text>" ]                        full-text search
  [ ORDER BY <field> [ASC|DESC] ]
  [ TRAVERSE <relation> ]                    graph traversal
  [ TRACE caused_by [REVERSE] ]              causal provenance (why? / what did this cause?)
  [ LIMIT <n> ]
  [ GROUP BY <field> [COUNT|SUM f|AVG f|MIN f|MAX f] ]

Combine both time axes:

# What did the system know at seq 200 about what was true on 2024-02-15?
db.query('FROM policy AS OF 200 VALID AS OF "2024-02-15"')

Performance

v2 DAG Rust server (v2.0.36, Intel iMac — 10k writes / 100k reads / 30k objects, AES-256-GCM on):

| Operation | Throughput | p50 | p99 | |---|---|---|---| | Sequential writes | 418 ops/s | 2.3 ms | 3.3 ms | | Point-lookup reads | 478 ops/s | 2.0 ms | 3.0 ms | | ORDER BY queries | 489 ops/s | 1.8 ms | 4.3 ms | | Batch writes (500 ops/req) | 1,104 ops/s | 0.9 ms | 1.2 ms | | Tamper-verify (30k objects) | ~21,000 BLAKE2b/sec | — | 1.38 s total |

p99 latencies hold because of TCP_NODELAY on the axum listener — without it macOS loopback adds the Nagle algorithm's 40–200 ms delay on small writes.

v1 Python server (baseline — single-threaded AOF):

| Operation | Throughput | p99 latency | |---|---|---| | Sequential PUT | ~23/s | 44 ms | | Concurrent PUT (16 workers) | ~92/s | 48 ms | | Batch PUT (500 ops/request) | ~520 ops/s | 1.9 ms/op | | Point-lookup read (NQL) | ~23/s | 44 ms | | Rust napi PUT (FFI) | ~70K/s | — | | Rust napi GET (FFI) | ~330K/s | — |

Reproduce with the included benchmark:

NEDBD_DAG=1 nedbd --data /tmp/perf &
python3 tests/test_dag_perf.py --n 10000 --reads 100000

Architecture

            ┌──────────────────────────────────────────────────────────┐
  put/del → │  OpLog  (BLAKE2b hash chain · per-client nonce ·          │ ← single source of truth
  link      │          idempotency keys · causal provenance fields)     │
            └───────────────┬──────────────────────────────────────────┘
            deterministic fold │ (state = pure function of the log)
     ┌──────────────┬──────────┴──────┬───────────────┬────────────────┐
     ▼              ▼                 ▼               ▼                ▼
MVCC store     Relations          Indexes         CauseMap          BlobStore
(time-travel)  (graph+AS OF)      eq/ord/search   (reverse index)   (Cascade CDC)

                     ┌─────────────────────────────────┐
  Thread-safe →      │  Sequencer (group-commit)         │ ← single writer, parallel readers
                     │  — one committer thread/db        │
                     │  — batch fsync                    │
                     └─────────────────────────────────┘

Compatibility adapters:  SQL  ·  Redis  ·  MongoDB
Wire protocols:          HTTP/JSON  ·  RESP2
Encryption:              AES-256-GCM at-rest (TMK/DEK double-envelope)

nedb-client — lightweight HTTP client

Connect to any running nedbd instance from Python or TypeScript without embedding the engine:

pip install nedb-engine-client          # async Python
npm install nedb-engine-client   # TypeScript / Node.js 18+

from nedb_client import NedbClient

async with NedbClient("http://127.0.0.1:7070", db="mydb") as db:
    await db.put("blocks", "618000", {"height": 618000})
    rows = await db.query("FROM blocks ORDER BY height DESC LIMIT 10")
    head = await db.head()    # BLAKE2b Merkle root — changes on every write
    ok   = await db.verify()  # tamper-evidence check across all objects

import { NedbClient } from "nedb-engine-client";
const db = new NedbClient({ url: "http://127.0.0.1:7070", db: "mydb" });
await db.put("blocks", "618000", { height: 618000 });
const rows = await db.query("FROM blocks LIMIT 10");

Repo layout

python/nedb/        reference engine (pure Python — always-works baseline)
rust/
  nedb-core/        v1 production Rust engine (shared by both runtimes)
  nedb-py/          maturin PyO3 binding → PyPI native wheels
  nedb-node/        napi-rs binding → npm native addons
  nedb-v2/          v2 DAG engine (tokio + axum + BLAKE2b DAG)
client/
  python/           nedb-client — async Python HTTP client (pip install nedb-engine-client)
  node/             nedb-client — TypeScript HTTP client  (npm install nedb-client)
tests/              engine + concurrent + causal + bitemporal + deploy + perf benchmarks
examples/           resp2_python.py  resp2_demo.sh
docs/               index.html  reference.html  SPEC.md

Roadmap

• [x] Hash-chained append-only log — tamper evidence, replay protection, idempotency
• [x] MVCC time-travel — AS OF seq
• [x] Bi-temporal — VALID AS OF "date" (transaction time + valid time)
• [x] Causal Write Provenance — caused_by, evidence, confidence, TRACE
• [x] Durable AOF persistence + snapshot checkpoints
• [x] Concurrent group-commit sequencer (nedbd, 15K writes/s under load)
• [x] AES-256-GCM at-rest encryption (TMK/DEK double-envelope)
• [x] SQL / Redis / MongoDB compatibility adapters
• [x] RESP2 wire protocol (redis-cli / redis-benchmark compatible)
• [x] Rust native core — napi-rs (npm) + maturin PyO3 (PyPI)
• [x] Self-healing AOF — auto-truncates corrupt tail on startup, never hangs
• [x] v2 DAG engine — content-addressed Merkle DAG, atomic writes, instant cold start
• [x] nedbd --dag — one flag switches to v2 Rust engine; v1 untouched
• [x] BLAKE2b Merkle head — tamper-evident root on every response
• [x] Tombstone deletes — history preserved in DAG, live id removed from index
• [x] Auto-migration — v1 AOF → v2 DAG on first --dag startup
• [x] nedb-client — async Python + TypeScript HTTP client (pip/npm install nedb-client)
• [x] Intel Mac support — native wheels for aarch64 + x86_64 Apple Darwin
• [ ] In-memory DAG mode — Db::in_memory() for zero-disk ephemeral sessions
• [ ] PyO3 + napi-rs bindings updated to v2 DAG API
• [ ] NEDB Studio DAG mode toggle
• [ ] Merkle inclusion proofs — prove a document existed at a specific time to a third party
• [ ] Git-style branching — fork database state, experiment, merge or discard
• [ ] Agent Memory SDK — Memory.remember() / Memory.recall() / Memory.trace()
• [ ] Live query subscriptions (SSE) — push diffs when query results change

NEDB Studio

Prompt-to-database scaffolding GUI with schema graph, NQL console, time-travel slider, causal provenance panel, and MongoDB/SQL/Redis tabs. Deploy from a description, query live data, edit inline.

studio.interchained.org · github.com/aiassistsecure/nedb-studio (GPLv3)

Repos

| Repo | Description | |---|---| | aiassistsecure/nedb | Source — engine, Rust core, CI | | aiassistsecure/nedb-studio | Studio UI (GPLv3) |

Packages: PyPI nedb-engine · npm nedb-engine

License

See LICENSE file. · © INTERCHAINED, LLC — interchained.org

Authors

Built by Mark Allen Evans Jr. (INTERCHAINED, LLC) with Claude Sonnet 4.6 on Hyperagent.

"Take one idea, turn it into an LP, then an app, then a system, then a platform, then infrastructure that is irreplaceable."