🧰 DenkMitDB

DenkMitDB is a distributed key-value database built on IPFS (Helia), using a Merkle tree as the consistency controller. Every record is a signed, content-addressed block; replicas converge by broadcasting a single root CID over libp2p pubsub and diffing Merkle trees to fetch only what they are missing.

⚠️ Status: experimental, approaching v2. Databases are creator-only by default (only the creating identity may write; publicWrite: true opts into a world-writable database), every merged entry is signature-verified before indexing, records can be deleted via signed tombstones, and a database reopens its own last state without a live peer. Remaining pre-release work (API freeze, hardening) is tracked in ROADMAP.md; open items in KNOWN_ISSUES.md.

🎁 Support: Donate

This project is free, open source and I try to provide excellent free support. Why donate? I work on this project several hours in my spare time and try to keep it up to date and working. THANK YOU!

💡 Features

• Distributed storage: all state lives in IPFS as signed, content-addressed dag-cbor blocks.
• Efficient replication: peers exchange one head CID and Merkle-diff their trees, so sync cost scales with the difference, not the database size.
• Signed writes: every entry is a JWS tied to a self-certifying identity (its CID).
• Access control: a deterministic json-logic policy in the manifest — creator-only by default, world-writable by explicit opt-in — enforced on local writes and merged entries alike.
• Delete: signed tombstones in the same last-write-wins order as puts; a newer write resurrects the key.
• Restart durability: the last head is persisted and re-validated on open, so a node recovers its own state without a live peer.

See ARCHITECTURE.md for how the pieces fit together.

📈 Performance

node scripts/bench.mjs 10000 on one developer machine (Node 24, in-memory stores, single writer, ~40-byte values — orders of magnitude, not SLAs):

| Operation | per-op | ops/s | |---|---:|---:| | set() — sign + index + Merkle rebuild (coalesced) | 4.1 ms | ~245 | | get() warm (cache) | 0.007 ms | ~144,000 | | get() cold (blockstore fetch + JWS verify) | 1.1 ms | ~890 | | keys() full walk (10k keys) | 0.001 ms | ~730,000 | | Reopen from persisted head (10k records, restore + reindex) | 0.55 ms | ~1,800 | | Full replication to a fresh peer (verify + index + rebuild) | 14.3 ms | ~70 |

Bulk replication is the known wall (fetch-latency-bound and superlinear at depth; see ROADMAP.md — batch sync and the persisted materialized index are the follow-ups).

💾 Installation

To set up DenkMitDB, follow these steps:

1. Install module:

   npm install --save @denkmitdb/denkmitdb

2. Install dependencies:

   npm install

🚀 Usage

After installation, you can start using DenkMitDB by following these steps:

1. Import modules:

   import { floodsub } from "@libp2p/floodsub";
   import { noise } from "@chainsafe/libp2p-noise";
   import { yamux } from "@chainsafe/libp2p-yamux";
   import { identify } from "@libp2p/identify";
   import { tcp } from "@libp2p/tcp";
   import { withBitswap } from "@helia/bitswap";
   import { withLibp2pLight } from "@helia/libp2p";
   import * as dagCbor from "@ipld/dag-cbor";
   import { createHeliaLight } from "helia";
   import { createDenkmitDatabase, createIdentity } from "@denkmitdb/denkmitdb";

2. Initialize Helia (with the libp2p and bitswap mixins):

   const helia = withBitswap(
       withLibp2pLight(createHeliaLight({ codecs: [dagCbor] }), {
           addresses: { listen: ["/ip4/0.0.0.0/tcp/0"] },
           transports: [tcp()],
           connectionEncrypters: [noise()],
           streamMuxers: [yamux()],
           services: {
               identify: identify(),
               pubsub: floodsub({ emitSelf: true }),
           },
       }),
   );
   await helia.start();

3. Create new Database Identity and new Database:

   const identity = await createIdentity("user", "password", helia);
   
   const db = await createDenkmitDatabase("test", { helia, identity });
   console.log("Database address: ", db.id);

4. Add new data to Database:

   await db.set("key1", { value: "value1" });
   await db.set("key2", { value: "value2" });
   
   for await (const e of db.iterator()) {
       console.log(e);
   }

5. Retrieve data from Database:

   const value1 = await db.get("key1");
   console.log("Value 1: ", value1);

6. Close Database

   await db.close();
   await helia.stop(); // also stops the embedded libp2p

📚 Documentation

| Document | Contents | |---|---| | ARCHITECTURE.md | Data model, the pollard Merkle tree, write/read paths, sync protocol, trust model | | KNOWN_ISSUES.md | Verified bugs (several pinned by failing tests) and open design concerns | | ROADMAP.md | Where the project is going: spec → correctness → upgrades → features → v2.0.0 | | VISION.md | Post-v2 strategy: verifiable shared memory for AI agents — the MCP server as the product | | specs/ordering.md | Accepted v2 spec: composite sort key, last-write-wins, format versioning | | CHANGELOG.md | Release history | | CODEX_REVIEW.md | Independent adversarial review of the Phase 0 safety net (July 2026) | | PHASE_PRIORITIES.md | Independent prioritization review of the remaining Phase 4 work (July 2026) | | mcp/ | MCP server: DenkMitDB as shared, signed agent memory for Claude Code/Codex/any MCP client | | docs/ | Generated API reference (typedoc) |

🛠️ Development

corepack enable        # provides the pinned pnpm version
pnpm install
pnpm test              # vitest: unit + integration (real libp2p nodes over TCP)
pnpm lint              # eslint over src, test, examples and configs
pnpm typecheck         # tsc over tests/configs (tsconfig.test.json)
pnpm build
pnpm test:package      # packs the tarball and smoke-imports the packed code

Notes:

• Tests marked it.fails document known bugs (see KNOWN_ISSUES.md); when you fix one, flip its test to a normal it.
• Requires Node 22+ (helia 5 uses Promise.withResolvers). CI runs lint, typecheck, build, tests, and the package smoke test on Node 22 and 24 for every push and pull request.

👨‍💻 Contributing

We welcome contributions! Please fork the repository and submit pull requests. For major changes, please open an issue to discuss what you would like to change. Good starting points are the items in KNOWN_ISSUES.md with test pins.

💫 License

This project is licensed under the MIT License. See the LICENSE file for details.

🦄 Contact

For more information, please contact the project maintainer at [email protected].