@snaha/swarm-pot

Pure-TypeScript Proximity Order Trie (POT) for Swarm — a WASM-free port of potjs that's wire-compatible with the Go reference implementation.

This package is a derivative work of potjs by @brainiac-five and the Go proximity-order-trie by Ethereum Swarm. See NOTICE for full attribution.

Why

potjs ships a ~2 MB WASM blob wrapping the Go POT. That adds a native-module load on Node and a network fetch in the browser, and makes debugging a black box. The algorithm is small enough to port directly, and doing so gets us:

• One language end-to-end; stack traces point at real TS source.
• No vendored pot-node.js + pot.wasm, no createRequire dance.
• Freedom to change serialisation, add new persisters, or extract sub-algorithms (e.g. for LSM-backed scale).

Refs produced by either implementation are interchangeable — you can write with @snaha/swarm-pot and read with potjs, or vice versa.

Install

pnpm add @snaha/swarm-pot
# or: npm i @snaha/swarm-pot

Requires Node ≥ 18 (uses Web Crypto + native fetch).

API

import { Index, SingleOrder, SwarmKvs, BeeLoadSaver, createPotCompat } from '@snaha/swarm-pot'

// Low level
const idx = Index.create(new SingleOrder(256))
await idx.add({ key: () => k /* , ... Entry interface */ })

// High level (Bee-backed key/value store)
const ls = new BeeLoadSaver({ beeUrl, postageBatchId })
const kvs = SwarmKvs.create(ls)
await kvs.put(key32, value)
const ref = await kvs.save() // Uint8Array(32)
const reader = await SwarmKvs.fromReference(ls, ref)

// potjs-compat: drop-in for `globalThis.pot.new` / `pot.load`
const pot = createPotCompat()
const k = await pot.new(beeUrl, batchIdHex)
await k.putRaw(42, refBytes) // number / string / Uint8Array keys
const refHex = await k.save() // hex string, matches WASM

Layout

• src/elements/ — algorithm core: MemNode, SwarmNode, SingleOrder / SwarmPot modes, Find / Update / Iterate, the Wedge / Whirl / Whack ops.
• src/persister/ — LoadSaver abstraction, InmemLoadSaver (SHA-256, tests), BeeLoadSaver (/bytes HTTP).
• src/index.ts — Index (mutable POT with async write-mutex), public exports.
• src/kvs.ts, src/swarm-entry.ts — high-level SwarmKvs on top of Index.
• src/compat.ts — drop-in globalThis.pot shim matching the potjs/WASM surface.

Key findings from the port

1. JS signed modulo corrupts the bitmap for PO ≥ 8. The Go SwarmNode.MarshalBinary sets bits with 1 << ((7 - n) % 8) where n is uint8, so the subtraction wraps into [0, 255] before the modulo. In JS (7 - 12) % 8 === -5 and 1 << -5 shifts left by 27 — bits land in the wrong byte. Decoding uses the complementary (i % 8) form and happened to be right, so the bug only surfaced on save+reload with close-PO keys. Fix in src/elements/swarm-node.ts uses 1 << (7 - (n % 8)). Regression: test/close-po.integration.test.ts.

2. iterate() needs a defensive unpack the Go version skips. Go's iterate recurses into Slice sibling forks without unpacking them. On in-memory trees that's fine (forks keep their MemNode after Pack); on trees freshly loaded from a reference, those siblings are packed and Empty() would panic. This port calls mode.unpack at the top of each recursion. No-op cost on SingleOrder / already-loaded nodes.

3. Wire format: Bee's /bytes, not /chunks. Matches the Go SwarmLoadSaver. Bee handles chunk splitting internally and returns a 32-byte ref regardless of payload size, so the POT doesn't need a chunk-size policy. POST body = raw data, header swarm-postage-batch-id; expect HTTP 201 with {"reference": "<64 hex>"}.

4. Key/value coercion is exactly the Go jsToKey / typeEncodedBytes. Numbers → 8-byte BE IEEE-754, strings → UTF-8, Uint8Array ≤ 32 bytes; all right-padded with zeros to 32. Typed values (put/get) prepend a 1-byte tag: 0 null, 1 boolean, 2 number, 3 string, 4 bytes. Raw values (putRaw/getRaw) are stored verbatim.

5. Empty-KVS save/load needs a potjs-specific sentinel. Go's Index.Save throws root node is nil when the pot has no entries. The potjs/WASM wrapper catches that and returns 32 zero bytes; on load(ref) with all-zeros it creates a fresh KVS instead of fetching. Consumers depending on this behaviour (e.g. an empty-block index) need it. The core Index stays spec-faithful (throws); the translation lives on the compat seam, matching potjs.go _save / _load.

Status vs the WASM it replaces

Proven compatible, both directions, against a live Bee:

• TS writer → WASM (potjs) reader ✓
• WASM (potjs) writer → TS reader ✓
• Compat-layer writer → WASM reader ✓ (putRaw(numericKey, refBytes) + hex-ref save())
• WASM writer → compat reader ✓
• All five typed-value shapes (null / bool / number / string / Uint8Array) round-trip ✓

Not yet covered:

• BMT hashing for InmemLoadSaver — uses SHA-256. Good enough for in-memory tests; irrelevant when BeeLoadSaver is the persister since Bee computes refs.
• In-memory mode for pot.new() / pot.load() (no beeUrl / batchId). Current compat requires both.
• Proof generation (pkg/proof/ in the Go ref) — separate slice.
• Index.Iterate with prefix on reloaded trees: works in our tests, but the Go impl has latent gaps here that the test suite doesn't exercise. Worth re-auditing if we start using prefix iteration over persisted pots.

Testing

pnpm test                                                # offline tests
BEE_URL=http://127.0.0.1:1633 BEE_STAMP=<id> pnpm test   # + integration tests

Integration tests auto-skip without env. Spin up a local Bee with @snaha/bee-compose (or your own stack) and grab a postage batch id.

License

Apache-2.0. See NOTICE for upstream attribution.