logboat

A small, pragmatic raft for Node.js — built to run real replicated state machines on modest hardware, and to be embedded rather than deployed.

logboat was extracted from a working sharded transaction system where it replicates state machines at thousands of commits per second on consumer NVMe drives. Its design bias is operational honesty over feature count: a small surface you can read in an afternoon, storage engineered for the write patterns raft actually produces, and deliberate scope limits documented up front.

What you get

• RaftNode — the consensus core. Byte-payload proposals (propose(Buffer)), an ordered apply loop, snapshot/restore callbacks, leadership notifications, and per-peer replication with fast conflict backoff.
• Catch-up-gated leadership — a new leader is announced to your app only after its state machine has applied everything committed before the election, so you never serve stale reads from a "leader" that is still replaying.
• Pluggable log storage (LogStore):
  • SegmentLogStore (default) — a preallocated segment-file WAL with ~10-25x lower device write amplification than an LSM store under synced append load. A synced append is one pure data write: no compaction, no filesystem-journal traffic. Purge unlinks whole segments. Built for consumer SSDs whose SLC caches choke on sustained tens-of-MB/s.
  • LevelLogStore — classic-level (LevelDB) backed alternative.
  • MemoryLogStore — for tests and ephemeral state machines.
• Pluggable transport (RaftTransport): a built-in length-prefixed TCP transport (request-id correlated, lazy redial, timeout teardown), and an in-memory network (MemoryRaftNetwork) with partition injection for fast deterministic tests.
• Typed errors — NotLeaderError (carries the current leader id for redirects), ReplicationLagError, RaftStoppedError.
• Injectable logging — a structural, pino-compatible RaftLogger; silent by default.

Deliberate scope (read this before adopting)

These are design decisions, not roadmap gaps:

• Single-term leader semantics, no pre-vote. A partitioned node can bump terms and force an election on rejoin. Election windows default to ~1-2s (etcd-style) so real network + fsync latency never causes storms.
• Static membership. The cluster is fixed at construction (peers); there is no joint-consensus reconfiguration.
• Single-blob InstallSnapshot. Snapshots ship as one message; size your state machine snapshots accordingly (or return an empty buffer from buildSnapshot() to opt out of snapshotting and rely on log replay — that contract is supported and tested).
• Snapshots pause the apply loop rather than forking state.
• Peer wire format may change in 0.x minors — restart clusters together when upgrading. On-disk formats (segment WAL, LevelDB entries, snapshot blobs) are stable.

Quick start

import { RaftNode, SegmentLogStore, DEFAULT_RAFT_PARAMS } from "logboat";

const peers = [
  { nodeId: 0, host: "10.0.0.1", raftPort: 16001 },
  { nodeId: 1, host: "10.0.0.2", raftPort: 16001 },
  { nodeId: 2, host: "10.0.0.3", raftPort: 16001 },
];

const node = new RaftNode({
  nodeId: 0,
  peers,
  logStore: new SegmentLogStore("data/raft-log"),
  params: DEFAULT_RAFT_PARAMS,
  apply: async (logIndex, term, isLeader, payload) => {
    // Apply the committed payload to your state machine.
    // The return value is surfaced to the proposer on the leader.
    return myStateMachine.apply(payload);
  },
  buildSnapshot: () => myStateMachine.serialize(),
  installSnapshot: (data) => myStateMachine.restore(data),
  snapshotDir: "data/raft-snapshots",
});

await node.start();

node.onLeadership((leader) => {
  // Gate your request serving on this — it fires true only once caught up.
});

// On the leader:
const { logIndex, response } = await node.propose(encodeMyCommand(cmd));

Redirect handling on followers:

import { NotLeaderError } from "logboat";

try {
  await node.propose(payload);
} catch (e) {
  if (e instanceof NotLeaderError) redirectClientTo(e.leaderNodeId);
  else throw e;
}

Storage: why the segment WAL exists

Raft log storage has a peculiar shape: append at the head, purge from the tail after each snapshot, read only the recent window. LSM stores handle this badly under synced load — every group-commit fsync rewrites a WAL block and commits the filesystem journal, and compaction endlessly rewrites a key range that is being deleted from behind. Measured on real hardware, that turned ~1.3MB/s of logical raft data into ~34MB/s of device writes — enough to exhaust a consumer QLC SSD's SLC write cache in minutes and put the drive into periodic write-stall cycles.

SegmentLogStore writes into preallocated, zero-filled segment files, so a synced append touches only already-allocated data blocks (fdatasync, no journal metadata). Purge unlinks dead segments. The unpurged window also lives in memory (bounded by your snapshot interval), so reads never touch disk after recovery.

License

MIT