@nxtedition/cache

A two-tier async cache with SQLite persistence, in-memory pseudo-LRU, stale-while-revalidate, best-effort cross-thread duplicate-work suppression, and automatic request coalescing.

Features

• Two-tier storage — In-memory cache backed by SQLite on disk
• File-sharded SQLite — Keys are hash-routed across N independent SQLite files, bypassing SQLite's per-file writer serialization. ~50% higher throughput under multi-thread write contention (see Benchmarks).
• Stale-while-revalidate — Serve stale data synchronously while refreshing in the background
• Request coalescing — Concurrent fetches for the same key share a single in-flight Promise
• Cross-thread locking — SharedArrayBuffer + Atomics.compareExchange / Atomics.waitAsync normally reduce redundant valueSelector calls across worker threads in the same process that share the same location
• Async value resolution — Transparently fetches missing values via a user-defined valueSelector
• Binary support — Store and retrieve Buffer / Uint8Array alongside JSON values
• Size-bounded storage — Configurable max database size with automatic eviction of oldest entries
• Custom serialization — Pluggable serialize/deserialize for non-JSON value types
• Batched writes — SQLite writes are coalesced into transactions via setImmediate, reducing I/O
• Disposable — Implements Symbol.dispose for use with using declarations

Usage

import { Cache } from '@nxtedition/cache'

const cache = new Cache(
  './my-cache.db', // SQLite file path, or ':memory:'
  async (id: string) => {
    const res = await fetch(`https://api.example.com/items/${id}`)
    return res.json()
  },
  (id: string) => id, // keySelector: derive cache key from arguments
  {
    ttl: 60_000, // 60 s before value is considered stale
    stale: 30_000, // serve stale for 30 s while revalidating
  },
)

const result = cache.get('item-123')

if (result.async) {
  // Cache miss — value is being fetched
  const value = await result.value
} else {
  // Cache hit — value returned synchronously
  const value = result.value
}

Using Symbol.dispose

{
  using cache = new Cache('./tmp.db', fetchItem, (id) => id, { ttl: 5_000 })
  const result = cache.get('key')
  // ...
} // cache.close() called automatically

API

new Cache(location, valueSelector?, keySelector?, opts?)

| Parameter | Type | Description | | --------------- | ---------------------------------- | ---------------------------------------------------------------------------- | | location | string | Non-empty SQLite path, resolved to absolute at construction, or ':memory:' | | valueSelector | (...args) => V \| PromiseLike<V> | Function to fetch a value on cache miss | | keySelector | (...args) => string | Function to derive a cache key from arguments | | opts | CacheOptions<V> | Optional configuration |

keySelector must return a non-empty string. Database-backed or cross-thread-locked caches also require well-formed Unicode because keys are UTF-8 encoded for hashing and SQLite. A pure-memory cache does not encode its keys and accepts any JavaScript string, avoiding an unnecessary Unicode scan on its miss path.

CacheOptions

| Option | Type | Default | Description | | ------------ | ---------------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ttl | number \| (value, key) => number | MAX_SAFE_INTEGER | Time-to-live in milliseconds. After this, the entry is stale. | | stale | number \| (value, key) => number | MAX_SAFE_INTEGER | Stale-while-revalidate window in ms. After ttl + stale, the entry is purged. | | memory | MemoryOptions \| false \| null | { maxSize: 16MB, maxCount: 16384 } | In-memory cache config, or false/null to disable. | | database | DatabaseOptions \| false \| null | { timeout: 10, maxSize: 128MB } | SQLite config, or false/null to disable persistence. | | lock | boolean \| null | enabled | Best-effort cross-thread duplicate-work suppression (see Cross-Thread Locking); not an exactly-once primitive. false/null disables it; true force-enables it. An enabled lock with a disabled database tier — or true at :memory: — throws. | | serializer | Serializer<V> | JSON + ArrayBufferView passthrough | Custom { serialize, deserialize } for value encoding. |

MemoryOptions

| Option | Type | Default | Description | | ---------- | -------- | -------------------------- | ---------------------------------------------- | | maxSize | number | 16 * 1024 * 1024 (16 MB) | Maximum total size in bytes of cached entries. | | maxCount | number | 16 * 1024 (16384) | Maximum number of entries in memory. |

The in-memory tier uses a random-two-choice eviction strategy: when the cache is full, two entries are sampled at random and the least recently accessed one is evicted. This provides near-LRU behavior with O(1) eviction cost.

DatabaseOptions

| Option | Type | Default | Description | | ----------- | --------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | timeout | number | 10 | SQLite busy timeout in milliseconds. Must be an integer from 0 through 2147483647. | | maxSize | number | 128 * 1024 * 1024 (128 MB) | Maximum total database size across all shards. Divided evenly per shard. Oldest entries are evicted when a shard is full. | | shards | number | 4 (1 for ':memory:') | Number of SQLite files (shards) the cache is spread across. Must be an integer in the inclusive range 1–32 (values outside it throw). ':memory:' supports only 1 (any other value throws). See File Sharding. Use 1 for a single shard. | | writeLock | boolean | true | Best-effort per-shard flush serialization across threads to avoid SQLITE_BUSY. See Write Lock. |

Serializer<V>

| Method | Signature | Description | | ------------- | ---------------------------------------------- | --------------------------- | | serialize | (value: V) => Buffer \| Uint8Array \| string | Encode a value for storage. | | deserialize | (data: Buffer \| string) => V | Decode a stored value. |

The default serializer passes ArrayBufferView values through as-is and uses JSON.stringify/JSON.parse for everything else.

Round-trip asymmetry: only Buffer/Uint8Array values round-trip faithfully through the default serializer. Any other view type (Float64Array, DataView, …) is stored as raw bytes and comes back as a Buffer — but only when the read is served from the database tier (after memory eviction, gc(), or a process restart); reads served from the memory tier return the original object. If you store non-Uint8Array views, provide a custom serializer that reconstructs the concrete type.

CacheResult<V>

Both get() and peek() return a CacheResult<V>, a discriminated union on the async property:

| async | value | Meaning | | ------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | false | V \| undefined | Cache hit — the value is available synchronously. Also returned for stale entries (a background refresh is triggered). undefined when peek() has no cached entry. | | true | Promise<V> | Cache miss — value is a Promise that resolves when the valueSelector completes. |

const result = cache.get('key')

if (result.async) {
  const value = await result.value // miss — await the fetch
} else {
  const value = result.value // hit (fresh or stale) — use directly
}

Methods

cache.get(...args): CacheResult<V>

Returns a cached value or triggers a fetch on cache miss. If the entry is stale and the valueSelector is async, returns the stale value synchronously (async: false) while a background refresh runs. If the valueSelector throws during a stale revalidation, the error is emitted and the stale value is preserved.

cache.peek(...args): CacheResult<V>

Same as get() but does not trigger a refresh on cache miss or stale entry. Returns { value: undefined, async: false } for missing/expired entries, or the stale value if within the stale window.

cache.refresh(...args): CacheResult<V>

Forces a new fetch via valueSelector regardless of cache state. Unlike get(), concurrent refresh() calls for the same key do not deduplicate — each call invokes the valueSelector. However, get() calls during a pending refresh() will return the in-flight promise.

cache.delete(...args): void

Remove an entry from both memory and SQLite. Also cancels any in-flight deduplication for that key — a pending fetch will still resolve for its callers, but the result is not written to the cache.

The SQLite DELETE is retried with a short in-thread backoff (~16 × 5 ms) if another connection holds the write lock. If contention outlasts the budget, the failure is surfaced via the error event and the row may survive in the database — meaning a later read can resurrect the value. Listen for error if delete-for-correctness matters to your workload.

cache.gc(): Promise<void>

Remove all expired entries (past ttl + stale) from both the in-memory cache and SQLite, and run PRAGMA wal_checkpoint(TRUNCATE) + PRAGMA optimize. File-backed maintenance runs in a worker; await the returned promise to know when it has completed. Errors are emitted and the promise resolves rather than rejecting.

cache.clear(): void

Empty the cache entirely: drains any pending batched writes, cancels all in-flight deduplication, clears the in-memory tier, and runs DELETE FROM on every SQLite shard. Unlike gc() (which removes only expired entries), clear() removes everything regardless of TTL/stale state. In-flight fetches still resolve for their callers but their results are not written back to the cache. See also the nxt:clearCache broadcast.

As with delete(), SQLite work is retried with a bounded backoff when another connection holds the write lock. If contention outlasts that budget, the failure is surfaced via the error event and affected database rows may survive even though the in-memory tier has been cleared.

cache.flushSync(): void

Synchronously drain all pending batched writes to SQLite. The cache remains open and usable afterwards. This is useful when you need to guarantee persistence at a specific point without closing the cache (e.g. before handing off to another cache instance that shares the same database).

cache.close(): void

Calls flushSync() to drain pending writes, then closes the SQLite database and releases resources. Clears cached values, cancels in-flight deduplication, and immediately releases owned key locks. Operations after close() throw.

Also available as [Symbol.dispose]() for use with using declarations.

Open caches are automatically closed on the beforeExit event, ensuring pending writes are flushed before the process exits.

cache.stats

Returns runtime statistics:

{
  hits,
  misses,
  dedupe: { size },
  memory: { size, maxSize, count, maxCount } | undefined,
  database: { location, size } | undefined,
}

Deduplication

Within one Cache instance, ordinary overlapping calls to get() for the same key share a single in-flight Promise. In that normal path, the valueSelector is called only once:

// valueSelector is called once, both promises resolve to the same value
const [a, b] = await Promise.all([cache.get('key').value, cache.get('key').value])

If a fetch fails, the deduplication entry is cleaned up and subsequent calls retry.

Calling cache.delete(key) while a fetch is in-flight invalidates the deduplication entry. The pending promise still resolves for its callers, but the result is not written to the cache.

refresh() does not deduplicate with itself — each call starts a new fetch. On an otherwise-missing key, get() calls see the most recent pending promise; fresh and stale hits intentionally keep their synchronous behavior.

To keep misses cheap without retaining an unbounded per-key generation map, each instance remembers a bounded window of recent invalidations and explicit refreshes. A newer same-key refresh() supersedes older fills, while same-key delete() and any clear()/close() prevent an older operation from repopulating memory or SQLite. If an unusually slow operation spans more than 256 such events, its result is conservatively returned to its caller but not cached.

Cross-thread coordination is deliberately weaker. It reduces duplicate work in normal operation, but timeout recovery, worker termination, or a rare release/takeover race may run valueSelector more than once. Selectors must be safe to repeat; do not use the cache lock as an exactly-once execution primitive.

Stale-While-Revalidate

When an entry's TTL has expired but is still within the stale window, get() returns the stale value synchronously (async: false) and normally triggers a background refresh (when the valueSelector is async). If another worker currently owns the advisory key slot, the stale value is returned without parking another background waiter; a later read retries after the slot becomes available. If a refresh fails, the stale value is preserved.

Once the stale window expires, the entry is purged entirely and the next get() returns async: true.

|--- ttl ---|--- stale ---|
    fresh        stale      expired
      ↓            ↓           ↓
  sync hit     sync hit    async miss
             + bg refresh

Cross-Thread Locking

Worker threads in the same process that pass the same resolved location to new Cache(...) share a 256 KiB SharedArrayBuffer (64K Int32 slots, acquired via @nxtedition/shared's getOrCreate registry). That buffer is treated as a hash table of small per-key advisory slots (0 = free, 1/2 = held states):

• Acquire: Atomics.compareExchange(slot, 0, 1). The winner normally runs valueSelector; losers call Atomics.waitAsync and then read the value the winner persisted.
• Timeout recovery: After 5 seconds, a waiter may replace the apparently orphaned held state (1 → 2 or 2 → 1) in one CAS. The threshold is intentionally well beyond ordinary scheduling/network jitter, so a healthy selector should rarely encounter it. Simultaneous timeout callbacks still normally select one replacement producer without unique ownership tokens. A sufficiently late original holder/release can race a later reuse and duplicate the work.
• Release: The holder compares its small held state back to 0 and notifies waiters. Within one Cache, a local object-identity lease prevents an old operation from releasing its replacement. Across instances, a late release can still race with a later reuse and temporarily weaken deduplication, but it cannot make the slot drift or remain permanently stuck.
• Exception and invalidation safety: Throws, rejections, delete()/clear()/close(), and serializer/ttl/stale failures release locally owned slots. Timeout recovery handles a terminated or never-settling owner that cannot run cleanup.

The SAB is a contention-reduction mechanism, not the cache's correctness boundary. Returned values, invalidation, and lifecycle cleanup do not rely on exactly one worker holding the slot. SQLite transactions remain authoritative for persisted writes.

lock is enabled by default, false/null disables it, and true force-enables it. An enabled lock needs a real, shared on-disk database to coordinate through, so an enabled lock (the default or an explicit true) throws at construction when the database tier is disabled — and lock: true likewise throws at ':memory:'.

':memory:' is inherently per-instance: each ':memory:' open is its own private SQLite connection, not shared even between two instances in the same thread, so there is nothing for the SAB lock to coordinate through. As a convenience, an unspecified lock at ':memory:' is silently treated as disabled (so the common ephemeral-cache path need not spell out lock: false); passing lock: true there is instead rejected, since it can never be honored.

Pass { lock: false } (or lock: null) to opt out of the SAB lock on any cache — the instance-local #dedupe Map still coalesces concurrent get() calls on the same key, but sibling threads hitting the same DB file will each call valueSelector independently.

There is no cross-process coordination — two separate Node processes pointed at the same SQLite file may both run valueSelector for the same key. If you need cross-process dedup, do it at a layer above (e.g. a request-coalescing service).

When to disable the lock

• ':memory:' — already disabled for you when lock is left unspecified; nothing to configure (and lock: true there throws).
• A disabled database tier (database: false/null) — required: the lock has nothing shareable to coordinate through, so construction throws unless you pass lock: false/null.
• Single-threaded app that doesn't spawn workers — disable to skip the cheap SAB round-trip on every get() cache miss.
• Workload where each worker uses a disjoint keyspace — the SAB adds overhead without any dedupe benefit.

File Sharding

The cache partitions its SQLite storage across N physical files (default shards: 4). Each key is hash-routed (via xxhash32) to one shard, and reads/writes for that key only touch that shard's connection. This is intended to reduce the SQLite writer-serialization ceiling: SQLite allows only one writer at a time per database file, even in WAL mode, so a single file caps concurrent-writer throughput at roughly one thread's worth of work regardless of how many threads the process spawns.

On-disk layout: the layout is uniform across shard counts — every shard file embeds its index and the shard count.

| shards | Files | | ----------- | ---------------------------------------------------------------------------------- | | 1 | <location>.0_1 | | N (N ≥ 2) | <location>.0_N, <location>.1_N, …, <location>.{N-1}_N (shard count embedded) |

There is no special bare-<location> single-file form; shards: 1 writes <location>.0_1.

Cross-instance consistency: hash routing is deterministic (xxhash32 is pure), so two Cache instances pointing at the same location with the same shards count will route the same keys to the same shards. Data persists across instance restarts.

Changing shards discards old data. Shard filenames embed the shard count, so opening a cache with shards: 2 after previously using shards: 4 starts from a fresh, empty file set. The constructor reclaims the stale file set of the previous configuration by clearing each file (dropping its cache tables via SQLite) and then unlinking it. Clearing first means switching back to the old count can never resurrect entries that delete()/clear() invalidated in the meantime — even if the unlink cannot remove the file — and, against a concurrently-live instance of that mismatched count, dropping its table is a safe cache flush (it re-populates from the valueSelector) rather than file-level corruption.

Still, do not point two live instances with different shard counts at the same location: key routing and the per-location cross-thread lock both assume a single shard count per path, and on POSIX the reclaiming unlink can still ghost a live mismatched instance after its cache is flushed. A leftover bare <location> file written by a pre-uniform-layout version is never read and simply leaks — delete it manually if you need the space.

Per-shard state: maxSize divides evenly across shards (maxSize / shards per shard), eviction runs per-shard on SQLITE_FULL, and stats.database.size is summed across all shards.

Batched Writes

SQLite writes are batched using setImmediate — multiple set() calls within the same microtask turn are coalesced into a single BEGIN/COMMIT transaction per shard. While a write batch is pending, the in-memory cache is corked (eviction deferred) to avoid dropping entries before they reach disk. If a per-shard batch exceeds 512 items, it is flushed immediately.

Flushes iterate shards starting at a random index on each call, so no single shard is starved when the 10 ms per-flush time budget is exhausted mid-pass — the next flush picks a different starting shard.

If a shard's database is full (SQLITE_FULL), the cache adaptively halves the transaction until a prefix fits. When one item still needs space, it evicts old entries in bounded 256-row chunks until the item fits or no rows remain. An item that cannot fit even in the emptied shard is dropped from persistence without discarding later writes, and the error is emitted. Learned prefix sizes and the 10 ms async budget keep event-loop stalls bounded; on other errors, the entire pending batch for that shard is dropped (items remain in the in-memory cache until natural eviction/TTL) to prevent error floods from persistent failures such as a read-only database.

Write Lock

SQLite serializes writers: only one connection may write a given shard file at a time, even in WAL mode. When multiple worker threads in the same process flush the same shard concurrently, the losers' BEGIN IMMEDIATE returns SQLITE_BUSY and the flush falls into a retry loop (wasted BEGIN/ROLLBACK work, jittered re-arming, and synchronous busy_timeout stalls).

With database.writeLock (default true), each shard has an advisory lock in a small SharedArrayBuffer — a single Int32 per shard (0 = free, otherwise the holder's acquire timestamp), each padded to its own cache line to avoid false sharing. Before a flush opens its transaction it normally acquires the shard's lock; a thread that finds the lock held defers its flush (re-arming shortly) instead of attempting a doomed transaction. This reduces optimistic-collide-and-retry work and improves multi-thread write throughput.

• Threads only. The SharedArrayBuffer is shared within a process, not across processes — separate processes writing the same files still rely on timeout/retry (see Error Handling).
• Best-effort, not exclusive. If a holder is terminated mid-flush, a contender that finds the slot held longer than max(2 × timeout, 50 ms) reclaims it. A false stale judgment or release/takeover race may briefly admit two flushers. SQLite's own BEGIN IMMEDIATE + busy_timeout is the correctness guard; the loser defers or retries normally.
• Independent of lock. The two mutexes protect different things — deduplicating valueSelector calls vs. serializing SQLite write transactions — and neither requires the other. writeLock works the same whether lock is enabled or disabled.
• Works at :memory: too. Every :memory: cache in the process shares one write-lock SharedArrayBuffer by location, so an unrelated sibling's flush can defer this one's — harmless, self-healing false contention, not a correctness concern. The synchronous drain (flushSync()/close()/clear()) bypasses the mutex entirely and keeps the existing optimistic + retry path.

Set database: { writeLock: false } to opt out.

Error Handling

Cache extends EventEmitter. Non-fatal errors (SQLite failures, stale revalidation failures, background-refresh rejections from the SWR fire-and-forget path) are normalized to Error objects and emitted as 'error' events when a listener is attached. If no 'error' listener is registered, errors are surfaced via process.emitWarning() instead, avoiding unhandled crashes even when user code rejects with a non-Error value.

Off-Peak Purge

All cache instances listen on the nxt:offPeak BroadcastChannel. When a message is received, gc() is called on every active instance, enabling coordinated cleanup during low-traffic periods.

Cross-Instance Invalidation

All cache instances also listen on the nxt:clearCache BroadcastChannel. When a message is received, clear() is called on every active instance (memory + every SQLite shard), so an invalidation broadcast in any thread of the process requests a wipe of all caches in that process. SQLite failures are surfaced as described under clear(). As with the off-peak channel, the listener is unref()'d and errors from a misbehaving instance are isolated (emitted on that instance's 'error' event, or surfaced via process.emitWarning()).

Benchmarks

Measured on Apple M3 Pro (12 CPUs), Node 25.6.1. Throughput is ops/sec; latency is ns (median).

Single-thread hot paths

| Operation | ops/sec | p50 | p99 | | ------------------------------------------- | ------- | ------ | ------- | | get() memory hit (sequential keys) | 4.90 M | 125 ns | 666 ns | | get() memory hit (random keys) | 6.05 M | 125 ns | 584 ns | | peek() memory hit | 8.05 M | 125 ns | 250 ns | | get() memory miss, DB hit (sequential) | 303 K | 2.3 µs | 10.6 µs | | get() memory miss, DB hit (random) | 399 K | 2.3 µs | 6.4 µs | | get() cold (sync valueSelector) | 274 K | 1.8 µs | 4.5 µs | | get() memory-only hit (no DB) | 5.39 M | 125 ns | 500 ns | | get() memory-only cold (no DB) | 1.16 M | 708 ns | 2.3 µs | | get() eviction pressure (maxCount=1000) | 1.64 M | 542 ns | 1.5 µs | | delete() existing keys | 67 K | 13 µs | 44 µs | | gc() 10 K expired entries | 7.2 ms | — | — |

Shard-count comparison

Single-thread overhead of sharding on the hot paths is ≤5 % in either direction. The pay-off is under multi-thread write contention:

12 threads, partitioned cold writes (each worker writes unique keys, stressing concurrent writers):

| shards | Aggregate throughput | Scaling vs. 1 thread | Δ vs. shards: 1 | | -------- | -------------------- | -------------------- | ----------------- | | 1 | 154 K ops/s | 0.80× | baseline | | 2 | 223 K ops/s | 1.04× | +45 % | | 4 | 237 K ops/s | 1.37× | +54 % | | 8 | 182 K ops/s | 1.08× | +19 % |

12 threads, shared-keys hot-hit (mostly memory-resident; sharding shouldn't help much here):

| shards | Aggregate throughput | | -------- | -------------------- | | 1 | 7.69 M ops/s | | 2 | 7.86 M ops/s | | 4 | 7.70 M ops/s | | 8 | 7.51 M ops/s |

So shards: 4 (the default) is a large win for write-heavy multi-threaded workloads and roughly break-even otherwise. If you run single-threaded or strictly read-heavy, shards: 1 removes all sharding overhead and restores the single-file on-disk layout.

Lock option overhead (single-thread)

| Path | lock enabled | lock disabled | | --------------------------------- | ------------ | ------------- | | get() cold (sync valueSelector) | 298 K ops/s | 306 K ops/s | | get() memory hit | 7.44 M ops/s | 6.19 M ops/s |

Cross-thread SAB locking adds ~3 % on the cold path and is negligible on memory hits. It is worth leaving on unless the workload is strictly single-threaded or partitioned across workers.

Reproducing

yarn build
node scripts/bench.mjs              # full bench suite
node scripts/bench-shards.mjs       # shard-count comparison (1, 2, 4, 8 shards)

Scripts

yarn test               # run tests
yarn test:types         # run the public TypeScript contract tests
yarn test:coverage      # run tests with a coverage report
yarn typecheck          # type-check without emitting
yarn build              # build for publishing