tr-pg-name-value-store

A persistent name → value store backed by a single PostgreSQL table. A thin, durable getter/setter: values are stored as JSONB, so a value may be anything JSON can carry (number, string, boolean, object, array, or null) and is returned as the corresponding JavaScript value.

• Self-maintaining schema. The table is created idempotently by the class itself on first use — no manual setup, no migrations. A pre-existing table is verified, never altered.
• Multiple named stores. A namespace gives each store its own table in the same database.
• Listing. list() pages through the names (with values, or without for a cheap walk) in byte order with keyset paging, and count() counts them, both optionally under a literal prefix — so nobody needs to query the table.
• Exact previous values. Every write (set, remove, update) resolves to the value it replaced. Writes to the same name are serialized across all processes sharing the database, so that previous value is always the one left by the immediately preceding write.
• Atomic read-modify-write. update() runs a callback inside a transaction, serialized per name, so concurrent updates compose correctly.
• Presence vs. null. The store distinguishes no value (undefined) from the value null.

Install

npm install tr-pg-name-value-store pg

pg is a peer dependency (>= 8). Requires Node >= 18 and PostgreSQL >= 9.5 (for INSERT … ON CONFLICT). The test suite runs against PostgreSQL 9.6, 16 and 18.

Quick start

import { Pool } from 'pg';
import { PgNameValueStore } from 'tr-pg-name-value-store';

const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const store = new PgNameValueStore(pool, { namespace: 'myapp' });

await store.set('greeting', 'hello');
await store.get('greeting');            // → 'hello'

await store.set('config', { retries: 3, tags: ['a', 'b'] });
await store.get('config');              // → { retries: 3, tags: ['a', 'b'] }

const previous = await store.set('greeting', 'hi'); // → 'hello' (the old value)
await store.remove('greeting');         // → 'hi'   (the removed value)
await store.get('greeting');            // → undefined

Value model

A stored value is any JSON-serializable JavaScript value: number, string, boolean, object, array, or null.

• Presence vs. null. A name that has never been set (or has been removed) reads back as undefined. A name explicitly set to JSON null reads back as null. "The value does not exist" is always represented as the row being absent — there is no SQL NULL value.
• undefined is not a value. set(name, undefined) throws TypeError; use remove(name) to delete. (Inside update, a callback that returns undefined is the one place undefined is meaningful — it means delete.)
• JSONB normalization. PostgreSQL normalizes JSONB on storage: object keys are reordered, insignificant whitespace is dropped, duplicate keys collapse to the last, and numbers are canonicalized. A value read back is semantically equal but may not be textually identical to the value written (e.g. key order).
• No U+0000 in strings. PostgreSQL's jsonb cannot store the NUL character inside a string; the server's error propagates unchanged and nothing is written.

Names

A name is any non-empty string of at most 1024 bytes of UTF-8 (so up to 1024 ASCII characters, fewer for non-ASCII) that does not contain U+0000. Anything else throws TypeError before any I/O. Within those bounds names are opaque: case-sensitive, whitespace-significant, any punctuation or Unicode. Listing orders them by byte order of their UTF-8 encoding (see Listing).

API

new PgNameValueStore(pool, options?)

Constructs a store over a pg Pool. Performs no I/O. Options:

| Option | Type | Default | Meaning | |-------------|----------|---------|---------| | namespace | string | none | Backing table is <namespace>_name_value_store. Must match /^[a-z][a-z0-9_]*$/, ≤ 41 chars. Throws TypeError if invalid. Omit for the bare table name_value_store. |

init(): Promise<void>

Idempotently ensures the schema (creating the table if absent, verifying it if present). Called automatically on first use of any method; call it explicitly to surface schema/connection errors at startup. Safe to call repeatedly and concurrently (across processes too). A failed init() is not cached; the next call retries.

get(name): Promise<value>

Resolves to the current value, or undefined if name has no value. A single lock-free SELECT that reads the latest committed value.

set(name, value): Promise<previous>

Stores value (insert or overwrite); resolves to the previous value, or undefined if there was none. Throws TypeError if value is undefined or not JSON-serializable (writing nothing).

remove(name): Promise<previous>

Deletes name; resolves to its previous value, or undefined if it had none.

update(name, callback): Promise<previous>

Atomic read-modify-write. Fetches the current value, calls callback(current) (awaiting a returned promise), then commits one outcome based on what the callback does:

| Callback… | Effect | update… | |--------------------------------------------------------|---------------------------------|------------------------------------| | returns a JSON-serializable value (not undefined) | stores it | resolves to the previous value | | returns undefined | removes the name | resolves to the previous value | | returns a non-serializable value (function, BigInt, …) | nothing (rolled back) | throws TypeError | | throws null or undefined | nothing (graceful cancel) | resolves to the previous value | | throws anything else | nothing (rolled back) | re-throws that value unchanged |

// atomic counter (creates from absent, then increments)
await store.update('hits', (n) => (typeof n === 'number' ? n : 0) + 1);

// conditional update; abort with no change and no error
await store.update('config', (cfg) => {
  if (!cfg) throw null;                 // graceful cancel
  return { ...(cfg as object), seen: true };
});

// delete via update
await store.update('stale', () => undefined);

The callback must not write to the same name through this or any other store instance — it would wait forever on the lock the callback itself holds. Writes to other names, and reads of anything, are fine.

removeAll(): Promise<void>

Removes every pair from this store's namespace. Other namespaces are untouched.

Listing

list(options?): Promise<ListResult>

Lists one page of names, in ascending byte order of the name, with their values unless values: false.

| Option | Type | Default | Meaning | |----------|-----------|---------|---------| | prefix | string | none | Only names starting with this exact string. Literal, not a pattern: %, _ and \ match themselves. Validated like a name (≤ 1024 bytes, no U+0000); empty means no filter. | | after | string | none | Keyset cursor: only names strictly after this one in byte order — normally the last name of the previous page. Need not be an existing name. Validated like prefix; empty means start from the beginning. | | limit | number | 1000 | Page size, a positive integer. The maximum is 1000; larger values clamp silently (keep paging while have_more is true). A non-integer or a value below 1 throws TypeError. | | values | boolean | true | With false, value is omitted from every entry and the value column is not read, so a store of large documents can be walked cheaply. |

The result:

| Field | Type | Meaning | |-------------|---------------|---------| | entries | ListEntry[] | The page, each { name, value?, updated_at }. value is present (and may be null) whenever values is on; a name with no value has no entry at all. updated_at is a Date (millisecond precision) of the last write. | | have_more | boolean | true if more names followed this page in the same snapshot; continue with after set to the last entry's name. |

Paging loop:

const names: string[] = [];
let after: string | undefined;
for (;;) {
  const page = await store.list({ prefix: 'job:', after, values: false });
  names.push(...page.entries.map((e) => e.name));
  if (!page.have_more) break;
  after = page.entries.at(-1)!.name; // a page with have_more is never empty
}

Each page is one lock-free statement over its own snapshot. Names written between two pages may or may not appear, and a name removed in between is simply absent (so have_more: true followed by an empty page is possible). There are no offsets: a name that exists for the whole walk is returned exactly once.

Ordering is byte order of the UTF-8 name (COLLATE "C"), identical on every database regardless of its locale — so 'B' < '_x' < 'a'. It differs from JavaScript's default string comparison only for characters outside the Basic Multilingual Plane, which matters if you merge pages with locally sorted data. On databases whose default collation is C the primary-key index serves after and prefix directly; on other databases each page is a scan and sort of the store's names, which is fine for the sizes a per-service store reaches. No index is added — the schema stays frozen.

count(options?): Promise<number>

Resolves to the number of names in the store, or of those starting with options.prefix (same semantics as above). A single lock-free statement that scans the matching names.

Concurrency

Every write to a given name — set, remove, and update — runs in its own transaction on a dedicated pooled connection and holds a per-(namespace, name) transaction-scoped advisory lock (pg_advisory_xact_lock) for its duration. Writes to the same name are therefore strictly serialized, across every process that shares the database, and each one observes exactly the committed result of the one before it. That is what makes the previous value contract exact and what lets update callbacks compose (including the create-from-absent case, which a row lock alone cannot cover).

Writes to different names never block each other. Reads — get, list and count — are single lock-free statements that see the latest committed state and neither block writes nor are blocked by them. removeAll is a single bulk DELETE that does not take per-name locks; it waits for in-flight writes on individual rows like any other statement would.

Advisory-lock keys are derived in Node (SHA-256 of the namespace and name) — no server-side extension is required.

Multiple stores

Each namespace is an independent store with its own table:

const sessions = new PgNameValueStore(pool, { namespace: 'sessions' });
const settings = new PgNameValueStore(pool, { namespace: 'settings' });
// sessions.* and settings.* never collide; removeAll() on one leaves the other intact

Several instances over the same namespace (in one process or many) share the same data and the same per-name serialization.

Errors

| Error | When | |-----------------------|------| | TypeError | Invalid namespace; invalid name (not a non-empty string, more than 1024 bytes of UTF-8, or containing U+0000); a set value or update return that is not JSON-serializable; an update callback that is not a function; invalid list/count options (a prefix or after that is not a string within the name bounds, a limit that is not a positive integer, a non-boolean values). | | SchemaMismatchError | A table with this namespace's name already exists with a different shape. It is left untouched. Exported by the package. | | re-thrown value | update re-throws any non-null/undefined value its callback throws. |

Operational/connection failures propagate from the underlying pg calls. Argument validation errors are thrown synchronously by the constructor and as rejected promises by every other method, always before any I/O.

Schema

One table per namespace ({{ns}} is <namespace>_, or empty):

CREATE TABLE {{ns}}name_value_store (
  name       TEXT        NOT NULL,
  v          JSONB       NOT NULL,
  updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (name)
);

The module issues only CREATE TABLE IF NOT EXISTS and verifies the columns of a pre-existing table against this shape — it never runs ALTER or DROP. A matching table created by an operator (with any existing rows) is adopted as-is. updated_at is maintained on every write for operational inspection; it has no public accessor.

Development

npm install
npm test            # embeds sql/, typechecks src + tests, runs vitest
npm run build       # emits dist/
npm pack --dry-run  # audit the tarball contents before publishing

The tests run against a real PostgreSQL. By default a throwaway cluster is bootstrapped with initdb/pg_ctl in a temp directory (PostgreSQL server binaries must be on PATH) and torn down afterwards. To use an existing server instead — for example the bundled compose file — point TR_PG_NAME_VALUE_STORE_TEST_URL at an admin connection; a scratch database named tr_pg_name_value_store_test is (re)created on it:

docker compose -f test/docker-compose.yml up -d
TR_PG_NAME_VALUE_STORE_TEST_URL=postgres://postgres:postgres@localhost:5434/postgres npm test
docker compose -f test/docker-compose.yml down

License

MIT © Timo J. Rinne <[email protected]>