@kstackz/std-toolkit

Single-table design toolkit: database-agnostic sync over single-table item collections, with schema evolution, portable database adapters, and TanStack DB integration

Big picture

Applications that store many entity types in one table, then mirror that data into a browser, end up writing the same three things by hand: a schema that can still read last year's rows, a storage layer that is bolted to one database, and a sync loop that keeps a client cache fresh. std-toolkit provides each as a separate subpath that shares one Entity model, so they compose without glue code.

core defines the Entity envelope and metadata every other subpath speaks. eschema gives versioned schemas that migrate on read. db defines a StdTable once, and the DynamoDB, SQLite, IndexedDB, and Memory adapters realize it without changing application code. sync drives TanStack DB Collections from any backend and persists its replica through the same StdTable contract. snapshot captures the resulting storage contract as one document per table, and studio-rpc serves it to Std Studio. alchemy deploys a table and refuses a deploy that would break a stored version, keeping the accepted snapshot in Alchemy state. @kstackz/std-toolkit/snapshot/vitest is the one test a table needs. Snapshot never runs inside an adapter or at request time.

Each subpath owns its vocabulary in a CONTEXT.md: core, eschema, snapshot, db, sync. The context map explains how they relate. Decisions live in docs/adr/, src/db/docs/adr/, and src/sync/docs/adr/. Longer reads: Evolving schema, Sync guide. The stories folder is a guided walkthrough that runs as tests.

Install

npm install std-toolkit effect

Node 24 or later. Peer dependencies:

• effect (required): every subpath is built on Effect Services, Layers, and Schema.
• @tanstack/react-db (optional): @kstackz/std-toolkit/sync creates and drives TanStack DB Collections.
• react (optional): required by @tanstack/react-db; only needed when you use @kstackz/std-toolkit/sync.
• alchemy (optional): only @kstackz/std-toolkit/alchemy imports it, to deploy a table and guard its snapshot.

Exports

@kstackz/std-toolkit/core

See src/core/README.md.

@kstackz/std-toolkit/eschema

See src/eschema/README.md.

@kstackz/std-toolkit/snapshot

| Export | What it does | | ----------------------------- | ---------------------------------------------------------------------------------------- | | TableSnapshot.capture | Captures a table's contract as plain data: topology, entities, and every schema version. | | TableSnapshot.parse | Reads a stored document, migrating older formats forward and validating its references. | | TableSnapshot.diff | Compares a previous and a current snapshot into classified semantic changes. | | TableSnapshot.renderChanges | Renders a list of changes as human-readable text, grouped by impact. | | TableSnapshotESchema | The ESchema of the stored table snapshot document. | | SnapshotIncompatible | Error raised when a new snapshot is not upgradable from the accepted one. |

@kstackz/std-toolkit/snapshot/vitest

The recommended test for a table, in one call. It needs vitest as a peer.

import { expectTableSnapshot } from '@kstackz/std-toolkit/snapshot/vitest';
import { it } from 'vitest';
import { table } from '../src/table.js';

it('keeps the table in step with its committed snapshot', async () => {
  await expectTableSnapshot(table, './fixtures/table.snapshot.json');
});

The file holds the table snapshot: topology, entities, and every version of every schema the table reaches. It is compared as parsed data, so formatting never fails the test. On a mismatch the failure lists every change with its classification. vitest -u accepts the current document, like any file snapshot.

| Export | What it does | | --------------------- | ---------------------------------------------------------------------- | | expectTableSnapshot | Captures the table's snapshot and compares it with the committed file. |

@kstackz/std-toolkit/alchemy

Deploys a StdTable with Alchemy. Each target keeps the last accepted table snapshot in Alchemy state and fails the deploy when the new one is not upgradable from it. Add providers() to the stack's providers.

| Export | What it does | | ---------------- | --------------------------------------------------------------------------------- | | DynamoDB.table | Guards the snapshot, then creates or updates the DynamoDB table and its indexes. | | D1.table | Guards the snapshot, then creates or updates the table and its indexes in D1. | | providers | Every provider std-toolkit's resources need, to merge into a stack's providers. |

@kstackz/std-toolkit/studio-rpc

See src/studio-rpc/README.md.

@kstackz/std-toolkit/db

| Export | What it does | | --------------- | ------------------------------------------------------------------------------------------------- | | StdTable.make | Starts a table builder from a logical name; chain primary, lsi, gsi, then build. | | DatabaseError | Tagged error every StdTable operation fails with, carrying a reason such as a failed condition. |

The built StdTable exposes entity, singleEntity, transact, scan, subscribe, drift, reindex, and dangerouslyRemoveAllItems. See src/db/CONTEXT.md.

@kstackz/std-toolkit/db/dynamodb

See src/db/dynamodb/README.md.

@kstackz/std-toolkit/db/idb

See src/db/idb/README.md.

@kstackz/std-toolkit/db/memory

See src/db/memory/README.md.

@kstackz/std-toolkit/db/sqlite

See src/db/sqlite/README.md. It also covers the driver entrypoints ./db/sqlite/node, ./db/sqlite/bun, ./db/sqlite/better-sqlite3, ./db/sqlite/d1, and ./db/sqlite/durable-object.

@kstackz/std-toolkit/sync

See src/sync/README.md. It also covers ./sync/paced and ./sync/platform/browser.

Usage

Define a schema, store it in a table, sync it to the browser

The same Task schema serves storage and sync. The table is realized in memory here; swapping Memory.make(table) for a SQLite, IndexedDB, or DynamoDB adapter changes nothing else. The sync instance polls the table for changes and projects them into a TanStack DB Collection. Lifted from stories 01, 03, and 25.

import { createLiveQueryCollection, eq } from '@tanstack/react-db';
import { Effect, Schema } from 'effect';
import { StdTable } from '@kstackz/std-toolkit/db';
import { Memory } from '@kstackz/std-toolkit/db/memory';
import { EntityESchema } from '@kstackz/std-toolkit/eschema';
import { createStdSync, strategy } from '@kstackz/std-toolkit/sync';

// 1. The shape of a task; `taskId` identifies one.
const Task = EntityESchema.make('Task', 'taskId', {
  boardId: Schema.String,
  title: Schema.String,
  status: Schema.Literals(['open', 'done']),
}).build();

// 2. A table, and Task bound to it: `boardId` fills the partition key.
const table = StdTable.make('board').primary('pk', 'sk').build();
const task = table
  .entity(Task)
  .primary({ pk: ['boardId'] })
  .build();
const board = Memory.make(table);

// 3. The server side of sync: tasks on one board changed after `cursor`.
const changesOn = (boardId: string, cursor: { meta: { _u: string } } | null) =>
  task.query('primary', { pk: { boardId }, '>=': null }).pipe(
    Effect.map((page) =>
      page.items
        .filter((item) => cursor === null || item.meta._u > cursor.meta._u)
        .sort((a, b) => (a.meta._u < b.meta._u ? -1 : 1)),
    ),
    Effect.provide(board.layer),
  );

// 4. A sync instance and a Collection that reads one board at a time.
const app = createStdSync({ name: 'board' });
const tasks = app.collection(Task, {
  sync: {
    partitions: {
      boardId: (boardId) =>
        strategy.oldToNew({
          fetch: ({ after }) => changesOn(boardId, after),
          pollEvery: '1 second',
        }),
    },
  },
  onInsert: (items) =>
    Effect.forEach(items, (item) => task.insert(item)).pipe(
      Effect.provide(board.layer),
    ),
});

// 5. A live query for the `work` board starts the sync for that partition.
const screen = createLiveQueryCollection({
  query: (q) =>
    q.from({ task: tasks }).where(({ task }) => eq(task.boardId, 'work')),
  startSync: true,
});

• EntityESchema.make(...).build() produces a schema that encodes with a _v stamp and decodes any past version to the latest shape.
• table.entity(Task).primary({ pk: ['boardId'] }) maps key paths of the value to the table's key attributes; the sort key is always the id field. A key path may reach into nested objects and union branches (owner.teamId) and must end at a string or number.
• Memory.make(table).layer satisfies the StdTableService<'board'> requirement of every task.* call. Any other adapter's layer does the same.
• createStdSync keeps its local copy in memory by default. In a real page pass platform: browser() from @kstackz/std-toolkit/sync/platform/browser to keep it in IndexedDB and let one tab read for all.
• A partition's strategy starts when a TanStack query filters on boardId. after is exclusive: return entities strictly after it.
• onInsert writes through to the same table, so the next poll confirms the optimistic row.

Deploy a table and refuse a breaking change

An application deploys its table through @kstackz/std-toolkit/alchemy. The first deploy records the table snapshot in Alchemy state. Every later deploy compares the new snapshot with the accepted one and fails before the table is touched if a stored version was edited or removed, or the key layout moved. Lifted from apps/alchemy-console/alchemy.run.ts.

import { Stack, Stage } from 'alchemy';
import * as Cloudflare from 'alchemy/Cloudflare';
import * as Effect from 'effect/Effect';
import * as Layer from 'effect/Layer';
import {
  D1,
  providers as stdToolkitProviders,
} from '@kstackz/std-toolkit/alchemy';
import { consoleTable } from './src/server/storage/table/index.ts';
// Entities register on the table as their modules load; the snapshot must see all of them.
import './src/server/storage/stores/index.ts';

export const Database = Cloudflare.D1.Database(
  'Database',
  Effect.gen(function* () {
    return { name: `alchemy-console-${yield* Stage}` };
  }),
);

export const Worker = Cloudflare.Website.Vite(
  'Worker',
  Effect.gen(function* () {
    const database = yield* Database;
    yield* D1.table('ConsoleTable', { table: consoleTable, database });
    return { env: { DB: Database } };
  }),
);

export default Stack(
  'AlchemyConsole',
  {
    providers: Layer.merge(Cloudflare.providers(), stdToolkitProviders()),
    state: Cloudflare.state(),
  },
  Worker,
);

• D1.table registers one resource. Its provider checks the accepted snapshot before creating the physical table or reconciling indexes. Snapshot, database, and physical table name changes trigger a deploy update; a new physical target starts a fresh baseline.
• DynamoDB.table does the same for DynamoDB; the table resource creates the indexes itself, so no setup action follows.
• At runtime the application provides an adapter layer, SQLite.make(table, { database }).layer, and never touches snapshot.
• The guard is only as durable as the Alchemy state store behind the stack.