@effected/store

Durable local state for Effect: two services over one primitive. Store is a schema-versioned, migrated SqlClient — a managed database connection with a user-defined migration ledger that supports up, down, rollback and a status projection. Cache is a key → Uint8Array cache with TTL, tags, bulk invalidation, an eviction policy and a PubSub of lifecycle events. Both run on SQLite through Node's built-in node:sqlite, so there is no native compile step, and both surface their failures as tagged errors that carry the underlying SqlError structurally.

Pre-release. This package is part of the @effected/* kit, in pre-1.0.0 development against a single pinned Effect v4 prerelease. Packages graduate to 1.0.0 once Effect 4.0.0 ships. To hold your own effect versions at exactly the ones the kit is built and tested against, install @effected/pnpm-plugin-effect.

Stability: unstable. This package's API surface is not yet considered complete and may change across 0.x releases. Pin an exact version — even a package marked stable before 1.0.0 can introduce a breaking change by accident, and an exact pin turns that into a type-check error rather than a runtime surprise. Full policy: release strategy.

Why @effected/store

A store and a cache look like the same thing with a flag on it, and treating them that way is how caches end up holding data nobody can afford to lose. An evicted cache entry is correct behaviour; a lost state row is a bug. So they are two services here, with different contracts — only Cache has TTL, tags and eviction, and only Store has a migration ledger you own. They do share the ledger engine underneath, keyed by table name, so a Store and a Cache can live in the same database file without colliding.

The other thing this package refuses is defect laundering. A migration that throws is a programmer error, not a database failure, and it stays a defect rather than arriving as a StoreError you might be tempted to retry. Only a typed SqlError becomes a domain error, and it is carried whole rather than flattened into a reason string. Layer construction runs pending migrations and puts the failure on the layer's typed error channel — no orDie hiding a broken schema behind a working service.

Install

npm install @effected/store effect

pnpm add @effected/store effect

Requires Node.js >=24.11.0.

All @effected/* packages are ESM-only: the exports maps publish only import conditions, so require() — including tools that resolve in CJS mode — fails with Node's ERR_PACKAGE_PATH_NOT_EXPORTED rather than loading a CJS build that does not exist. Import from an ES module.

effect v4 is the only peer dependency. The SQLite driver (@effect/sql-sqlite-node) is a regular dependency of this package, so you do not install it yourself — it rides Node's built-in node:sqlite, with no native build and no transitive peers of its own. That single runtime dependency is what makes this the repo's one integrated-tier package: anything that depends on @effected/store inherits the driver.

Quick start

Declare your migrations, bind the layer to a const, and use store.client for your own queries:

import { Store, type StoreMigration } from "@effected/store";
import { Effect } from "effect";

const migrations: ReadonlyArray<StoreMigration> = [
  {
    id: 1,
    name: "create-notes",
    up: (sql) => sql`CREATE TABLE notes (id INTEGER PRIMARY KEY, body TEXT NOT NULL)`,
    down: (sql) => sql`DROP TABLE notes`,
  },
];

// The filename reaches SQLite as given, so its parent directory must already
// exist. Use `@effected/xdg` to resolve (and create) a real application data dir.
const StoreLive = Store.layerSqlite({ filename: "state.db", migrations });

const program = Effect.gen(function* () {
  const store = yield* Store;
  const sql = store.client;
  yield* sql`INSERT INTO notes (body) VALUES (${"first"})`;
  return yield* sql<{ id: number; body: string }>`SELECT id, body FROM notes`;
});

Effect.runPromise(program.pipe(Effect.provide(StoreLive))).then(console.log);
// [ { id: 1, body: "first" } ]

The layer statics are parameterized factories, not layers. Calling Store.layerSqlite(...) twice builds two layers, and Layer memoization is by reference — bind the result to a const, as above, or the database is opened twice.

The parent directory of filename must exist. The SQLite driver's construction has no error channel, so a missing directory arrives as a defect rather than a typed failure; creating it is the caller's job, and @effected/xdg is the package that knows where the directory belongs.

The layer trio

Both services expose the same three statics, and the split is the seam:

| Static | What it provides | Requirements | | ------ | ---------------- | ------------ | | layer(options) | The service over an abstract SqlClient — any Effect SQL driver satisfies it | SqlClient | | layerSqlite(options & { filename }) | The service plus the SQLite driver | none | | layerTest(options) | layerSqlite at :memory:; hermetic, what the suites use | none |

The SQL core lives in effect itself, under effect/unstable/sql — there is no @effect/sql package on the v4 line, so SqlClient is imported from effect/unstable/sql/SqlClient.

Driver options

layerSqlite takes the rest of the driver's configuration through client, so tuning the SQLite client no longer means dropping to the abstract layer with a hand-wired driver:

const StoreLive = Store.layerSqlite({
  filename: "state.db",
  migrations,
  client: { disableWAL: true, prepareCacheSize: 50 },
  checkpointOnClose: true,
});

• client passes through everything SqliteClient.layer accepts except filename (owned by this layer) and the two name-transform options — transformResultNames/transformQueryNames would rewrite the result names of the migration ledger's own queries and silently report every migration pending. If you need name transforms, wire your own client under the abstract layer.
• checkpointOnClose: true registers a PRAGMA wal_checkpoint(TRUNCATE) finalizer that runs against the still-open connection, before the driver closes it — the finalizer every durable-SQLite consumer was writing by hand. Best-effort: a failing checkpoint never turns a clean shutdown into a failed one. SQLite-specific, so it lives on the sqlite layers only; layerTest (:memory:) has no WAL and never checkpoints.

Cache.layerSqlite takes the same two options.

Migrations

Migrations are a list you own: a positive-integer id, a name recorded in the ledger, an up, and an optional down. Both return Effect<unknown, SqlError>, so a tagged SQL template goes back as-is — the engine discards the value, and a CREATE TABLE that already describes itself needs no Effect.asVoid wrapped around it. Layer construction ensures the ledger table and applies everything pending, so a freshly built Store is already migrated. migrate re-runs pending migrations, rollback(toId) unwinds everything with id > toId newest-first (rollback(0) unwinds all of it), and status projects the full list with each migration's appliedAt:

import { Store } from "@effected/store";
import { Effect } from "effect";

const program = Effect.gen(function* () {
  const store = yield* Store;
  yield* store.rollback(0);
  return yield* store.status;
});
// Every migration is listed, each with `appliedAt` absent — they are all pending again.

Duplicate ids, non-positive-integer ids and a non-integer toId are wiring errors, not data conditions: they die at layer construction rather than failing typed. A migration that throws stays a defect too, and the surrounding transaction rolls back.

Adopting a database migrated by effect's Migrator

Store is your schema — migrations create your tables, client queries them — but its ledger is its own. A database previously migrated by effect/unstable/sql/Migrator records what ran in effect_sql_migrations (migration_id, name, created_at), which Store does not read: on first construction over such a file, Store sees an empty _store_migrations ledger and re-runs every migration. If your migrations are idempotent (CREATE TABLE IF NOT EXISTS …), that re-run is harmless and you can skip all of this. If they are not — a bare CREATE TABLE, a seeding INSERT — seed the ledger before the first Store layer is built, because layer construction itself runs pending migrations.

Detect the old ledger:

SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'effect_sql_migrations';

Seed _store_migrations from it — this is a one-time step against the closed database file, so plain node:sqlite is the right tool:

import { DatabaseSync } from "node:sqlite";

const db = new DatabaseSync("state.db");
db.exec(`
  CREATE TABLE IF NOT EXISTS _store_migrations (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    applied_at TEXT NOT NULL
  );
  INSERT INTO _store_migrations (id, name, applied_at)
  SELECT migration_id, name, strftime('%Y-%m-%dT%H:%M:%fZ', created_at)
  FROM effect_sql_migrations
  WHERE migration_id NOT IN (SELECT id FROM _store_migrations);
`);
db.close();

Two things to hold constant across the adoption: the ids in your StoreMigration list must equal the migration_ids the old Migrator recorded (it derived them from the migration file names), and the migration contents must describe the schema the database already has — the seed only tells Store "these already ran", it does not verify them. The old effect_sql_migrations table can be dropped afterwards or left in place; Store never touches it.

Cache

Cache stores bytes under string keys, with an optional TTL and a set of tags for bulk invalidation:

import { Cache } from "@effected/store";
import { Duration, Effect } from "effect";

const CacheLive = Cache.layerSqlite({ filename: "cache.db", maxEntries: 1000 });

const program = Effect.gen(function* () {
  const cache = yield* Cache;

  yield* cache.set({
    key: "npm:effect",
    value: new TextEncoder().encode(`{"name":"effect"}`),
    contentType: "application/json",
    tags: ["npm", "registry"],
    ttl: Duration.minutes(10),
  });

  const hit = yield* cache.get("npm:effect");
  // Option.some(CacheEntry) while the entry is live; Option.none() once the TTL has passed

  return yield* cache.invalidateByTag("npm");
  // { count: 1, keys: [ "npm:effect" ] }
});

Effect.runPromise(program.pipe(Effect.provide(CacheLive))).then(console.log);

The invariants worth knowing:

• Expiry is lazy. get and has delete an expired row on read; prune sweeps in bulk. The clock is read through DateTime.now, so TestClock drives expiry deterministically in tests — provide TestClock.layer() outside the Effect.provide that supplies the cache, never beneath it, or the test body has no TestClock in its context, TestClock.adjust dies as a defect, and nothing you try to expire ever expires.
• Eviction is least-recently-written, not LRU-read. With maxEntries set, a set evicts the oldest-written entries in the same transaction until the bound holds, and publishes an Evicted event.
• onRemoved runs inside the delete transaction. invalidate, invalidateByTag, invalidateAll and prune each take an optional callback that runs before the delete commits: a typed failure rolls the delete back and suppresses the event, and your error type survives in the signature as CacheError | E. This is how you keep a cache entry and the file it points at from drifting apart.
• Keys and tags are data, never SQL. Everything reaches SQLite through the tagged-template SqlClient, and tag matching escapes %, _ and \ before it interpolates, so a tag containing a backslash matches its own entries.

Degrading to a miss

A cache that cannot be constructed fails its layer, and that failure belongs to the whole program. Cache.degrading wraps any Cache layer so a construction failure yields a working, empty cache instead: reads miss, writes are discarded, removals report nothing removed, no operation can fail, and the cause is logged once at warning level.

import { Cache } from "@effected/store";
import { Effect } from "effect";

const CacheLive = Cache.degrading(Cache.layerSqlite({ filename: "cache.db" }));

const program = Effect.gen(function* () {
  const cache = yield* Cache;
  return cache.degraded;
  // false for a live cache; true when construction failed and this is the fallback
});

It is opt-in because the opposite posture is legitimate — a consumer that wants a cache problem to be fatal, or that wants the narrower per-operation form, keeps exactly that by not calling it. Two details make it worth having rather than hand-writing. The SQLite driver reports construction failures — a filename whose parent directory does not exist, the common case — as defects rather than typed failures, so a failure-only catch misses the case this exists for. And interruption is deliberately re-raised with its interrupting fiber intact, because a caller shutting down is not a broken cache. degraded is a plain field rather than a CacheEvent because degradation is decided at construction, before any subscriber exists, and the events hub does not replay.

Read-through, in one call

The loop above — get, decode, fetch on a miss, encode, set — is the entire reason to have a cache, so it is a single call rather than twenty-five lines in every consumer:

import { Cache } from "@effected/store";
import { Effect, Schema } from "effect";

const Members = Schema.Struct({ login: Schema.String });

const program = Effect.gen(function* () {
  const members = yield* Cache.through("team:platform", Schema.fromJsonString(Members), {
    ttl: "1 hour",
    tags: ["team"],
  })(fetchMembersFromApi); // ← only runs on a miss

  return members;
});

schema encodes to string; the last step to bytes is Uint8ArrayFromUtf8, so the encoding decision lives in one audited place instead of one per consumer. Use Cache.throughVerbose when the caller needs to know where the value came from — it returns { value, hit }, which is what you want for printing (cached) next to a line of output. The CacheEvent PubSub is the right channel for telemetry and the wrong one for a fact the read-through already knew.

Two policies this makes the package's rather than yours:

• A value that fails to decode is a miss, not a failure. Those bytes were written by an older build of your own program: the user did not cause it, cannot fix it without knowing the cache exists, and everything in here is re-derivable by definition. Failing would strand them behind a cache they cannot see. The stale entry is overwritten on the way out.
• CacheError is surfaced, not swallowed. A cache is additive, and you may well want to push through a broken one — but that is your call to make with Effect.catchTag("CacheError", …), because a database that cannot be read is a real and reportable condition. This package will not hide it from you by default.

Uint8ArrayFromUtf8

Cache values are bytes, and the honest way to produce them is a schema. Core ships Uint8ArrayFromBase64, Uint8ArrayFromBase64Url and Uint8ArrayFromHex — and nothing for UTF-8, so Schema.fromJsonString(schema) gets you to string and stops one inch short. Uint8ArrayFromUtf8 is that inch:

const Payload = Schema.fromJsonString(Settings).pipe(Schema.encodeTo(Uint8ArrayFromUtf8));

Encoding fails on malformed UTF-8 rather than substituting replacement characters, so a corrupt value stays distinguishable from a valid one that happens to contain U+FFFD. If core ever ships an equivalent, prefer that one.

Every operation publishes to cache.events, an unbounded PubSub<CacheEvent> — Hit, Miss, Set, Expired, Evicted, Invalidated, InvalidatedByTag, InvalidatedAll and Pruned. It is unbounded on purpose: a slow subscriber must never backpressure a cache write.

Errors

| Tag | Means | Recovery | | --- | --- | --- | | StoreError | A store operation's own SQL failed — ledger bookkeeping, or the queries around a migration. Carries operation and the structural cause. | Usually fatal; report the operation and the cause. | | StoreMigrationError | A user-supplied migration failed with a typed SqlError. Carries direction, id, name and the structural cause. | Report which migration and which direction; the ledger is left consistent. | | CacheError | A cache operation's SQL failed. Carries operation, an optional key and the structural cause. | A cache is a cache — falling back to the origin is usually right. |

Defects are not errors here. A throwing migration callback, a throwing onRemoved, a maxEntries that is not a positive integer: all of those are programmer mistakes and stay on the defect channel where they belong.

Features

• Store — a migrated SqlClient with migrate, rollback, status and the raw client for your own schema-aware queries.
• Cache — TTL, tags, bulk invalidation, a maxEntries eviction policy and a CacheEvent stream, over key → Uint8Array.
• Cache.degrading — an opt-in layer combinator that turns a construction failure into a working, empty cache; the degraded field on the service tells the two apart.
• layer / layerSqlite / layerTest on both — driver-agnostic, batteries-included and in-memory, with the same options.
• StoreError, StoreMigrationError, CacheError — tagged errors carrying the underlying SqlError structurally, never a reason string.
• CacheEntry, CacheEntryMeta, CacheRemovalResult, StoreMigrationStatus — the returned records; entries lists metadata without loading BLOBs.
• Named spans on every public fallible method (Store.migrate, Cache.get, …), nesting over the driver's own statement spans.

License

MIT