@proseql/core

Runtime-agnostic in-memory database with type-safe queries, relationships, and Effect integration.

Install

npm install @proseql/core

Quick Start

import { Effect, Schema } from "effect"
import { createEffectDatabase } from "@proseql/core"

const BookSchema = Schema.Struct({
  id: Schema.String,
  title: Schema.String,
  author: Schema.String,
  year: Schema.Number,
  genre: Schema.String,
})

const config = {
  collections: {
    books: {
      schema: BookSchema,
      relationships: {},
    },
  },
} as const

const program = Effect.gen(function* () {
  const db = yield* createEffectDatabase(config, {
    books: [
      { id: "1", title: "Dune", author: "Frank Herbert", year: 1965, genre: "sci-fi" },
    ],
  })

  // query
  const scifi = await db.books.query({
    where: { genre: "sci-fi" },
  }).runPromise

  // create
  const book = await db.books.create({
    title: "Neuromancer",
    author: "William Gibson",
    year: 1984,
    genre: "sci-fi",
  }).runPromise

  // update
  await db.books.update("1", { genre: "classic" }).runPromise
})

await Effect.runPromise(program)

Schema Definition

Schemas use Effect's Schema.Struct for type-safe validation:

import { Schema } from "effect"

const BookSchema = Schema.Struct({
  id: Schema.String,
  title: Schema.String,
  author: Schema.String,
  year: Schema.Number,
  genre: Schema.String,
})

const AuthorSchema = Schema.Struct({
  id: Schema.String,
  name: Schema.String,
  birthYear: Schema.Number,
  country: Schema.String,
})

Nested Data

Schemas can contain nested objects. ProseQL supports them everywhere — filtering, sorting, updates, aggregation, indexing, search, and computed fields.

const BookSchema = Schema.Struct({
  id: Schema.String,
  title: Schema.String,
  genre: Schema.String,
  metadata: Schema.Struct({
    views: Schema.Number,
    rating: Schema.Number,
    tags: Schema.Array(Schema.String),
    description: Schema.String,
  }),
  author: Schema.Struct({
    name: Schema.String,
    country: Schema.String,
  }),
})

Two ways to reference nested fields:

// shape-mirroring — mirrors the object structure
await db.books.query({ where: { metadata: { rating: 5 } } }).runPromise

// dot-notation — flat string path
await db.books.query({ where: { "metadata.rating": 5 } }).runPromise

Both are equivalent. Use whichever reads better in context.

CRUD

Type-safe operations with .runPromise for convenience.

// create one
const book = await db.books.create({
  title: "The Dispossessed",
  author: "Ursula K. Le Guin",
  year: 1974,
  genre: "sci-fi",
}).runPromise

// create a bunch
const batch = await db.books.createMany([
  { title: "Snow Crash", author: "Neal Stephenson", year: 1992, genre: "sci-fi" },
  { title: "Parable of the Sower", author: "Octavia Butler", year: 1993, genre: "sci-fi" },
]).runPromise

// find by ID — O(1), not a scan
const found = await db.books.findById("1").runPromise

// update
await db.books.update("1", { genre: "prophetic" }).runPromise

// upsert — create if missing, update if found
const result = await db.books.upsert({
  where: { id: "42" },
  create: { title: "Hitchhiker's Guide", author: "Douglas Adams", year: 1979, genre: "comedy" },
  update: { genre: "documentary" },
}).runPromise

// update many by predicate
await db.books.updateMany(
  (book) => book.genre === "sci-fi",
  { genre: "speculative fiction" },
).runPromise

// upsert many
await db.books.upsertMany([
  { where: { id: "1" }, create: { title: "Dune", author: "Frank Herbert", year: 1965, genre: "sci-fi" }, update: { genre: "classic" } },
  { where: { id: "99" }, create: { title: "New Book", author: "New Author", year: 2024, genre: "new" }, update: { genre: "updated" } },
]).runPromise

// delete
await db.books.delete("1").runPromise

// delete by predicate
await db.books.deleteMany(
  (book) => book.year < 1970,
).runPromise

Update Operators

Atomic, type-safe mutations.

// increment/decrement numbers
await db.books.update("1", { year: { $increment: 1 } }).runPromise
await db.books.update("1", { year: { $decrement: 5 } }).runPromise
await db.books.update("1", { year: { $multiply: 2 } }).runPromise

// string append/prepend
await db.books.update("1", { title: { $append: " (Revised)" } }).runPromise
await db.books.update("1", { title: { $prepend: "The " } }).runPromise

// array operations
await db.books.update("1", { tags: { $append: "classic" } }).runPromise
await db.books.update("1", { tags: { $prepend: "must-read" } }).runPromise
await db.books.update("1", { tags: { $remove: "draft" } }).runPromise

// toggle booleans
await db.books.update("1", { inStock: { $toggle: true } }).runPromise

// explicit set (same as plain value, but composable with other operators)
await db.books.update("1", { genre: { $set: "masterpiece" } }).runPromise

// nested updates — deep merge preserves sibling fields
await db.books.update("1", { metadata: { views: 500 } }).runPromise
// → metadata.views = 500, metadata.rating/tags/description unchanged

// nested operators
await db.books.update("1", { metadata: { views: { $increment: 100 } } }).runPromise

// update multiple nested paths at once
await db.books.update("1", {
  metadata: { rating: 5, views: { $increment: 200 } },
  author: { country: "CA" },
}).runPromise

| Operator | Works On | What It Does | |----------|----------|-------------| | $set | everything | Explicit set (equivalent to plain value) | | $increment | numbers | Add to current value | | $decrement | numbers | Subtract from current value | | $multiply | numbers | Multiply current value | | $append | strings, arrays | Append to end | | $prepend | strings, arrays | Prepend to beginning | | $remove | arrays | Remove matching element(s) | | $toggle | booleans | Flip the value |

Querying

Filtering

const results = await db.books.query({
  where: {
    year: { $gte: 1960, $lt: 1990 },
    genre: { $in: ["sci-fi", "fantasy"] },
    title: { $contains: "Dark" },
  },
}).runPromise

| Operator | Works On | What It Does | |----------|----------|-------------| | $eq | everything | Equals | | $ne | everything | Not equals | | $in | everything | In list | | $nin | everything | Not in list | | $gt $gte $lt $lte | numbers, strings | Comparisons | | $startsWith $endsWith $contains | strings | String matching | | $search | strings | Token-based text search (see Full-Text Search) | | $contains $all $size | arrays | Array matching | | $or | clauses | Match any of the given conditions | | $and | clauses | Match all of the given conditions | | $not | clause | Negate a condition |

Nested fields work with any operator — use shape-mirroring or dot-notation:

// shape-mirroring
const popular = await db.books.query({
  where: { metadata: { views: { $gt: 700 } } },
}).runPromise

// dot-notation
const highlyRated = await db.books.query({
  where: { "metadata.rating": { $gte: 4 } },
}).runPromise

Or just pass a value for exact match:

const scifi = await db.books.query({ where: { genre: "sci-fi" } }).runPromise

Logical Operators

Combine conditions with $or, $and, and $not:

// books that are either sci-fi OR published before 1970
const results = await db.books.query({
  where: {
    $or: [
      { genre: "sci-fi" },
      { year: { $lt: 1970 } },
    ],
  },
}).runPromise

// NOT fantasy
const notFantasy = await db.books.query({
  where: {
    $not: { genre: "fantasy" },
  },
}).runPromise

// combine with field-level filters
const complex = await db.books.query({
  where: {
    $and: [
      { year: { $gte: 1960 } },
      { $or: [{ genre: "sci-fi" }, { genre: "fantasy" }] },
    ],
  },
}).runPromise

// logical operators work with nested fields too
const featured = await db.books.query({
  where: {
    $or: [
      { metadata: { rating: 5 } },
      { author: { country: "UK" } },
    ],
  },
}).runPromise

Sorting

const sorted = await db.books.query({
  sort: { year: "desc", title: "asc" },
}).runPromise

// sort by nested fields using dot-notation
const mostViewed = await db.books.query({
  sort: { "metadata.views": "desc" },
}).runPromise

Field Selection

const titles = await db.books.query({
  select: ["title", "author"],
}).runPromise
// → [{ title: "Dune", author: "Frank Herbert" }, ...]

Pagination

// offset-based (the simple kind)
const page = await db.books.query({
  sort: { title: "asc" },
  limit: 10,
  offset: 20,
}).runPromise

// cursor-based (the stable kind — inserts and deletes don't break it)
const page1 = await db.books.query({
  sort: { title: "asc" },
  cursor: { key: "title", limit: 10 },
}).runPromise
// page1.pageInfo.endCursor → "Neuromancer"
// page1.pageInfo.hasNextPage → true

const page2 = await db.books.query({
  sort: { title: "asc" },
  cursor: { key: "title", after: page1.pageInfo.endCursor, limit: 10 },
}).runPromise

Aggregation

const stats = await db.books.aggregate({
  count: true,
  sum: "pages",
  min: "year",
  max: "year",
  avg: "year",
}).runPromise
// → { count: 42, sum: { pages: 12840 }, min: { year: 1818 }, max: { year: 2024 }, avg: { year: 1973.5 } }

// by genre
const genres = await db.books.aggregate({
  groupBy: "genre",
  count: true,
}).runPromise
// → [
//   { group: { genre: "sci-fi" }, count: 23 },
//   { group: { genre: "fantasy" }, count: 12 },
//   { group: { genre: "literary horror" }, count: 7 },
// ]

// filtered
const modern = await db.books.aggregate({
  where: { year: { $gte: 2000 } },
  count: true,
}).runPromise

// aggregate nested fields using dot-notation
const nested = await db.books.aggregate({
  where: { metadata: { rating: { $gte: 4 } } },
  count: true,
  sum: "metadata.views",
  avg: "metadata.rating",
}).runPromise
// → { count: 4, sum: { "metadata.views": 3600 }, avg: { "metadata.rating": 4.5 } }

// group by nested field
const byCountry = await db.books.aggregate({
  groupBy: "author.country",
  count: true,
}).runPromise
// → [
//   { group: { "author.country": "USA" }, count: 3 },
//   { group: { "author.country": "UK" }, count: 2 },
// ]

Full-Text Search

Search text fields with token-based matching. Results are ranked by relevance.

// field-level search
const results = await db.books.query({
  where: { title: { $search: "left hand" } },
}).runPromise

// multi-field search — terms can span across fields
const results = await db.books.query({
  where: {
    $search: { query: "herbert dune", fields: ["title", "author"] },
  },
}).runPromise

// search all string fields (omit fields)
const results = await db.books.query({
  where: {
    $search: { query: "cyberpunk" },
  },
}).runPromise

Search nested fields by specifying dot-paths:

const results = await db.books.query({
  where: {
    $search: { query: "cyberpunk", fields: ["metadata.description"] },
  },
}).runPromise

Add a searchIndex for faster lookups on large collections:

const config = {
  books: {
    schema: BookSchema,
    searchIndex: ["title", "metadata.description", "author.name"],
    relationships: {},
  },
} as const

Without a search index, $search scans all entities (still works, just slower). With one, it hits the inverted index for O(tokens) candidate lookup.

Computed Fields

Derived values that exist only at query time. Never persisted, zero overhead when not selected.

const config = {
  books: {
    schema: BookSchema,
    computed: {
      displayName: (book) => `${book.title} (${book.year})`,
      isClassic: (book) => book.year < 1980,
      // computed fields can read nested data
      viewCount: (book) => book.metadata.views,
      isHighlyRated: (book) => book.metadata.rating >= 4,
      summary: (book) => `${book.title} by ${book.author.name} (${book.metadata.rating}/5)`,
    },
    relationships: {},
  },
} as const

const program = Effect.gen(function* () {
  const db = yield* createEffectDatabase(config, { books: initialBooks })

  // computed fields appear in query results automatically
  const books = await db.books.query().runPromise
  // → [{ title: "Dune", year: 1965, displayName: "Dune (1965)", isClassic: true, ... }]

  // filter on computed fields
  const classics = await db.books.query({
    where: { isClassic: true },
  }).runPromise

  // select specific fields (including computed)
  const labels = await db.books.query({
    select: ["displayName"],
  }).runPromise
  // → [{ displayName: "Dune (1965)" }, ...]

  // sort by computed fields
  const sorted = await db.books.query({
    sort: { displayName: "asc" },
  }).runPromise
})

Relationships

Books have authors. Authors write books. ProseQL gets it.

const config = {
  books: {
    schema: BookSchema,
    relationships: {
      author: { type: "ref" as const, target: "authors" as const, foreignKey: "authorId" },
    },
  },
  authors: {
    schema: AuthorSchema,
    relationships: {
      books: { type: "inverse" as const, target: "books" as const, foreignKey: "authorId" },
    },
  },
} as const

Populate related data in queries:

const booksWithAuthors = await db.books.query({
  populate: { author: true },
}).runPromise
// → [{ title: "Dune", author: { name: "Frank Herbert", ... } }, ...]

Foreign keys are enforced. Try referencing a ghost author:

await db.books.create({
  title: "Mystery Novel",
  authorId: "definitely-not-real",
}).runPromise
// → ForeignKeyError. Nice try.

Indexing

Full scans are for people with time to kill. Declare indexes for O(1) lookups.

const config = {
  books: {
    schema: BookSchema,
    indexes: ["genre", "authorId", ["genre", "year"]],
    relationships: {},
  },
} as const

Nested fields use dot-notation in index declarations:

const config = {
  books: {
    schema: BookSchema,
    indexes: ["metadata.views", "metadata.rating", "author.country"],
    relationships: {},
  },
} as const

Indexes are maintained automatically. Queries on indexed fields just... go fast.

// hits the genre index
const scifi = await db.books.query({
  where: { genre: "sci-fi" },
}).runPromise

// hits the compound [genre, year] index
const recent = await db.books.query({
  where: { genre: "sci-fi", year: 2024 },
}).runPromise

Reactive Queries

Subscribe to live query results. Mutations automatically push updates through the stream.

import { Effect, Stream, Scope } from "effect"

// watch a filtered query — emits current results, then re-emits on every change
const program = Effect.gen(function* () {
  const db = yield* createEffectDatabase(config, initialData)

  const stream = yield* db.books.watch({
    where: { genre: "sci-fi" },
    sort: { year: "desc" },
  })

  // process each emission
  yield* Stream.runForEach(stream, (books) =>
    Effect.sync(() => console.log("Current sci-fi:", books.length))
  )
})

// run with a scope (stream cleans up automatically when scope closes)
await Effect.runPromise(Effect.scoped(program))

Watch a single entity by ID:

const program = Effect.gen(function* () {
  const db = yield* createEffectDatabase(config, initialData)

  const stream = yield* db.books.watchById("1")

  // emits the entity (or null if it doesn't exist)
  // re-emits on update, emits null on deletion
  yield* Stream.runForEach(stream, (book) =>
    Effect.sync(() => {
      if (book) console.log("Book updated:", book.title)
      else console.log("Book was deleted")
    })
  )
})

Streams are debounced and deduplicated automatically — rapid mutations produce at most one emission after the debounce interval settles. Nested field changes trigger emissions too — updating metadata.views on a watched entity re-emits the stream.

Lifecycle Hooks

Run logic before or after mutations. Normalize data, enforce rules, log things, live your best life.

import { Effect } from "effect"
import { HookError } from "@proseql/core"

const config = {
  books: {
    schema: BookSchema,
    hooks: {
      beforeCreate: (ctx) =>
        Effect.succeed({
          ...ctx.data,
          title: ctx.data.title.trim(),
          createdAt: new Date().toISOString(),
        }),

      afterCreate: (ctx) =>
        Effect.sync(() => console.log(`New book: "${ctx.entity.title}"`)),

      beforeUpdate: (ctx) =>
        Effect.succeed({
          ...ctx.changes,
          updatedAt: new Date().toISOString(),
        }),

      onChange: (ctx) =>
        Effect.sync(() => console.log(`${ctx.operation} on books`)),
    },
    relationships: {},
  },
} as const

Hooks can reject operations:

beforeCreate: (ctx) =>
  ctx.data.year > new Date().getFullYear()
    ? Effect.fail(new HookError({
        hook: "beforeCreate",
        collection: "books",
        operation: "create",
        reason: "We don't accept books from the future",
        message: "We don't accept books from the future",
      }))
    : Effect.succeed(ctx.data),

Schema Migrations

Schemas change. Data shouldn't break. Migrations run automatically on load.

import type { Migration } from "@proseql/core"

const migrations: ReadonlyArray<Migration> = [
  {
    from: 0,
    to: 1,
    // v1 added a "genre" field
    transform: (book) => ({
      ...book,
      genre: book.genre ?? "uncategorized",
    }),
  },
  {
    from: 1,
    to: 2,
    // v2 split "author" string into "authorFirst" and "authorLast"
    transform: (book) => ({
      ...book,
      authorFirst: book.author?.split(" ")[0] ?? "",
      authorLast: book.author?.split(" ").slice(1).join(" ") ?? "",
      author: undefined,
    }),
  },
]

const config = {
  books: {
    schema: BookSchemaV2,
    version: 2,
    migrations,
    relationships: {},
  },
} as const

Data at version 0? Runs 0 → 1 → 2, validates, continues. Data already at version 2? Loaded normally. Migration fails? Original data untouched.

Transactions

All or nothing. If any operation fails, everything rolls back.

await db.$transaction(async (tx) => {
  const author = await tx.authors.create({
    name: "Becky Chambers",
  }).runPromise

  await tx.books.createMany([
    { title: "The Long Way to a Small, Angry Planet", authorId: author.id, year: 2014, genre: "sci-fi" },
    { title: "A Closed and Common Orbit", authorId: author.id, year: 2016, genre: "sci-fi" },
    { title: "Record of a Spaceborn Few", authorId: author.id, year: 2018, genre: "sci-fi" },
  ]).runPromise

  // if anything above throws, none of it happened
})

Unique Constraints

Some things should only exist once.

const config = {
  books: {
    schema: BookSchema,
    uniqueFields: ["isbn"],
    relationships: {},
  },
  reviews: {
    schema: ReviewSchema,
    uniqueFields: [["userId", "bookId"]],  // one review per user per book
    relationships: {},
  },
} as const

await db.books.create({ title: "Dune", isbn: "978-0441172719", ... }).runPromise
await db.books.create({ title: "Dune (but again)", isbn: "978-0441172719", ... }).runPromise
// → UniqueConstraintError. There can be only one.

Plugin System

Extend ProseQL with custom codecs, operators, ID generators, and global hooks.

import type { ProseQLPlugin } from "@proseql/core"

const regexPlugin: ProseQLPlugin = {
  name: "regex-search",
  operators: [{
    name: "$regex",
    types: ["string"],
    evaluate: (value, pattern) =>
      typeof value === "string" && new RegExp(pattern as string).test(value),
  }],
}

const snowflakePlugin: ProseQLPlugin = {
  name: "snowflake-ids",
  idGenerators: [{
    name: "snowflake",
    generate: () => generateSnowflakeId(),
  }],
}

const program = Effect.gen(function* () {
  const db = yield* createEffectDatabase(config, initialData, {
    plugins: [regexPlugin, snowflakePlugin],
  })

  // use the custom operator in queries
  const matches = await db.books.query({
    where: { title: { $regex: "^The.*" } },
  }).runPromise
})

// reference the custom ID generator in collection config
const config = {
  books: {
    schema: BookSchema,
    idGenerator: "snowflake",  // uses the plugin's generator
    relationships: {},
  },
} as const

Plugins can also contribute format codecs and global lifecycle hooks that run across all collections.

Error Handling

Every error is tagged. Catch exactly what you want.

import { Effect } from "effect"
import { NotFoundError } from "@proseql/core"

const result = await Effect.runPromise(
  db.books.findById("nope").pipe(
    Effect.catchTag("NotFoundError", () =>
      Effect.succeed({ title: "Book not found", suggestion: "Try the library?" }),
    ),
  ),
)

Or use try/catch if that's more your speed:

try {
  await db.books.findById("nope").runPromise
} catch (err) {
  if (err instanceof NotFoundError) {
    console.log("Have you tried the library?")
  }
}

| Error | When | |-------|------| | NotFoundError | ID doesn't exist | | ValidationError | Schema says no | | DuplicateKeyError | ID already taken | | UniqueConstraintError | Unique field collision | | ForeignKeyError | Referenced entity is a ghost | | HookError | Lifecycle hook rejected it | | TransactionError | Transaction couldn't begin/commit/rollback | | StorageError | Storage adapter trouble | | SerializationError | Couldn't encode/decode | | MigrationError | Migration went sideways | | PluginError | Plugin validation or conflict |

ID Generation

Pick a strategy. Or don't — we'll generate one for you.

import {
  generateUUID,
  generateNanoId,
  generateULID,
  generateTimestampId,
  generatePrefixedId,
  generateTypedId,
} from "@proseql/core"

generateUUID()              // "550e8400-e29b-41d4-a716-446655440000"
generateNanoId()            // "V1StGXR8_Z5jdHi6B-myT"
generateULID()              // "01ARZ3NDEKTSV4RRFFQ69G5FAV"
generateTimestampId()       // "1704067200000-a3f2-0001"
generatePrefixedId("book")  // "book_1704067200000-a3f2-0001"
generateTypedId("book")     // "book_V1StGXR8_Z5jdHi6B-myT"

Serialization

The core package includes all serialization codecs, which are runtime-agnostic:

| Format | Extension | Codec | |--------|-----------|-------| | JSON | .json | jsonCodec() | | JSONL | .jsonl | jsonlCodec() | | YAML | .yaml | yamlCodec() | | JSON5 | .json5 | json5Codec() | | JSONC | .jsonc | jsoncCodec() | | TOML | .toml | tomlCodec() | | TOON | .toon | toonCodec() | | Hjson | .hjson | hjsonCodec() | | Prose | .prose | proseCodec() |

import {
  makeSerializerLayer,
  jsonCodec,
  yamlCodec,
  tomlCodec,
  AllTextFormatsLayer,
} from "@proseql/core"

// pick and choose
makeSerializerLayer([jsonCodec(), yamlCodec(), tomlCodec()])

// or take them all (except prose, which must be registered explicitly)
AllTextFormatsLayer

Prose Format

Prose is a human-readable format where data looks like English sentences:

import { proseCodec } from "@proseql/core"

// explicit template
proseCodec({ template: '[{id}] "{title}" by {author} ({year}) — {genre}' })

// or let it learn from the @prose directive in the file
proseCodec()

On disk, prose files look like this:

@prose [{id}] "{title}" by {author} ({year}) — {genre}

[1] "Dune" by Frank Herbert (1965) — sci-fi
[2] "Neuromancer" by William Gibson (1984) — sci-fi

Persistence

Collections define schemas, relationships, indexes, hooks, migrations, and identity policy. Persistent storage is configured separately with database-level sources.

const config = {
  collections: {
    games: {
      schema: GamePayload,
      id: { kind: "derivedFromKey", field: "id" },
      relationships: {},
    },
    systems: {
      schema: SystemPayload,
      id: { kind: "derivedFromKey", field: "id" },
      relationships: {},
    },
  },
  sources: [
    {
      id: "library",
      kind: "documents",
      root: "./data/library",
      include: "**/*.yaml",
      format: "yaml",
      collections: "all",
      outbox: "generated.yaml",
    },
  ],
} as const

A YAML document source expects each file to be a top-level object keyed by collection name:

systems:
  snes:
    name: Super Nintendo

games:
  smw:
    title: Super Mario World
    systemId: snes

With id: { kind: "derivedFromKey", field: "id" }, persisted records omit a physical id field. At runtime ProseQL hydrates id from the object key (smw above). A physical id inside the persisted payload is invalid.

Document sources are strict by default:

• duplicate (collection, id) records across matching files fail database load;
• unknown top-level collection keys fail unless unknownCollections: "preserve" is configured;
• new records without an origin file are written to the source outbox on debounced save or flush();
• existing records update/delete in their origin file;
• empty matched files and existing empty source roots are valid;
• missing source roots fail by default unless optional: true is configured.

flush() is the durability boundary for pending debounced writes. It surfaces persistence failures to callers; background debounced writes are best-effort. Document-source writes preserve data and sibling collection sections, but YAML comments and original formatting are not preserved.

Read-only document graphs

A documentGraph source assembles one effective, read-only collection graph from an ordered set of directory roots. Where the documents source treats files as independent write targets (duplicates are errors), a documentGraph collapses many physical fragments into one logical read model: later fragments overlay earlier ones, and nothing is ever written back. This suits config-overlay scenarios — a base layer plus environment or host overrides merged into one view.

const config = {
  collections: {
    foods: {
      schema: FoodPayload,
      id: { kind: "derivedFromKey", field: "id" },
      relationships: {},
    },
  },
  sources: [
    {
      id: "config-graph",
      kind: "documentGraph",
      // A graph is multi-format by definition; fragments are decoded by extension.
      include: "**/*.config.{yaml,json,toml}",
      roots: [
        { root: "./config/base" },
        // Root-level collections narrow what this root may contribute.
        { root: "./config/overrides", optional: true, collections: ["foods"] },
      ],
      // Default: "error". Opt into "skip-fragment" or "skip-root" when
      // a bad removable/user-provided fragment should not reject the whole graph.
      onFragmentError: "skip-fragment",
      // Optional pure decode transform per fragment (returns a Result).
      // transform: (doc, ctx) => Result.succeed(reshape(doc)),
    },
  ],
} as const

How a graph is built:

• Discovery is opt-in. Every root needs an effective include glob — supplied at the graph level or per root (a root-level include overrides the graph-level one; exclude patterns combine). Real glob semantics (**, brace groups) are supported.
• Mixed formats by extension. Each matched file is decoded by its extension through the active serializer registry, so one graph can mix YAML, JSON, TOML, and so on. A matched file whose extension is not registered fails the load with a path-attributed error unless an explicit skip policy contains it.
• Pure decode transform. An optional transform(document, context) hook may reshape each decoded fragment, returning a Result (Result.succeed → transformed document, Result.fail → typed error). After the transform the value must be a plain object.
• Root collection allowlists. A root-level collections setting narrows the graph-level collection set for that root. Known graph collections outside the root allowlist are ignored for that root; truly unknown top-level keys still fail or are skipped according to onFragmentError.
• Deterministic overlay merge. Fragments merge in order — roots in configured order, then files lexically within a root, later wins. Objects merge recursively; arrays, scalars, and null replace. There is no delete/tombstone syntax.
• Migrate per fragment, validate after merge. Per-collection migrations run on each fragment using that fragment's own _version (bringing every fragment to the current schema version) before merge; the merged effective records are validated once. A partial overlay that is only valid once combined still passes.

Read-only and lifecycle behavior:

• Graph-owned collections are read-only. Every mutation (create/update/delete/upsert and their *Many / *WithRelationships variants) fails with OperationError (reason: "read-only-source") before touching in-memory state, including inside $transaction. No writes are ever scheduled.
• initialData for a graph-owned collection fails database creation, as does an invalid initial graph under the default strict policy.
• Fragment/root containment is opt-in. onFragmentError: "error" is the default. "skip-fragment" excludes only the bad fragment; "skip-root" excludes the entire root for that rebuild. Skipped entries are available through diagnostics so leniency is not silent.
• Watched roots reload with last-known-good. A valid fragment change rebuilds and replaces the active graph and notifies watch() subscribers; an invalid reload keeps the previous valid graph and logs a warning. When a skip policy turns a bad fragment/root into a valid rebuild, the active graph and diagnostics update together. Roots absent at startup are not watched (no late detection).

Graph metadata is available without changing record shapes:

const appleProvenance = yield* db.$documentGraph.getRecordProvenance(
  "foods",
  "apple",
)

const diagnostics = yield* db.$documentGraph.getDiagnostics()

appleProvenance includes the source id, collection, record id, ordered contributors, and latest/effective contributor. Diagnostics include skipped fragments/roots and ignored root-disallowed collections with source/root/path context.

v1 limits: CLI command support, write/outbox support, delete/tombstone semantics, and a full graph lifecycle event API are deferred; the existing documents source is unchanged.

For Node.js filesystem persistence and watcher support, see @proseql/node. For browser storage (localStorage, sessionStorage, IndexedDB), see @proseql/browser.

License

MIT