@syncular/typegen
v0.98.0
Published
Syncular schema-to-TypeScript type generator and the `syncular` CLI
Downloads
21,474
Maintainers
Readme
@syncular/typegen
SQL migrations + one manifest → neutral schema IR (JSON) → generated TS
module. This file is the authoritative contract for the three
tool-level formats: the manifest, the IR, and the SQL subset. Wire-protocol
semantics (column types, scope patterns, schema-version gating) live in
../../docs/SPEC.md §2.4, §3.1, §1.5 — this document never
overrides it.
syncular generate [--manifest-dir <dir>] [--check] [--watch]
syncular migrations baseline|check|upgrade-lock [--manifest-dir <dir>]
syncular init [--manifest-dir <dir>]generate reads <dir>/syncular.json plus its migrations directory and
writes the migration lock, IR JSON, and configured generated modules.
QueryIR version 4 includes positional table-relation boundaries for every
physical SQL statement. Generated TypeScript descriptors expose these as
relationPlans for remote query registration. Regenerate existing query
modules before upgrading the server.
--check regenerates in memory and exits 1 unless every output on disk matches
byte-exactly. --watch regenerates on any change under the manifest dir
(Bun's recursive fs.watch, debounced; it skips the write when outputs are
already fresh so it never loops on its own output). --check and --watch
are mutually exclusive.
migrations baseline creates the first
syncular.migrations.lock.json and refuses to replace one that already
exists. migrations check is the fast, CI-friendly history check without
query analysis or output generation. New migrations are appended to the lock
by ordinary generate; locked migrations may never be edited, removed,
renamed, or reordered. A lock in format 1 or 2 predates per-migration schema
versions: generate and migrations check refuse it with
typegen.migration_lock_outdated until migrations upgrade-lock converts it.
init scaffolds a starter syncular.json, first migration, and migration
lock into an existing project (the "add syncular to my app" path). It refuses
to overwrite any of them. New projects usually start from the scaffolder
instead (bun create syncular-app my-app), which emits the same shape plus a
server + client.
When generate cannot find the manifest or migrations, the error points at the
schema guide and suggests syncular init.
1. Manifest reference (syncular.json)
{
"manifestVersion": 1,
"migrations": "./migrations",
"output": { "ir": "./syncular.ir.json", "module": "./syncular.generated.ts" },
"schemaVersions": [
{ "version": 1, "through": "0001_initial" },
{ "version": 2, "through": "0002_add_task_estimate" }
],
"tables": [
{ "name": "tasks", "scopes": ["project:{project_id}"] },
{
"name": "docs",
"scopes": [
"org:{org_id}",
{ "pattern": "project:{projectId}", "column": "project_id" }
]
}
],
"subscriptions": [
{ "name": "projectTasks", "scopes": { "project_id": "{projectId}" } }
],
"extensions": {}
}| Key | Required | Semantics |
|---|---|---|
| manifestVersion | yes | Must be 1. Format growth happens by bumping this, not by tolerating unknown keys. |
| migrations | no (default ./migrations) | Directory of NNNN_name/up.sql migrations, relative to the manifest. |
| queries | no (default ./queries) | Directory of .sql / .syql named-query files (see §6). Only read when some output requests a queries file. |
| naming | no (default "camel") | "camel" maps snake_case SQL names to camelCase in generated row types/params (projections lower with AS aliases so runtime keys match); "preserve" keeps SQL-truth names. Swift, Kotlin, and Dart row fields are camelCase in both modes. In both modes, generation fails on two table or column names that differ only in letter case (typegen.duplicate_identifier), two tables or columns mapping to one generated name on an emitted target (typegen.identifier_collision), and a generated field that is a reserved word on an emitted target or starts with _ on Dart (typegen.reserved_identifier). |
| queryBackend | no (default "auto") | Advanced revision-1 SYQL lowering override: "neutralize" emits guarded statements, "variants" enumerates activation states, and "auto" deterministically enumerates at ≤ 2 activation controls. This never changes source meaning or the public API. |
| output.ir | no (default ./syncular.ir.json) | IR output path, relative to the manifest. |
| output.module | no (default ./syncular.generated.ts) | Generated TS module path, relative to the manifest. |
| output.queryIr | no | Deterministic analyzed QueryIR JSON. Its hash keys generated reactive query descriptors, so SQL-only changes invalidate caches. |
| output.queries | no | Opt-in TS named-queries output path (see §6). A sibling .ts file. |
| output.swift | no | Opt-in Swift emitter (see §5). A path string, or { "path", "enumName", "queriesPath" } (default enumName SyncularSchema; queriesPath opts into §6 Swift named queries). |
| output.kotlin | no | Opt-in Kotlin emitter (see §5). A path string, or { "path", "package", "objectName", "queriesPath" } (defaults syncular.generated / SyncularSchema). |
| output.dart | no | Opt-in Dart emitter (see §5). A path string, or { "path", "queriesPath" }. |
| output.rust | no | Opt-in Rust named-query emitter (see §6). An object { "queriesPath", "clientCrate"? }; clientCrate defaults to syncular_client. Rust consumes neutral schema IR rather than a generated schema source file. |
| schemaVersions | yes, non-empty | The §1.5 version history. See below. |
| tables | yes, non-empty | Synced tables. Array order is the handler-declared bootstrap order (§4.7) and flows unchanged into the IR and schema.tables. |
| tables[].name | yes | Must be created by a migration. |
| tables[].scopes | yes, ≥ 1 | §3.1 scope patterns: 'prefix:{variable}' (column = variable) or { "pattern", "column" }. The column must exist on the table; one variable must not map to two different columns. |
| tables[].serverColumns | no (default []) | Columns the server sets inside the push transaction (§6.7). The IR marks each serverOwned: true; the TS insert and update types omit it. Unknown, primary key, NOT NULL, crdt and encrypted columns fail with typegen.server_column_unknown, _primary_key, _not_null, _crdt and _encrypted. See Server-owned columns. |
| tables[].extensions | no (default {}) | Opaque passthrough into the IR table's extensions slot. |
| subscriptions | no (default []) | Unit templates (SPEC §4.1). name must be a valid JS identifier and unique. A template names no table: its unit covers every table that declares all of its scope variables. A template whose variables no single table declares fails with typegen.subscription_uncovered. |
| subscriptions[].scopes | yes, non-empty | Map of variable → one value. The value is either a literal or exactly "{param}". Partial templates ("p-{x}") and "*" (§3.2 rejects requested wildcards) fail with typegen.subscription_invalid_value. Several values of one variable are several templates. An entry with a table key or a value list (the wire version 3 shape) fails with typegen.subscription_legacy_shape, whose message names this shape. |
| extensions | no (default {}) | Opaque passthrough into the IR document's extensions slot. |
Unknown keys are hard errors at every level, except inside extensions
objects, which pass through verbatim.
schemaVersions semantics. Each entry is { "version", "through" }:
version is an integer ≥ 1, strictly increasing across entries; through
names the last migration that version includes. Entries partition the
migration list in order: version n covers every migration after the
previous entry's through up to and including its own. Gap rule: the
final entry's through must be the final migration — a migration not
covered by any version is a hard error (so is a through naming a
missing or out-of-order migration). Locked versions: the migration lock
records the version of every locked migration. A locked migration keeps its
version (typegen.migration_version_changed otherwise), and a migration added
after the lock belongs to a version above the highest locked one
(typegen.schema_version_locked otherwise). A deployed version therefore never
gains a column that its deployed clients would not reset for (§7.4).
Immutable migration history
syncular.migrations.lock.json is the version-controlled deployment baseline.
Format 3 records every migration name, its SHA-256 checksum (line endings
normalized), and the schema version it belongs to, plus one canonical
head-schema column snapshot used only for diagnostics. It contains no SQL, filesystem path, row data, or runtime database
state, and grows with migration metadata plus the current schema rather than
repeating the full cumulative schema after every migration.
For an existing project adopting this contract, review the current history once, then run:
syncular migrations baseline --manifest-dir .
git add syncular.migrations.lock.json
syncular migrations check --manifest-dir .Projects with a committed format-1 or format-2 lock run one upgrade before
generating. upgrade-lock validates the locked history, records each locked
migration's schema version from the manifest's schemaVersions, and leaves
migrations appended after the old lock for generate to lock. Run it on a
manifest whose schemaVersions still describe the deployed versions:
syncular migrations upgrade-lock --manifest-dir .
git add syncular.migrations.lock.json
syncular migrations check --manifest-dir .The upgrade command refuses an already-current lock, and generate never
changes the lock format.
After that, a deployed migration is immutable. Restore an accidentally edited
migration and express the repair in a new NNNN_name/up.sql; do not delete and
re-baseline the lock. Existing table layouts are append-only and added columns
must be nullable so stored payloads and every server backend can upgrade
safely. The one in-place change is ALTER TABLE t ALTER [COLUMN] c DROP NOT
NULL, which makes a required column nullable (SPEC §2.4); typegen refuses it
for the primary key, an unknown column and an already nullable column. A SQL DEFAULT does not make a required append safe: existing
Syncular row payloads remain the source of truth, and no storage backend may
silently invent a value for them. baseline, migrations check, and
generate therefore reject ALTER TABLE … ADD COLUMN … NOT NULL, including
one with a literal default.
Data changes and backfills
Migration SQL is schema-only. UPDATE, INSERT, and DELETE do not mutate
accepted Syncular row payloads and are rejected before their inner SQL is
parsed, with a diagnostic linking to this rollout contract. Keep the old
representation available until the new one is proven complete.
Use an expand/backfill/validate/retire rollout instead: add the trailing column as
nullable; deploy that schema; fill existing rows with versioned,
server-authoritative writes under a new idempotency key; then enforce the
required value in host validation. Validate the backfill and every supported
client before retiring an old column or table in a later schema version. Do not
tighten the synced column's nullability in a later migration. generate
appends a valid new migration to the lock, while
generate --check fails until that updated lock and generated outputs are
committed. Drift diagnostics name the locked migration and the first affected
table/column without printing SQL or local paths.
Every-migrated-table rule. Every table created by the migrations must
appear in tables. A migrated-but-unlisted table is a hard error, not a
silent skip; internal/unsynced tables will be a future manifest flag.
2. IR reference (irVersion 1)
The IR is the language-neutral contract that all emitters (TS today, Swift/Kotlin later) consume. No TS types appear in it — column types are the six §2.4 names.
{
"irVersion": 1,
"schemaVersion": 2,
"schemaVersions": [{ "version": 1, "migrations": ["0001_initial"] }],
"tables": [
{
"name": "tasks",
"primaryKey": "id",
"columns": [{ "name": "id", "type": "string", "nullable": false }],
"scopes": [
{
"pattern": "project:{project_id}",
"variable": "project_id",
"column": "project_id"
}
],
"extensions": {}
}
],
"subscriptions": [
{
"name": "projectTasks",
"scopes": [
{
"variable": "project_id",
"value": { "kind": "parameter", "name": "projectId" }
}
]
}
],
"extensions": {}
}schemaVersionis the current (last) version;schemaVersionsis the full history with the migrations each version added, in application order.tablesis in manifest order (bootstrap order).columnsis in SQL declaration order (CREATE columns, then ADD COLUMNs) — this is the §2.4 row-codec positional order; emitters must never reorder it.scopesentries are pre-resolved:variableandcolumnare materialized so non-TS emitters never re-parsepattern.indexes(additive,irVersion1) is a per-table array of{ "name", "columns": [...], "unique" }in declaration order — emitted only when the table declares at least one index, so index-free tables (every pre-index manifest) keep byte-identical IR + generated output. It is a client-side/query-check concern; see §3's index note.subscriptions[].scopesis sorted byvariable; eachvalueis{ "kind": "literal", "value" }or{ "kind": "parameter", "name" }.- Determinism: equal inputs produce byte-identical output — fixed key order (as shown above), 2-space indent, LF, trailing newline, extension objects recursively key-sorted. The IR file diffs cleanly.
- Identity: the IR's identity is
sha256:<hex>over the exact IR file bytes (UTF-8). Every emitter must stamp this hash into its generated output header so--check-style freshness verification works per language. extensions(document- and table-level): reserved, opaque passthrough from the manifest — the designed-once home for the WP-49 apply/read-model hooks. Always present,{}when unused. Emitters must round-trip unknown extension content without interpreting it.- Head-version-only:
tablesdescribes the current schema version only. Per-version table snapshots (for serving multiple row codecs from one IR) are a planned additive change under a futureirVersionbump;schemaVersionsalready carries the migration grouping needed to build them.
3. SQL subset
Migrations live in NNNN_name/up.sql (numeric prefix orders them;
duplicate ordinals are errors). The parser accepts exactly:
CREATE TABLE [IF NOT EXISTS] name ( column-defs…, [PRIMARY KEY (col)] ) [WITHOUT ROWID]ALTER TABLE name ADD [COLUMN] column-defALTER TABLE name ALTER [COLUMN] col DROP NOT NULL(a required column becomes nullable, SPEC §2.4)CREATE [UNIQUE] INDEX [IF NOT EXISTS] name ON table ( col [, col…] )DROP INDEX [IF EXISTS] nameDROP TABLE [IF EXISTS] nameCREATE VIRTUAL TABLE [IF NOT EXISTS] name USING fts5( text-col [, text-col…], content = table [, tokenize = 'allowlisted tokenizer'] )- column-def:
name TYPE [PRIMARY KEY] [NOT NULL] [NULL] [DEFAULT literal] [REFERENCES parent(pk) [ON DELETE RESTRICT | CASCADE | SET NULL]] --line comments and/* … */block comments
Type map (case-insensitive) to the six §2.4 types plus the two semantic types:
| SQL keyword | IR type |
|---|---|
| TEXT | string |
| INTEGER, INT, BIGINT, SMALLINT | integer |
| REAL, FLOAT, DOUBLE | float |
| BOOLEAN, BOOL | boolean |
| JSON, JSONB | json |
| BLOB, BYTEA | bytes |
| BLOB_REF, BLOBREF | blob_ref: a nullable blob reference string holding the canonical BlobRef document (§5.9) |
| CRDT | crdt: opaque server-merged CRDT bytes, crdtType defaults to yjs-doc (§5.10) |
Nullability: NOT NULL (or being the primary key) → non-nullable;
otherwise nullable. Exactly one single-column primary key per table,
inline or table-level.
Indexes (CREATE [UNIQUE] INDEX). A migration may declare local
secondary indexes on an already-created table: CREATE INDEX name ON table
(col), a compound … (a, b) (order preserved), the UNIQUE variant, and
IF NOT EXISTS. Each index becomes an IrTable.indexes entry
({ name, columns, unique }, declaration order) and is materialized by the
clients (the TS web-client mirror + the Rust core's base/visible table pair),
typegen's named-query type-check DB, and the server's relational app-table
projection. Index columns must exist on the table; index names must be unique
across the schema; a column must not repeat within one index. The column list
is bare column names: ASC/DESC, expression columns (lower(a)), and
partial (WHERE …) indexes are hard errors (the IR models column names
only, so accepting a direction/expression would silently drop it).
Synced table, column, and secondary-index identifiers are checked against the
same portable relational rules used by server schema compilation. Names use a
maximum of 63 UTF-8 bytes (not JavaScript characters), and sync_/_sync
remain reserved for Syncular storage. Generation names the migration, object,
actual byte count, and portable limit before writing the migration lock or any
generated artifact. Exactly 63 bytes is accepted. Bare Unicode identifiers are
supported; quoted identifiers remain outside the migration subset.
Index replacement (DROP INDEX). A previously declared secondary index
may be removed with DROP INDEX name or conditionally with IF EXISTS.
Generation removes it from the head IR; a later CREATE INDEX may reuse the
name with a different uniqueness or column definition. On a server schema
bump, Syncular rebuilds the declared secondary indexes for its owned
relational projection tables before advancing the version marker. Client
mirrors already wipe and re-bootstrap application tables on every schema
version change. Portability validation applies to the final head schema: an
already-locked historical index with a non-portable name may therefore be
dropped and replaced in a forward migration, while a non-portable name that
remains in the head still fails generation.
Table retirement (DROP TABLE). A table created earlier in migration
history and dropped before the head version is absent from the IR and must be
omitted from the manifest's tables list. IF EXISTS is supported. Reusing a
dropped table name is rejected: the head-only IR cannot distinguish a new
table from an incompatible in-place rewrite on an upgrading server. On a
schema-version bump, clients wipe and re-bootstrap their synced tables; the
reference relational server drops the retired current-row table and its live
scope index. Append-only commit history follows the configured retention
policy—DROP TABLE is schema retirement, not a compliance erasure API.
Local FTS5 projections (CREATE VIRTUAL TABLE … USING fts5). The authored
content = table option declares which already-created synced table owns the
projection; it is not passed through as SQLite external-content mode. Typegen
attaches { name, columns, tokenize } to that table's ftsIndexes array and
emits it in TypeScript, Swift, Kotlin, and Dart schema values. The TypeScript
and Rust clients create a contentful FTS table with a private stable source-ID
column and deterministic maintenance triggers. FTS is local-only: it is not a
wire/server table, cannot be subscribed or mutated, and maps back to its owner
for named-query invalidation. Columns must precede options, be distinct
declared strings, and number 1–32. Encrypted declared-string columns are
allowed because the projection is built only from the decrypted local mirror;
ciphertext remains the only wire and server representation. content is
required. Supported tokenizers are unicode61 (default), its
remove_diacritics 0|1|2 variants, porter unicode61, and trigram. Other
modules/options/tokenizers or schema-object name collisions fail generation.
There is no automatic LIKE fallback when FTS5 is unavailable; local schema
creation fails loudly.
Hard errors (each names the construct and source file): any other
statement (CREATE TRIGGER/VIEW, DML, ALTER … RENAME, …); unknown
or parameterized types (VARCHAR(36)); quoted identifiers
("t", `t`, [t]); table constraints (FOREIGN KEY, UNIQUE,
CHECK, CONSTRAINT); column constraints beyond the list above
(CHECK, COLLATE, …); DEFAULT (expression) and a DEFAULT followed by
a word other than NULL, TRUE, FALSE, or CURRENT_TIME/DATE/TIMESTAMP
(DEFAULT NOT NULL fails with typegen.invalid_default); composite
or missing primary keys; PRIMARY KEY on ADD COLUMN; duplicate
tables/columns; ASC/DESC, expression, or partial (WHERE) index columns; a
duplicate or unknown-column index; unsupported virtual-table modules or FTS5
options; an FTS projection without an owner or with invalid columns;
trailing clauses (STRICT); a declared reference whose column no longer has
a single-column index after replay, for example after DROP INDEX of its
generated idx_<table>_<column>_ref (typegen.reference_index_missing; the
server reaches children through that index, §6.11).
DEFAULT literals on CREATE TABLE are accepted and ignored, including
signed numbers (DEFAULT -1): typegen
extracts the schema shape; executing migrations (where defaults matter) is
the host's job. Recording them is not needed by any emitter today. On
ALTER TABLE … ADD COLUMN, a literal default does not make a non-null append
safe and is rejected as described in the migration-lock section above.
Constraints: support and enforcement
Typegen accepts the declarations below. Unsupported constraints fail during parsing; accepted declarations are enforced at the listed layers.
| Declaration | Accepted | Enforced by |
|---|---|---|
| Column NOT NULL, or a primary-key column | yes | The §2.4 row codec, logically: the server candidate commit and every client write/apply reject a null in a non-nullable column. The TS local mirror also declares NOT NULL; the Rust local tables carry bare column names and rely on the codec. |
| Single-column PRIMARY KEY, inline or table-level | yes | The row codec validates the key. Both client mirrors declare a primary key; the server projection keys rows by partition and the encoded row identity. |
| Column DEFAULT literal | accepted, ignored | Nothing. Typegen records no default and fills none; the host runs the migration SQL where a default matters. |
| Nullable ALTER TABLE … ADD COLUMN | yes | A physical column in every mirror. A SQL DEFAULT on the same statement is ignored and never backfills Syncular payloads. |
| ALTER TABLE … ADD COLUMN … NOT NULL | rejected | not applicable |
| ALTER TABLE … ALTER [COLUMN] … DROP NOT NULL | yes | The column becomes nullable in the IR. The server drops the projection's NOT NULL at the bump (PostgreSQL in place, SQLite and D1 by a table rebuild) and keeps every payload; the bump raises the schema floor to its version (SPEC §2.4). |
| Any other ALTER COLUMN action | rejected | not applicable |
| Column REFERENCES parent(pk) [ON DELETE RESTRICT\|CASCADE\|SET NULL] | yes | The server commit boundary only (§6.11); the local SQLite DDL deliberately omits the clause. |
| CREATE [UNIQUE] INDEX over existing columns | yes | A physical index in both client mirrors, the server relational projection, and typegen's named-query type-check DB. The UNIQUE variant additionally enforces uniqueness atomically on a local write (§7.2: sync.constraint_violation, no outbox entry, no revision advance) and on the server commit. |
| Inline column CHECK (expr) | rejected | not applicable (see the gap below) |
| Table-level UNIQUE (…), CHECK (…), CONSTRAINT name …, FOREIGN KEY … | rejected | not applicable (see the gap below) |
| Column COLLATE … | rejected | not applicable |
The closed-value-set gap. The IR column (IrColumn) carries no enum or
check metadata, so a closed value set such as a state column cannot be
declared in migration SQL. Inline CHECK is rejected twice over: CHECK (n)
reads CHECK as an unsupported column constraint, and CHECK (n > 0) fails at
> because the lexer does not read comparison operators. Table-level CHECK
and CONSTRAINT are rejected as unsupported table constraints. Declare the
column with the type it stores and put the rule in a §6.7 server
write-validator, which validates each candidate row operation during commit
application. A ValidationRejection carries the host rejection code. That validator does not run on optimistic local
writes, so an application that needs immediate local feedback repeats the rule
in its own pre-write guard. The physical server projection does not carry an
app CHECK either; on a fresh CREATE TABLE it declares NOT NULL per the
schema, while columns appended by ALTER TABLE are physically nullable and the
row codec remains the type authority.
Uniqueness without table-level UNIQUE. SQLite accepts UNIQUE (a, b),
but the IR models uniqueness as a named CREATE UNIQUE INDEX entry, so
typegen rejects the table constraint. Declare
CREATE UNIQUE INDEX name ON table (a, b) instead. It is accepted with
IF NOT EXISTS, preserves column order, and is materialized by every mirror.
NULL values never conflict, matching SQLite unique-index semantics.
Table-level FOREIGN KEY stays rejected: a reference is a column constraint
(§6.11).
4. Generated-module contract
The emitted *.generated.ts module:
- Header:
// Generated by @syncular/typegen — DO NOT EDIT.plusirVersionand theirHashof the IR bytes it was produced from. - Zero imports.
schemasatisfies the server'sServerSchematype structurally (verified by the integration test) — using the generated file adds no dependency edge to@syncular/server. - Exports per table
T(PascalCase of the table name):TRow— one field per column in row-codec order; nullable columns are… | null.TInsert— non-nullable columns required; nullable columns optional (?: … | null).TUpdate— primary key required; every other column optional.- Insert/Update are client-side input conveniences only — the wire stays full-row upserts (§6.1); nothing partial is ever encoded.
- Exports per subscription
s:sSubscriptionwithnameand ascopes(params)builder returning the unit's scope mapRecord<string, string>, whichclient.subscribe(...)takes, plus anSParamsinterface when the template has{param}placeholders.
Lint/freshness split: *.generated.ts is excluded from oxlint and oxfmt
(see .oxlintrc.json / .oxfmtrc.json) — hand-format rules on machine
output only create churn.
Freshness and migration-history integrity are enforced instead by syncular
generate --check, which is byte-exact and refuses locked-history drift; the
generated fixture is still typechecked by tsrs and exercised by tests.
5. Native emitter contracts (Swift / Kotlin / Dart)
The same IR feeds three additional emitters, opt-in per manifest via
output.swift / output.kotlin / output.dart. TS stays the default
(always emitted); a native emitter runs only when its key is present. Every
native file carries the same discipline as the TS module: the
// Generated by @syncular/typegen — DO NOT EDIT. header, irVersion,
and the irHash of the IR bytes it was produced from — so syncular
generate --check gates freshness byte-exactly per language. Each language's
golden fixture lives under test/fixtures/basic/ and is asserted
byte-exactly + for determinism by test/golden-native.test.ts (the TS
golden's pattern). The emitters are dependency-free string builders; the
generated files depend only on the wrapper package's JSONValue/JsonValue.
Each generated file exports, per language:
- a schema constant parsed from a string literal that holds the exact
schema JSON the TS module's
schemaholds, with its references and indexes (test/golden-native.test.tsasserts the bytes are identical in all four languages); pass it straight to the wrapper'screate(schema:); - one typed row type per table with a
fromRowfactory that lifts aquery/readRowsrow into the typed shape (returning null/nil when a non-nullable column is missing or mistyped); - one unit-template builder per subscription,
{param}placeholders becoming typed function arguments. It returns the unit's scope map ([String: String],Map<String, String>), which the binding'ssubscribetakes.
| | Swift | Kotlin | Dart |
|---|---|---|---|
| Schema | enum <EnumName> { static let schema: JSONValue } | object <ObjectName> { val schema: JsonValue } | final Map<String, Object?> syncularSchema |
| Row | struct <Table> + init?(row: [String: JSONValue]) | data class <Table> + fromRow(row: JsonValue) | class <Table> + static <Table>? fromRow(Map<String, Object?>) |
| Subscription | enum subscriptions.<Name> w/ scopes(...) | object Subscriptions.<Name> w/ scopes(...) | class Syncular<Name>Subscription w/ static scopes(...) |
| Imports | Foundation, Syncular | dev.syncular.JsonValue | dart:convert |
Type mapping (the six §2.4 types + blob_ref/crdt):
| IR type | Swift | Kotlin | Dart | notes |
|---|---|---|---|---|
| string | String | String | String | |
| integer | Int | Long | int | JVM widens to Long; SQLite integers ride as JSON numbers |
| float | Double | Double | double | |
| boolean | Bool | Boolean | bool | see boolean note below |
| json | String | String | String | the raw canonical JSON string |
| bytes | [UInt8] | ByteArray | List<int> | decoded from the core's {"$bytes":"<hex>"} marshaling |
| blob_ref (§5.9) | String | String | String | the raw canonical BlobRef JSON string; the client's blob API parses it |
| crdt (§5.10) | [UInt8] | ByteArray | List<int> | opaque bytes; the Y.Doc accessor is an app-level helper, not generated |
Booleans and SQLite 0/1. SQLite has no boolean type — it stores 0/1.
fromRow therefore accepts either a real JSON boolean or a number
(0 → false, non-zero → true), in every language. This is the honest
mapping: a row read back from the local SQLite mirror surfaces done as 1,
while an optimistic in-memory row may surface it as true; both decode to the
typed Bool/Boolean/bool.
Nullability. A nullable column becomes an optional property (T? in
Swift/Kotlin, T? in Dart) and fromRow tolerates its absence; a
non-nullable column that is missing/mistyped makes fromRow return
null/nil (fail-soft at the decode boundary, not a crash).
6. Named queries (the .sql → typed-function tier)
The named-query tier is syncular's cross-platform answer to sqlc /
SQLDelight: you write a query file, and typegen transpiles it into a typed
function on every platform — killing query↔type drift by construction. It is
the type-safe read tier; raw query(sql, params) — guarded read-only —
stays the escape hatch for queries built at runtime.
Two frontends feed one QueryIR pipeline. .sql is plain SQL plus
:params, the compatibility floor documented below. .syql is the
revision-1 structured frontend: authoritative typed inputs, explicit
when(...) conjuncts, atomic optional groups, imported hygienic predicates,
inferred scope dependencies, explicit sync coverage, complete sort profiles,
bounded limits, and inferred identity. Its complete normative definition is
../../docs/SYQL.md. Both frontends produce the same
deterministic QueryIR consumed by every language emitter.
Tooling: syncular generate --print <name> prints every selected checked
statement and bind, syncular fmt is the semantic-preserving canonical
formatter, and syncular lsp provides project-aware diagnostics, hover,
definitions, references, symbols, and formatting. Casing follows the manifest
naming mode (§1): generated rows/inputs are camelCase and projections are
AS-aliased so runtime keys match.
import { matchesTitle } from "./predicates.syql";
sync query listTodos(
listId,
status?: string | null,
range?,
q?: string,
unassigned: bool = false,
) {
select id, title, status, created_at
from todos
where todos.list_id = :listId
and when(status) status is :status
and when(range) created_at between :range
and when(unassigned) assignee_id is null
and when(q) matchesTitle(:q)
order by sortBy default newest {
newest: created_at desc, id desc;
oldest: created_at asc, id asc;
}
limit pageSize default 50 max 200;
}Optional nullable scalars use a generated presence wrapper so absent,
present-null, and present-value remain distinct. A group is one optional host
object whose members are all required. A bool = false flag activates on
true. Sort is a generated enum/union of complete checked profiles; limit is validated as a
positive bounded integer before execution. integer inputs are exact signed
64-bit values in the SYQL API (bigint in TypeScript,
Int64/Long/int on Swift/Kotlin/Dart, and i64 on Rust).
File & naming convention. Named queries live in a queries/ directory
next to migrations/ (override with the top-level "queries" manifest key).
The directory is walked recursively — subfolders are pure organization.
Each language's queries file is a separate output (output.queries for TS,
and queriesPath on output.swift/output.kotlin/output.dart plus
output.rust.queriesPath), so
schema-only consumers never churn when a query changes. A queries output is
opt-in per language; when none is requested the queries/ dir is not even
read.
A query's default name is its path relative to the queries root — every folder segment plus the filename stem, joined and camelCased:
| Path | Default name |
|---|---|
| list-todos.sql | listTodos (flat files are unchanged) |
| billing/invoices/list.sql | billingInvoicesList |
| reporting/tasks-by-priority.sql | reportingTasksByPriority |
Every path segment and the filename stem must be lowercase kebab-case; a
stray List.sql or Billing/ is a hard error naming the offending
segment. (Flat single-statement layouts keep exactly today's names, so
existing consumers are unchanged.)
Name override. A comment line -- name: billingInvoicesList in a
statement's leading comment block overrides the full name verbatim (it must
be a valid camelCase identifier — validated loudly). The override is
deliberately the marker form (-- name: ident), not a bare
-- ident one-word comment: a plain prose comment must never silently
rename a query, so a stray -- todos above a statement is ignored and the
path-derived name stands. Prose comments remain allowed anywhere (and are
stripped from the emitted SQL).
One or many statements per file. A file may hold multiple statements,
split on top-level ; (a small splitter that respects single-quoted
strings, -- line comments, and /* … */ block comments; the SELECT-only
subset has no BEGIN/END blocks, so top-level semicolons are unambiguous, and
a trailing ; on the last statement is optional). The rule:
- a file with one statement may omit
-- name:— it gets the path-derived default; - a file with multiple statements requires
-- name:on every statement — a missing one is a generate-time error naming the file and the statement's position/first line.
-- name: and -- param :x <type> comments are scoped to the statement they
directly precede (the leading comment block between the previous statement's
; and this statement's first token), not file-header-scoped.
Global uniqueness. After collection, a duplicate name anywhere in the
manifest is a hard error listing both source locations (file + statement
position, e.g. dup.sql#1 / dup.sql#2). The filesystem no longer guarantees
uniqueness once -- name: overrides exist, so this is checked explicitly.
SELECT-only. Named queries are reads. Any statement that is not a SELECT
is a hard error at generate time, pointing at mutate() — writes go
through the outbox (SPEC §7.1).
Type-checking via SQLite itself (the load-bearing trick). At generate time
typegen synthesizes the schema's DDL from the IR (a CREATE TABLE per table,
the reverse of the migration parser), builds an in-memory bun:sqlite
database, and prepare()s each query. SQLite is the correctness authority:
a bad table/column reference or a syntax error throws with SQLite's own
message. The prepared statement then yields the result column names +
declaredTypes (SQLite's sqlite3_column_decltype).
For FTS-backed named queries the synthesized type-check database creates the
same declared columns plus _syncular_source_id UNINDEXED, and the client's
source-id mapping table. The emitted SQL reads every _syncular_source_id
reference it can bind through that mapping table on the projection rowid, so
FTS5 does not fetch a content row per match (SYQL §15); the authored SQL keeps
its types and identity. MATCH :query
infers a string parameter, and the FTS table reference is folded into its
owning synced table for dependency and scope-coverage metadata. bm25,
highlight, and snippet are accepted deterministic FTS5 auxiliary
functions only when the query references a schema-declared FTS projection;
they use the normal computed-expression fidelity rules. Arbitrary extension
functions remain outside the portable profile. A bounded FTS join must project
the FTS table's _syncular_source_id as well as every joined table's primary
key and end its total order in those identity fields. Typegen then proves the
composite row key instead of weakening bounded-query stability rules.
Typing fidelity (what bun:sqlite actually exposes, and its honest
boundary):
| Result column | Type source | Nullability | Fidelity |
|---|---|---|---|
| plain column ref (title, t.title, title AS x) | resolved to the IR column (decltype confirms) — exact IR type, incl. json/blob_ref/crdt | the IR column's nullability, lifted to nullable whenever LEFT, RIGHT, or FULL OUTER JOIN can null-extend its table alias | exact |
| computed expression (count(*), CAST(done + 1 AS INTEGER)) | decltype is null → the target type of a whole-expression CAST, or integer for a whole-expression count(...); any other expression fails with typegen.computed_column_type | always nullable (an expression's nullability is not knowable from decltype) | fallback |
Every local synced table also has the protocol-owned _sync_version column.
Named queries may project it with an explicit public alias when an application
needs the observed server version for optimistic concurrency:
query getTodo(listId, todoId) {
select id, title, _sync_version as server_version
from todos
where todos.list_id = :listId and todos.id = :todoId;
}The generated serverVersion field is an exact, non-null integer for a base
or required join relation; it is nullable when projected from an optional side
of an outer join. The
physical name is intentionally not a public result name: project it with an
alias such as server_version. _sync_version is query-only, is excluded from
schema and mutation types, and is not added by select *; this prevents client
code from writing engine-owned concurrency state. Pass a positive observed
value as a mutation baseVersion. Locally-created or still-unacknowledged rows
can carry a non-positive sentinel and should omit the base version until the
server assigns one.
bun:sqlite exposes columnNames, declaredTypes (once executed once — we
run the statement against the empty DB), and paramsCount. It does not
expose column origin (table/column), param names, or NOT NULL flags — so
typegen resolves plain refs against the IR itself (parse the SELECT list +
FROM/JOIN, match name/alias.name) to attach schema nullability plus SQL
outer-join null-extension, and parses :name placeholders from the SQL text
for param names. Each join chain is evaluated left-to-right: LEFT lifts the
new right relation unit, RIGHT lifts every prior relation in that chain, and
FULL lifts both. A parenthesized relation group is one unit for its enclosing
join while its nested join order remains intact. INNER, CROSS, and ordinary
JOIN do not remove nullability already introduced earlier in the chain.
Unqualified coalesced USING/NATURAL columns are not assigned a guessed
physical origin and stay nullable. A computed column takes its type from the
query text (SYQL §14.4): wrap the expression in
CAST(... AS <type>). A cast that is an operand, as in CAST(a AS INTEGER) + 1,
does not type the column.
Parameters use the :name convention. A param's type is inferred where
it compares against a plain column ref — WHERE list_id = :listId (equality,
</>/<=/>=/!=/LIKE/IS) or col IN (:a, :b) — taking that column's
IR type. Where inference is ambiguous (compared to an expression, used only in
a projection, …) a per-statement -- param comment (in that statement's
leading comment block) overrides:
-- param :sinceMs integer
SELECT id, title FROM tasks WHERE estimated_at > :sinceMs + 0Missing both inference and a comment is a generate-time error naming the
param and the fix. A -- param comment for a param the query does not use is
also an error. The inference is deliberately simple and honest.
Query plan check. Generation runs EXPLAIN QUERY PLAN on every emitted
statement (each physical statement of a .syql query) against the in-memory
schema database, which holds the declared indexes and no statistics, and fails
on four findings (SYQL §14.5):
| Code | Finding |
|---|---|
| typegen.query_scan | A scan that an index on one column of the scanned table removes; columns lists those columns |
| typegen.query_automatic_index | A transient index SQLite builds on every run for a join |
| typegen.query_unbounded | A top-level scan in a statement with no WHERE and no LIMIT outside parentheses |
| typegen.query_many_tables | More than four tables, each a reactive dependency |
The details carry query, file, the statement index, its positional sql,
the table and the plan lines (tables for many_tables). Declare the
index, change the query, or acknowledge the finding on the statement with a
reason. A .sql acknowledgement is a leading comment beside -- name: and
-- param; a .syql acknowledgement is an allow line above the query:
-- allow scan tasks: a project holds at most 200 tasks
SELECT id, title FROM tasks WHERE priority = :priorityallow scan tasks 'a project holds at most 200 tasks'
query tasksByPriority(priority) {
select id, title from tasks where priority = :priority;
}An acknowledgement without a reason fails with
typegen.query_acknowledgement_invalid (SYQL2013_INVALID_ACKNOWLEDGEMENT in
.syql), and one that matches no finding of its query fails with
typegen.query_acknowledgement_unused.
A query that only the server runs as a registered query takes
allow plan '<reason>' (-- allow plan: <reason> in .sql) instead: it
acknowledges every finding of that query, because the server's database plans
it with the server's indexes.
Emitted SQL. Wrappers take positional params (query(sql, params[])),
so the emitted SQL rewrites :name → ? (repeats included) and each runner
reorders the named params object into the positional array. Comments are
stripped and whitespace collapsed to a clean one-line string.
Reactive metadata. Each query emits a QueryIR-derived cache id, the
FROM/JOIN table set, table-associated scope dependencies, provable window
coverage, and a safe row identity when the primary key is projected by an
unambiguous single-table query. React uses these facts to share one query per
revision, route exact change batches, claim window coverage, and retain
unchanged row objects. The cache id is derived from the query IR rather than
the schema IR, so a SQL-only change cannot reuse old state. Boundary:
bun:sqlite exposes no authorizer and no statement table-list, and EXPLAIN
opcodes are fragile — so the set is derived by scanning every FROM/JOIN
identifier and keeping those that name an IR table. This captures subquery /
WHERE EXISTS (…) tables (they still appear under FROM/JOIN), and the
prepare() still guarantees the SQL itself is correct.
Inference is deliberately conservative around OR, grouping, ambiguous joins,
and computed identity. Ordinary scope predicates construct exact reactive facts;
sync query additionally requests checked coverage:
sync query compareLists(left, right) {
select id, title from tasks
where tasks.project_id in (:left, :right);
}Coverage must resolve every read schema table and bind at least one declared
scope on every table instance. Select the unit with by for a multi-scope
table. The descriptor includes all proven dimensions; unbound dimensions are
omitted. A required scope bind can propagate across a qualified
scope-column equality in an unconditional outer WHERE clause or a simple
mandatory ON clause. The analysis holds aggregate coverage for every table,
and readiness requires all of it. The generated descriptor's coverage(params)
returns window bases (SPEC §4.8), { base: { variable, fixedScopes? }, units }:
a base names no table, so tables proven by the same base share one entry, and
a fixed scope holds one value. A self-join claims coverage when
every instance of the table carries the same proof (the same scopes, operators,
and parameters); the descriptor then holds one coverage entry and one
dependency for that table. An unproven instance or differing proofs fail with
SYQL6005_INVALID_SYNC_QUERY. Bound fixed dimensions retain intersection
semantics. Scoped remote operations still require every declared scope. ON clauses containing OR, NOT, or nested
SQL cannot claim coverage. Only
required, non-null, exactly typed binds are allowed. Result identity is inferred
from schema keys and the projection; a self-join has no inferred identity. Without constructive proof an ordinary
query falls back to table-wide dependency, no coverage, and/or unkeyed
reconciliation; a sync query fails generation rather than claiming partial
completeness.
Use explicit JOIN ... ON relations. SQLite comma joins are rejected because
the compiler will not emit reactive metadata unless every read relation is
represented by the same proof model.
Emitted shape, per language (abbreviated):
| | Output |
|---|---|
| TS | listTodosRow interface + ListTodosParams + listTodosTables + async listTodos(client, params): Promise<ListTodosRow[]> + a listTodosQuery descriptor for React |
| Swift | struct ListTodosRow + init?(row:) and <Enum>Queries.listTodos(client:listId:) throws -> [ListTodosRow] + listTodosTables |
| Kotlin | data class ListTodosRow + fromRow and <Object>Queries.listTodos(client, listId): List<ListTodosRow> + listTodosTables |
| Dart | class ListTodosRow + fromRow and syncularListTodosQuery(client, listId: listId): List<ListTodosRow> + syncularListTodosQueryTables |
| Rust | list_todos::{Row, Params, DESCRIPTOR, select, run, snapshot} with strict decoding, i64 fidelity, reactive dependencies, WindowCoverage, and proven row_key |
Each projection row is its own generated type per query (the drift-kill:
the row shape is exactly what the SELECT returns). Swift/Kotlin/Dart rows use
their wrapper fromRow mapping. Rust returns a query/column-specific error for
missing or malformed values and accepts the client's lossless $bigint and
$bytes envelopes. --check gates every queries file byte-exactly, like the
schema files; each carries the DO-NOT-EDIT header + IR hash.
Rust is object-only in the manifest:
{
"output": {
"rust": {
"queriesPath": "./src/syncular_queries.rs",
"clientCrate": "syncular_client"
}
}
}Each query is a snake-case module. Required inputs go through Params::new;
optional nullable inputs use SyqlPresence<Option<T>>; sort profiles are
closed enums; and select(¶ms) exposes the exact checked SQL and binds for
diagnostics. run executes a typed local read. snapshot returns typed rows,
the local revision, and coverage from one SQLite read snapshot. The generated
DESCRIPTOR contains the QueryIR hash identity, tables, dependencies,
coverage, plan selector, and optional proven row-key function. It deliberately
does not prescribe a Rust UI observer.
TypeScript named queries also emit one mapRow decoder per projection. The
direct async runner and the React descriptor share that decoder, so a boolean
field is always a JavaScript boolean on the first read and every reactive
re-read (0 → false, finite non-zero → true, an existing boolean is
preserved, and nullable booleans preserve null). A value that does not match
the analyzed result type throws QueryResultDecodeError; raw client.query()
and useRawSql() remain storage-shaped unless the caller supplies a mapper.
React helper. @syncular/react exports useQuery(query, params?),
which takes the TS NamedQuery descriptor and observes the client-scoped
revisioned store:
import { listTodosQuery } from './syncular.queries';
const todos = useQuery(listTodosQuery, { listId });
// todos.rows: ListTodosRow[]; todos.phase: loading | partial | ready | blocked | error