usm-core

Universal Schema Model — the storage- and transport-agnostic core.

A reflective, Hasura-style data-modeling engine for Node.js. This package holds everything that doesn't depend on a database or an API protocol:

• Vocabulary — SCALAR_TYPES, OPERATORS, OPERATORS_BY_TYPE, coerce.
• Schema — CollectionSchema (a table descriptor) and SearchSchema (a registry that resolves relationships and vends searchers). SearchSchema is storage-agnostic: it takes a searcher factory supplied by an adapter.
• Model bases — abstract Collection → KeyedCollection (key normalization + TTL/LRU cache over an abstract batchLoadByKeyValues) → ModelCollection (hydrated getByKey + search/aggregate), plus ModelType.
• Searcher contract — Searcher (implemented by adapters).
• Subscriptions — SubscriptionManager (cohort multiplexing + change-driven or polling refetch, pushing on change, or on every change to a table named in the subscription's relatedTables — data a cohort reads through a relationship but never fetches itself), StreamingSubscriptionManager (cursor-based streams), and the ChangeSource interface.

It depends on nothing GraphQL- or SQL-specific, so it can back multiple adapters (Postgres today) and multiple API layers (GraphQL today).

Install

npm install usm-core

Live queries and streams

The two subscription engines answer different questions, and both push { [table]: rows } payloads for an API layer to project.

A live query answers "what does this result set look like now". It re-reads its whole window whenever the data under it changes and pushes the result when it has actually moved, so late subscribers and reconnects see current state. Identical subscriptions share a cohort and one refetch.

A stream answers "what has arrived since I last looked". It reads rows in the order of a cursor column and delivers each one once, in batches, keeping a per-subscriber cursor that advances with every batch. That suits a table that grows — a log of results — where a live query would re-read and re-send the whole window on every write. Streams are not multiplexed: each one costs its own query.

import { StreamingSubscriptionManager } from "usm-core";

const streams = new StreamingSubscriptionManager(model.searchSchema, changeSource, {
    defaultBatchSize: 100,  // when a subscriber does not ask for a size
    maxBatchSize: 1000,     // and the most it may ask for
    pollInterval: 1000,     // how often to read tables the change source misses
    debounceInterval: 50    // how long to let a burst of writes settle first
});

// each payload carries the next batch of rows past the one before it
for await (const { election_candidacy_states: rows } of streams.subscribe("ElectionCandidacyState", {
    where: { race_id: { _eq: raceID } },
    cursor: { column: "modified", initialValue: lastSeen, ordering: "ASC" },
    batchSize: 500
}))
    report(rows);

Omit initialValue to stream a collection from the beginning; pass the last value a client saw to resume where it left off.

A cursor column should be one that only moves forward — a modification timestamp or a monotonic id. Rows sharing a cursor value are handled (each is delivered once, and one written later at the same value is still picked up), but a row written behind the cursor is not: the stream has already read past it.

Within one subscription a row goes out once. Across a reconnect, delivery is at-least-once wherever the value a client holds is coarser than the one the store compares it against: a Postgres timestamp keeps microseconds, the Date it is read back as keeps milliseconds, so resuming from …:09.857 replays the rows written in that millisecond. Clients that care should key on the primary key.

Companion packages

• usm-adapter-postgres — knex/Postgres adapter: concrete searcher (the where/order_by → SQL compiler), knex collections, a SearchSchema factory, and a LISTEN/NOTIFY ChangeSource.
• usm-api-graphql — turns a SearchSchema into Hasura-style GraphQL (queries, aggregates, live-query and streaming subscriptions).

Shape

                usm-core  (this package)
                   ▲
        ┌──────────┴───────────┐
 usm-api-graphql        usm-adapter-postgres

Adapters and API layers depend on core; never on each other.

Tests

npm test    # node --test, no dependencies