@x12i/xronox-store

Generic, schema-agnostic, multi-collection document store on top of @x12i/xronox: registry-backed collections, per-primary-key L1 cache (LRU + optional TTL), optional tombstone/visibility rules (caller-defined field names), query + cache merge for readMany, retry queue, and index setup on init().

Install

npm install @x12i/xronox-store

Requirements

deleteMany() calls xronox.deleteMany() on the underlying engine. Use @x12i/xronox 3.2.0+ (or any build that exposes deleteMany).

Cache defaults (resolveCacheConfig)

Every collection may set cache?: { maxSize?: number; ttlMs?: number }. If omitted or partial, the store merges package defaults:

| Option | Default | Meaning | |----------|---------|---------| | maxSize | 10000 | LRU cap on cached primary keys (insert may evict the oldest entry when full). | | ttlMs | 0 | 0 = no age-based eviction; entries stay valid until LRU eviction, explicit cache clear, or invalidation via writes. ttlMs > 0 drops stale entries on read. |

Use resolveCacheConfig(cache) (or DEFAULT_CACHE_CONFIG) so your own config layer matches the store’s merge rules.

import { resolveCacheConfig, DEFAULT_CACHE_CONFIG } from '@x12i/xronox-store';

const effective = resolveCacheConfig(collectionDef.cache);
// { maxSize: 10000, ttlMs: 0 } when cache is undefined

Primary-key path: write-through and getByKey

For insert, update, and patchByKey:

1. The final document (including primary key and createdAt / updatedAt as today) is computed.
2. The in-memory cache for that primary key is updated before the async persistence call (ensureInsert / ensureUpdate).
3. On persistence failure, existing errorHandling applies (throw / queue / silent).

getByKey(key) returns a clone of the cached document when TTL allows; otherwise it loads from the database and repopulates the cache. Tombstone/visibility rules (below) are applied after resolution: a hidden document behaves like missing and returns null. Rows loaded from the database that are hidden are not cached; any stale cache entry for that key is removed when such a row is observed.

Query path: readMany

• Database query uses the same metadata/mongo routing as before.
• visibility (if configured): unless includeHidden: true, the store ANDs a generic “visible only” predicate onto your filter (see below).
• Default (no mergeCache): results come only from the database. Unpersisted or DB-invisible rows that exist only in the process cache are not included—documented limitation unless you opt into merge.
• mergeCache: true: the store unions database rows with matching in-memory cache entries, deduped by primary key. On conflict, the cached document wins (full replacement for that key). Cached entries respect TTL the same way as getByKey.
• limit / sort: apply only to the database leg. After a merge, the result length and order are not re-trimmed or re-sorted globally—callers that need a strict cap should slice or sort in application code.

Cache-side matching uses a best-effort in-process matcher for Mongo-shaped filters ($and, $or, $eq, $in, …). Unsupported shapes return no match for a cached row so cache-only rows are not over-included. For complex filters, prefer relying on the database leg or keep filters composable.

Ordered scan and count (DB layer)

For stable list + cursor pagination over Mongo metadata, collections expose:

• scanOrdered(filter, options) — deterministic sort on (orderKey, _id), bounded pages (limit), and seek continuation via options.after (the previous page’s cursor). The store reads limit + 1 rows to set hasMore. Returned rows are shaped like readMany (e.g. _id stripped); the continuation cursor is derived from the raw DB row before shaping.
• countMatching(filter, options?) — counts documents with the same visibility composition as readMany. This requires mongoDriverCount on XronoxStore because @x12i/xronox does not expose a generic count API.

Not supported on scanOrdered: mergeCache. Ordered scans reflect persisted DB state only (same caveat as readMany without merge for unpersisted rows). Under buffered write mode, scans can lag until flush.

Indexes: declare compound indexes on collections via CollectionDefinition.indexes, for example { tick: 1, _id: 1 } for ascending scans, or prefix namespace fields first: { namespace: 1, tick: 1, _id: 1 }.

Guarantees (quiescent data): paging with the returned cursor does not repeat rows and follows total order on (orderKey, _id).

Concurrent writes: rows inserted “ahead” of the cursor may appear on later pages; rows moved across the boundary by an orderKey update may be skipped or seen twice depending on timing—treat “modified stream” semantics as a product contract above this layer.

Buffered persistence mode (optional)

By default, writes are immediate (write-through): cache is updated and the store attempts persistence right away.

For high-frequency mutation workloads, the store also supports an opt-in buffered write-behind mode:

• Local visibility is immediate (L1 cache updated at write time)
• Durable persistence is delayed and flushed later by policy
• Cross-process readers that only read from Mongo may observe delayed state until flush
• Process crash before flush may lose unflushed writes (explicit tradeoff; opt-in only)

Enabling buffered mode

import { XronoxStore } from '@x12i/xronox-store';

const store = new XronoxStore({
  mongoUri: process.env.MONGO_URI,
  mongoDb: process.env.MONGO_DB,
  writeMode: 'buffered',
  bufferedWrite: {
    flushIntervalMs: 30_000,
    maxBufferedOps: 1000,
    maxBufferedRecords: 1000,
    maxRecordAgeMs: 60_000,
    flushOnShutdown: true,
    retryFlushFailures: true,
    maxFlushBatchSize: 100,
  },
  // Optional: see “Queryable fields contract” below.
  queryableFields: ['status', 'startTime', 'runContext.sessionId'],
  collections: [
    { name: 'records', primaryKey: 'recordId' },
  ],
});

Flush control and introspection

• Manual flush: await store.flushNow()
• Stats: store.getBufferedWriteStats()

Stats include pending op/record counts, oldest pending age, last flush time, flush-in-progress, and coalescing/drop counters.

Queryable fields contract (for cache-merged queries)

readMany(filter, { mergeCache:true }) can merge matching in-memory cache entries into DB results. When queryableFields is configured on the store, the cache leg is only used for filters expressed entirely over declared dot-path fields (plus a small set of safe operators like $and/$or/$eq/$in and basic comparisons).

If a filter is outside the declared contract, the store falls back to DB-only results for that query (it will not include cache-only rows) to avoid over-including results from cache when the predicate semantics are ambiguous.

Single-record lookup by Mongo _id (optional)

For single-record reads by Mongo _id, collections expose:

• await store.collection('records').getByMongoId(id)

This is intentionally separate from query/filter reads and does not change the readMany contract.

Tombstone / visibility (optional CollectionDefinition.visibility)

Optional, domain-agnostic soft-delete visibility:

visibility?: {
  field: string;
  mode?: 'hiddenIfNonNull' | 'hiddenIfTruthy';
};

• hiddenIfNonNull (default when mode is omitted): a document is visible iff the field is missing or null; any other value hides it from normal reads.
• hiddenIfTruthy: visible iff Boolean(doc[field]) is false when evaluated in process. The database predicate uses Mongo $expr with $toBool / $ifNull on the field path (dot paths supported in the usual Mongo sense).

Effects:

• getByKey: hidden ⇒ null.
• readMany: hidden rows excluded unless includeHidden: true.
• updateMany: by default the same visibility predicate is ANDed into the filter unless includeHidden: true (admin/repair).
• deleteMany: same default includeHidden: false; pass includeHidden: true to target tombstoned rows (for example retention jobs).

updateMany loads candidates via readMany with mergeCache defaulting to true so primary-key rows present only in the L1 cache still participate; set mergeCache: false to use only the database leg for matching.

purgeActivixRecordsFromEnv passes includeHidden: true on store bulk operations so hard-delete and soft-mark phases still match rows once a tombstone field is set.

Exported helpers (generic)

• resolveCacheConfig, DEFAULT_CACHE_CONFIG — cache default merge.
• isDocumentHidden(doc, visibility) — in-process visibility check.
• andWithVisibility(filter, visibility, includeHidden) — compose a Mongo filter with the visibility predicate (or return the filter unchanged).
• mongoVisibilityPredicate(field, mode) — the visibility clause alone.
• documentMatchesFilter(doc, filter) — used internally for mergeCache; safe to reuse for tests or app-side checks within the matcher’s limits.
• tryCloneForInsert(doc) — preflight validation: returns { ok:true, doc } or { ok:false, error:{ path, valueType, constructorName? } }.
• XronoxStoreCloneError — thrown when store-internal document cloning fails; includes error.meta (same shape as above) and preserves the original error as error.cause.

Usage

import { XronoxStore } from '@x12i/xronox-store';

const store = new XronoxStore({
  mongoUri: process.env.MONGO_URI,
  mongoDb: process.env.MONGO_DB,
  collections: [
    {
      name: 'orders',
      primaryKey: 'orderId',
      primaryKeyPrefix: 'ord-',
      visibility: { field: 'deletedAt' },
    },
  ],
});

await store.init();
const id = await store.collection('orders').insert({ status: 'pending' });
await store.collection('orders').update(id, { status: 'done' });

const rows = await store.collection('orders').readMany(
  { status: 'done' },
  { mergeCache: true, limit: 50, sort: { updatedAt: -1 } },
);

await store.close();

Notes:

• If mongoUri is set but has no /dbname path (for example mongodb://host:27017/), provide mongoDb. The store will use it as the default DB and will also inject it into the URI for nx-mongo compatibility.
• If your MongoDB user authenticates against the admin database, you may need to add ?authSource=admin to mongoUri.
• If you need the normalization in scripts, you can also call withMongoDbInUri(mongoUri, mongoDb) (exported from this package).

Bulk delete (deleteMany)

Uses the same Mongo routing as readMany. Optional limit bounds how many matching documents are removed; clearCache: 'all' clears the collection’s in-memory cache after a purge. Use includeHidden: true when the filter must match tombstoned documents.

Operator hooks (optional driver pass-through)

Some operator scripts need Mongo driver features that are intentionally outside the store’s normal read/write contract (for example bulkWrite for ETL throughput, or aggregate for reporting).

@x12i/xronox-store supports these as opt-in hooks on XronoxStoreOptions, exposed only via registered collections:

• mongoDriverBulkWrite → await store.collection(name).bulkWrite(operations, options?)
• mongoDriverAggregate → await store.collection(name).aggregate(pipeline, options?)

Notes:

• These hooks are not used by normal store operations (insert/update/readMany/etc.).
• They do not participate in the write queue or buffered write manager.
• bulkWrite clears the collection cache by default (clearCache: 'all') because arbitrary operations may invalidate unknown primary keys.
• aggregate is a read-only reporting path: the pipeline is owned by the caller (no visibility predicates are injected and output is not shaped like readMany).

Transactions / sessions (explicit non-goal)

xronox-store does not currently expose a way to run store writes under a shared Mongo ClientSession / transaction. Store writes, buffered flushes, and operator hooks should be treated as non-transactional relative to any external transaction boundaries.

Full collection reset (admin wipe)

To delete all documents in a collection (including tombstoned/hidden ones when visibility is configured), use:

await store.collection('yourCollection').deleteMany(
  {},
  { includeHidden: true, clearCache: 'all' },
);

Activix-style retention purge (env-driven)

For soft-delete-then-hard-delete or hard-only retention without talking to MongoDB directly from your app, use purgeActivixRecordsFromEnv. It reads mode and TTLs from environment variables (default prefix ACTIVIX_). Variable list and semantics are in .docs/xronox-store.spec.md. Bulk operations use includeHidden: true so they remain correct when the same collection defines visibility on the soft-delete field.

Write queue / errors

If an operation is queued after a persist or connection failure, cache behavior stays aligned with today: for example insert may return a primary key while persistence is still pending—callers should rely on documented errorHandling semantics.

Logging (environment only)

This package’s opt-in logging (xronox-store)

By default, @x12i/xronox-store emits errors only (quiet by default).

To enable richer local logs (debug/info/warn + error), set:

ENABLE_XRONOX_STORE_LOGXER=true
XRONOX_STORE_LOGS_LEVEL=debug

• ENABLE_XRONOX_STORE_LOGXER: only true (case-insensitive) enables rich logging.\n+- XRONOX_STORE_LOGS_LEVEL: debug|info|warn|error|off (only meaningful when enabled).\n+- Legacy fallback: XRONOX_STORE_LOG_LEVEL is read only when XRONOX_STORE_LOGS_LEVEL is unset.

Lines are prefixed with [xronox-store] so they are easy to filter. To inject your own sinks instead, pass logger on XronoxStore options. Helpers: getEffectiveLogLevel(), isRichLoggingEnabled(), resolveLogger().

@x12i/logxer levels (via @x12i/xronox)

The engine layer uses @x12i/logxer with prefix XRONOX. That is separate from the store’s own env contract above. If XRONOX_LOGS_LEVEL (canonical) and legacy XRONOX_LOG_LEVEL are both unset, the effective level is warn (warn + error only). Set XRONOX_LOGS_LEVEL=off (or none / silent) to silence those diagnostics.

Hosts: downstream and other packages’ logs

If you are wiring an application or upper-level package and need to see or silence logs from this stack and from other dependencies:

1. Per-library verbosity — Each library that uses @x12i/logxer documents a stable envPrefix (short token, not the scoped npm name). In .env or your process environment, set {PREFIX}_LOGS_LEVEL for each prefix you care about. Examples for this stack:

   • XRONOX_LOGS_LEVEL — @x12i/xronox engine (same rules as above).
   • Other npm deps will document their own SOME_LIB_LOGS_LEVEL (and legacy SOME_LIB_LOG_LEVEL only when _LOGS_LEVEL is unset). Raise to info, debug, or verbose to dig deeper; use off to mute that package only.

2. Where output goes (cross-cutting) — Console, file path, format, unified sink, and similar host knobs are not duplicated under every dependency prefix in a single shared contract: they follow @x12i/logxer’s env model for the logger instance your code path uses. For the xronox singleton that means variables named with the XRONOX_ prefix. Configure those once at the host for the transports you want.

3. This package’s extra switch — Store diagnostics are controlled by ENABLE_XRONOX_STORE_LOGXER + XRONOX_STORE_LOGS_LEVEL; that does not replace XRONOX_LOGS_LEVEL for xronox. Use both when you need engine noise and store-level diagnostics independently.

4. Finding prefixes downstream — Check each dependency’s README or exported types for “logging”, “logxer”, or _LOGS_LEVEL. If a library does not document a prefix, it may not emit through @x12i/logxer; rely on that package’s own docs.

Tests

Without Mongo: resolveCacheConfig, logging env / resolveLogger behavior, and a fake Xronox mergeCache smoke test run first.

With Mongo: copy .env.example to .env, set MONGO_URI and MONGO_DB, then:

npm test

Further documentation

See .docs/xronox-store.spec.md for additional narrative API detail (may not yet list every new option; this README is the current contract summary).