@exellix/exellix-db

Mongo-backed data tier for the Exellix job queue. This is the only package that imports mongodb. It exposes a small JobRunStore interface plus a native-driver implementation whose claim is a single atomic findOneAndUpdate.

A JobRun is one graph run on one input — the durable queue work-unit. It is distinct from a graph task node and from an ai-task (@exellix/ai-tasks). See temp/jobs/ for the full design.

Install

npm install @exellix/exellix-db

Usage

import { createJobRunStore } from '@exellix/exellix-db';

const { store, close } = await createJobRunStore({
  mongoUri: process.env.MONGO_URI,   // or set MONGO_URI in the environment
  // runtimeDb: 'exellix-jobs',       // default
  // configDb:  'exellix',            // default
  // ensureIndexes: true,             // default
});

const run = await store.claim('worker-1');   // atomic claim, or null
await store.close();                          // via close()

For tests, use the in-memory implementation:

import { createMemoryJobRunStore } from '@exellix/exellix-db';
const store = createMemoryJobRunStore();

JobRunStore interface

interface JobRunStore {
  insertMany(runs: JobRun[]): Promise<void>;
  update(id: string, patch: Partial<JobRun>): Promise<void>;
  updateMany(filter: JobRunFilter, patch: Partial<JobRun>): Promise<{ modifiedCount: number }>;

  claim(workerId: string, opts?: ClaimOpts): Promise<JobRun | null>;   // atomic

  get(id: string): Promise<JobRun | null>;
  find(filter: JobRunFilter): Promise<JobRun[]>;
  count(filter: JobRunFilter): Promise<number>;
  hasItem(jobDefId: string, itemId: string): Promise<boolean>;          // dedup

  decrementPendingDeps(id: string): Promise<JobRun | null>;             // atomic
  close(): Promise<void>;
}

Collections

| Collection | DB | Purpose | |------------|-----|---------| | job_runs | runtime (exellix-jobs) | the work-unit; source of truth | | job_defs | config (exellix) | one doc per JobDef | | graphs | config (exellix) | graph authoring JSON / per-graph defaults | | job_run_events (optional) | runtime | thin append-only audit log |

The (jobDefId, itemId) index (by_job_item) is intentionally non-unique — one item produces one job run per graph. Dedup is enforced by hasItem at enqueue time, not by a unique constraint.

Why a separate package

@x12i/xronox-store has no atomic conditional update (findOneAndUpdate), which forced the old Execution Matrix into in-process withClaimLock. A durable cross-process queue needs structural atomicity, so the data tier uses the native mongodb driver and is kept out of the queue logic.

Scripts

npm run build      # tsc → dist/
npm test           # unit + live (live runs only when MONGO_URI is set)
npm run test:live  # live Mongo integration tests only

Live tests read MONGO_URI from the environment, falling back to ../graph-engine/.env. Each run uses an isolated exellix-jobs-live-* database that is dropped on teardown.

License

exellix-license