@loynazkovacs/theitemapp-backend-sdk

Server-side SDK for TheItemApp app backends — the counterpart to the frontend @loynazkovacs/theitemapp-platform-sdk.

Every app with its own backend used to hand-roll the same three things and let them drift:

1. a core API client wrapping /api/dynamic CRUD,
2. a registration / heartbeat / deregister loop in index.ts,
3. an auth middleware proxying /api/auth/me.

This package is the single, canonical, typed implementation of all three.

Scope. This SDK is for apps that run their own backend. Apps with no backend ship via the seed-server base image and do not need it.

• Zero runtime dependencies. Node ≥ 18 (uses the global fetch/FormData/crypto).
• Everything opt-in & back-compat. New reliability/safety layers default off.

Table of contents

• Install
• Quick start
• Pillar 1 — Registration lifecycle
• Pillar 2 — CoreApiClient
  • Reads · Writes · Upserts · Files · Decrypt
  • Acting as an end user (asUser)
  • Reliability: automatic retry
  • Safety: x-ref validation
  • Error handling
  • Full config reference
• Pillar 3 — Auth
• Adoption recipe (existing backends)
• Versioning & publishing
• API index

Install

npm install @loynazkovacs/theitemapp-backend-sdk

Node ≥ 18, ESM ("type": "module").

Quick start

A typical Fastify app backend wires all three pillars at startup:

import Fastify from 'fastify';
import {
  CoreApiClient,
  startAppRegistration,
  createFastifyAuthPreHandler,
} from '@loynazkovacs/theitemapp-backend-sdk';
import manifest from './dbseed/manifest.json' assert { type: 'json' };

const app = Fastify();
const coreUrl = process.env.CORE_API_URL ?? 'http://backend:3001';

// 1) Core API client — key gets filled in by registration below.
const coreApi = new CoreApiClient({
  baseUrl: coreUrl,
  apiKey: null,
  retry: { maxAttempts: 5 },   // optional: survive 429 bursts
});

// 2) Registration: retry-until-up, capture the rotated key, heartbeat, deregister.
let registration;
app.post('/app/re-register', async () => { registration?.reRegister(); return { ok: true }; });

await app.listen({ host: '0.0.0.0', port: 3000 });

registration = startAppRegistration({
  coreUrl,
  manifest,
  selfUrl: process.env.SELF_URL ?? 'http://myapp:80',
  registrationKey: process.env.APP_REGISTRATION_KEY,
  installSignalHandlers: false,                 // we run our own shutdown below
  onApiKey: (key) => coreApi.updateApiKey(key), // core rotates the key per register
});

process.on('SIGTERM', async () => {
  registration?.stop();
  await registration?.deregister();
  await app.close();
  process.exit(0);
});

// 3) Auth on protected routes — verifies the caller against core.
app.get('/api/things', { preHandler: createFastifyAuthPreHandler(coreUrl) }, async () => {
  return coreApi.list('things');
});

Pillar 1 — Registration lifecycle

startAppRegistration(opts) owns the full handshake every backend needs: register-with-retry (core can take minutes to boot), capture the auto-provisioned & rotated functional API key, a 5-minute heartbeat, re-register on core's reboot, and deregister on shutdown.

const registration = startAppRegistration({
  coreUrl,                       // core base URL
  manifest,                      // your dbseed manifest (must contain appKey)
  selfUrl,                       // how core reaches this container, e.g. http://myapp:80
  registrationKey,               // optional shared secret (X-Registration-Key)
  onApiKey: (key) => coreApi.updateApiKey(key),
  heartbeatMs: 5 * 60_000,       // optional (default 5 min; 0 disables)
  maxRetries: 30,                // optional (default 30)
  retryIntervalMs: 5_000,        // optional (default 5s)
  installSignalHandlers: true,   // default true; set false to run your own shutdown
  logger: app.log,               // optional { info, warn, error }
});

| Returns | | |---|---| | register() | run one registration attempt now → Promise<boolean> | | reRegister() | fire-and-forget; wire into your POST /app/re-register route | | deregister() | best-effort DELETE from core's catalog | | stop() | stop the heartbeat timer |

Custom shutdown. If your app must clean up on exit (stop a collector, close a DB, end sessions), pass installSignalHandlers: false and call registration.stop() + await registration.deregister() inside your own SIGTERM/SIGINT handler.

Provisioned key. Core returns a fresh functional apiKey on each register (rotated, prefix-stable) only if the app seeds a functional user. onApiKey fires whenever a key is issued — use it to update your CoreApiClient. Apps that seed no functional user simply never receive one (that's fine).

Pillar 2 — CoreApiClient

A typed wrapper over /api/dynamic/<collection> plus the file and decrypt endpoints. Construct once; update the key when registration provides it.

const coreApi = new CoreApiClient({ baseUrl, apiKey: null });
coreApi.updateApiKey(key);   // after registration
coreApi.isReady();           // true once a non-empty key is set — gate writes on this

Reads

await coreApi.list('apps');                                  // _l=500 by default
await coreApi.list('apps', { _l: '50', _s: '-createdAt' });  // override params
await coreApi.get('apps', id);                               // null on 404, throws otherwise
await coreApi.get('apps', id, { populate: false });          // keep x-refs as id strings (?populate=0)
await coreApi.findBy('apps', 'key', 'system');               // first match by indexed field, or null
await coreApi.count('apps', { active: 'true' });             // GET .../count → number

populate. Core returns single x-ref fields as full objects by default. If your code compares x-ref ids as strings (e.g. diffing rows), read with { populate: false } so they stay as 24-hex strings.

Writes

await coreApi.create('apps', { name: 'X' });
await coreApi.update('apps', id, { name: 'Y' });             // PUT (partial merge)
await coreApi.delete('apps', id);                            // soft-delete → true (also on 404)
await coreApi.bulkCreate('events', docs);                    // ≤500 → { ok, created, failed, results }
await coreApi.hardDeleteByFilter('events', { stale: true }); // api-key only → { ok, deletedCount }

Per-call WriteOptions on create/update/upsertOn/bulkCreate:

await coreApi.create('audit', row, { skipWebhooks: false }); // let this write fan out
await coreApi.update('x', id, body, { headers: { 'x-foo': '1' } });

Upserts

// single field → returns the doc (or null); falls back to find+update/create on old cores
await coreApi.upsert('apps', 'key', 'system', body);

// skip the write if nothing changed (no spurious change-stream events)
await coreApi.upsert('host', 'name', 'localhost', body, { skipIfUnchanged: true });

// composite key → full result so you can branch on created vs updated
const { created, doc } = await coreApi.upsertOn('links', ['srcId', 'dstId'], body);

// bulk upsert ≤500 by composite key (usually ['_id'])
await coreApi.bulkUpsert('rows', ['_id'], docs);             // → { upsertedCount, modifiedCount, errors }

Files

const { _id } = await coreApi.uploadFile(bytes, {            // Uint8Array (a Buffer works)
  filename: 'cover.png', mimeType: 'image/png', kind: 'image',
});
const file = await coreApi.downloadFile(_id);               // { data, contentType, filename } | null

Decrypt

await coreApi.decrypt('connections', id);                    // null on failure (logs a warn)
await coreApi.decrypt('connections', id, { throwOnError: true });

Acting as an end user (asUser)

By default the client authenticates with the functional x-api-key (the app/system actor). To make core attribute a call to the human user — and apply their RBAC — derive a scoped client that forwards their session:

// attribute the write to the user (drops the functional key):
await coreApi.asUser({ authorization: req.headers.authorization }).create('notes', body);

// forward the cookie session, keep the functional key too (app identity + user):
await coreApi.asUser({ cookie: req.headers.cookie }, { keepApiKey: true })
  .uploadFile(bytes, { filename, mimeType });

// works for every method — reads, writes, upload, download, decrypt:
await coreApi.asUser({ jwt }).list('my_private_things');

The scoped client inherits the parent's retry / validateXrefs / logger config.

Reliability: automatic retry

Off by default. When enabled, requests retry on configured statuses (default [429]), honouring Retry-After (header or a "retry in N seconds" body), with jitter and a per-wait cap. Essential for fire-and-forget writes (transcript/audit/usage) that core may rate-limit.

new CoreApiClient({
  baseUrl, apiKey,
  retry: {
    maxAttempts: 5,        // default 5 when `retry` is set
    retryOn: [429],        // default [429]; add 503 etc. if you want
    honorRetryAfter: true, // default true
    maxDelayMs: 20_000,    // default 20s cap per wait
  },
});

Safety: x-ref validation

Off by default. When enabled, the client loads each collection's items schema (cached, single-flight), and before every write checks that x-ref fields hold valid 24-hex ObjectIds — catching a class of bugs where a name or slug is passed where an id belongs.

new CoreApiClient({
  baseUrl, apiKey,
  validateXrefs: true,
  onInvalidXref: 'warn',   // 'warn' (default, logs) or 'throw' (refuses the write)
});

The building blocks are also exported for custom use: extractXrefFields(schema), checkXrefFields(body, map).

Error handling

Reads return null for genuine "not found" (get/findBy on 404). Everything else throws CoreApiError on a non-2xx response:

import { CoreApiError } from '@loynazkovacs/theitemapp-backend-sdk';

try {
  await coreApi.update('apps', id, body);
} catch (err) {
  if (err instanceof CoreApiError) {
    err.status;     // 409
    err.method;     // 'UPDATE'
    err.collection; // 'apps'
    err.target;     // 'apps/<id>'
    err.body;       // raw response body
  }
}

Prefer a null-returns contract? Wrap the call: const doc = await coreApi.create(c, b).catch(() => null);

Full config reference

new CoreApiClient({
  baseUrl: 'http://backend:3001', // required
  apiKey: null,                   // functional key (set later via updateApiKey)
  skipWebhooks: true,             // default true (x-theitemapp-skip-webhooks on writes)
  retry: { /* RetryConfig */ },   // default off
  validateXrefs: false,           // default off
  onInvalidXref: 'warn',          // 'warn' | 'throw'
  logger: console,                // { info?, warn?, error? } — default console
});

Pillar 3 — Auth

App backends don't verify JWTs themselves by default — they forward the caller's Authorization/Cookie to core's GET /api/auth/me, the single source of truth for identity and group membership.

import {
  verifyUser, userInAnyGroup, normalizeId,
  createExpressAuthMiddleware, createFastifyAuthPreHandler,
  verifyJwtLocal,
} from '@loynazkovacs/theitemapp-backend-sdk';

Verify against core (default)

const user = await verifyUser(coreUrl, {
  cookie: req.headers.cookie,
  authorization: req.headers.authorization,
});
// → { _id, username, email?, groupIds, raw } | null
//   null = no credentials or core rejected them; throws only if core is unreachable.

Core's /api/auth/me returns the user id as id (the JWT sub), not _id — verifyUser handles this; user._id is always populated.

Group gating:

if (!userInAnyGroup(user, adminGroupIds)) return reply.code(403).send();

Drop-in middleware

// Fastify — assigns request.user
app.addHook('preHandler', createFastifyAuthPreHandler(coreUrl));

// Express — assigns res.locals.user
app.use(createExpressAuthMiddleware(coreUrl));

Both respond 401 (no/invalid creds) or 503 (core unreachable) themselves.

Offline verification

For backends that hold core's JWT_SECRET and want to skip the network round trip (e.g. high-frequency idempotency checks):

const payload = verifyJwtLocal(token, process.env.JWT_SECRET!);
// → decoded HS256 payload | null (null = malformed / bad signature / expired / not HS256)

Adoption recipe (existing backends)

Migrate a hand-rolled backend onto the SDK with zero behaviour change by making each existing module a thin shell over the SDK:

1. Add the dependency (^0.4.0). If the Dockerfile uses npm ci, regenerate package-lock.json.
2. Registration → replace the register/retry/heartbeat/deregister block with startAppRegistration (installSignalHandlers: false if you have a custom shutdown).
3. coreApiClient.ts → keep your exported class/signatures, but back it with an internal CoreApiClient and delegate. Preserve your conventions via config: populate=false reads, retry for a 429 layer, validateXrefs for id checks, asUser for user-attributed writes. Re-export your DynRow/error types so call-sites don't change.
4. Auth → swap your /api/auth/me proxy internals for verifyUser (or verifyJwtLocal if you verify offline). Keep your middleware's exported shape.
5. Typecheck, deploy, verify (registration log, auth 401/200, a real read/write), then commit + push.

The system backend is the reference implementation; see the backend-sdk-adoption agent skill for the full per-app checklist.

Versioning & publishing

Published from main by the Build Images workflow (publish-backend-sdk job) on any change under libs/backend-sdk/**. The job is idempotent (skips if the version already exists) and fails loudly if a publish genuinely fails.

cd libs/backend-sdk
npm run build      # tsc → dist/
npm publish        # uses .npmrc registry + publishConfig (public)

See CHANGELOG.md for version history. Caret ranges on 0.x do not auto-bump across minors (^0.3.0 ≠ 0.4.0), so pinned apps upgrade deliberately.

API index

Core client: CoreApiClient · CoreApiError · list get findBy count create bulkCreate update updateAsUser delete hardDeleteByFilter upsert upsertOn bulkUpsert uploadFile downloadFile decrypt · asUser · updateApiKey getApiKey isReady loadXrefFields

Utilities: extractXrefFields · checkXrefFields · deepEqual

Registration: startAppRegistration (RegistrationHandle, AppManifest)

Auth: verifyUser · verifyJwtLocal · userInAnyGroup · normalizeId · createExpressAuthMiddleware · createFastifyAuthPreHandler

Types: CoreApiConfig RetryConfig SdkLogger WriteOptions GetOptions UserCredentials AuthUser JwtPayload BulkCreateResult UpsertResult BulkUpsertResult UploadFileOptions DownloadedFile XrefFieldMap

Not covered (by design)

• aggregate() — core's pivot aggregation rides the dynamic list route (_agg), not a stable HTTP contract yet. Use list() with params.
• Seed/function HTTP serving — that's the seed-server image's job (apps without a backend).