@aria-framework/backup

The storage-agnostic half of a backup system: the encrypted archive envelope, the filename contract, grandfather-father-son retention, the destinations that hold archives, the compatibility guards, and the boot-time restore mechanism.

const backup = require('@aria-framework/backup');

const core = backup.createCore({
  magic: 'S101ARC',          // 7 ascii bytes, in the archive header. REQUIRED — pick your own
  prefix: 'support101-',     // filename prefix
  ext: '.abak'               // extension, dot included
});

core.naming.filenameFor(new Date(), 3);   // support101-20260812-201500-k3.abak
const enc = await core.createEncryptStream(password);
const dests = core.buildDestinations(settings, secrets);
await core.applyRetention(dests[0], policy, justWrittenName);

An archive says which app wrote it, and that is checked

checkCompatibility() requires the app's own name and refuses an archive whose manifest names a different one. The magic already made a foreign archive fail to decrypt, so this is not a new guarantee — it is the same one, stated where an operator can read it: "this archive was written by acc101, not support101" rather than "bad magic".

It matters most on the upload path, which deliberately does not trust the client filename and so had nothing but the magic between a mis-picked file and one app's database being replaced by another's. The manifest is inside the authenticated envelope, so this is not the weaker check of the two.

The parameter is required rather than optional, because an optional safety check is one nobody passes.

Integrating this into an app

docs/INTEGRATION.md is the start-to-finish guide: the order to wire things in, what stays the app's, and the traps. Everything in it is a mistake that actually shipped in one of the two consuming apps — a magic named after this package, a credential store marked optional, a stylesheet mounted but never linked, an optional peer never declared.

This README documents each subsystem's API. That document covers the seams BETWEEN them, which is where the time has actually gone.

What is not here

The orchestrator — scheduling, run history, the password key ring. It needs ports into an app's settings and secret storage, and arrives in a later version.

Capture IS here as of 0.2.0, but behind a source adapter rather than as one shared function — see Sources below for why that seam is not speculative.

Format identity is configured, not assumed

Two things carry an app's brand: the 7-byte magic in the archive header, and the filename prefix and extension. Both were literals in the code this came from.

All three are required, and the magic has no default. It used to fall back to ARIABAK — this package's own name — so a consumer that forgot the parameter branded every archive it wrote with the framework's name rather than its own. That is not cosmetic: the magic is written into the file, there is no read-set, and it cannot be corrected afterwards. Two consumers that both forgot would each open the other's archives as its own, which is the precise confusion a per-consumer magic exists to prevent. Choose seven bytes that name YOUR app.

There is no read-set. An archive opens only under a reader configured with the same magic — a clean cut taken while both consumers were pre-release. Adding one later means making FORMAT_VERSION a supported set rather than a scalar, which crypto.js already documents as the prerequisite for any format change.

The filename half fails more quietly and is worth stating plainly: isBackupName() gates both retention and the restore list. A name it does not recognise is never pruned and never offered — which presents as nothing happening rather than as an error. That is why naming is injected into destinations and retention instead of imported by them.

Envelope

MAGIC(7) + version(1) + headerLen(4) + header(JSON, plaintext) + frame*

Each frame is nonce(12) | flag(1) | ctLen(4) | ciphertext | tag(16), AES-256-GCM, with the header bytes plus the frame index and final-flag as AAD. Framing keeps verification incremental and makes truncation and reordering detectable; the key comes from scrypt with the parameters recorded in the header, so decryption stays symmetric if they are ever tuned.

The header is readable without the password — deliberately. It carries the key generation and key id, so an operator can tell which password opens an archive before trying one.

Destinations

LocalDestination and SftpDestination share an interface: write, list, remove, test. The SFTP one pins the host key (TOFU on first connect), verifies delete as well as write in test() — a destination that accepts uploads but refuses deletes passes a write-only check and breaks retention months later — and reports failures with the cause rather than the symptom.

ssh2-sftp-client is an optional peer, resolved on first connect rather than at import, so an app that only writes to a local directory does not have to carry it or its native install scripts.

Testing

npm test covers the envelope (round trip, multi-frame, tampering, truncation, wrong password, cross-brand rejection), the filename contract (including two apps not seeing each other's archives), retention under two namings, and the local destination end to end.

compat.js and restore-apply.js are carried across unchanged and are not yet covered here — their behaviour is pinned by the consuming app's suites, which exercise a real staged restore including rollback. They should grow tests here when the orchestrator lands and they stop being pass-through copies.

Sources (0.2.0)

The capture step, behind an adapter — because the two apps this was built for genuinely differ, and the difference was measured rather than assumed:

const source = backup.createSqliteSource({
  driver: require('better-sqlite3'),        // the CONSUMER's driver
  databases: [
    { name: 'main.db', path: config.dbPath, primary: true },
    { name: 'credentials.db', path: config.credentialsDbPath, optional: true }
  ],
  key: null,                                 // a SQLCipher hex key selects VACUUM INTO
  pragmas: ['kdf_iter = 256000']
});

const { stream, manifest, cleanup } = await core.createArchive({
  source,
  app: 'support101', appVersion, createdAt, password,
  keys:   [{ name: 'fields.json', value: { key: fieldsKeyHex } }],
  extras: [{ to: 'uploads', from: uploadsDir, key: 'uploads' }],
  installState: { credentialShape }
});

| | | |---|---| | plain better-sqlite3 | db.backup(dest) — the online-backup API | | SQLCipher | db.backup() throws backup is not supported with incompatible source and target databases. VACUUM INTO is the replacement, and keeps the snapshot encrypted. |

better-sqlite3 is an optional peer, never a dependency. In a SQLCipher app it is an alias for better-sqlite3-multiple-ciphers; if this package declared it, that app would get the plain driver and could not open its own database.

keys is what makes an archive portable — it can be restored on a machine that has never seen the install. One consumer carries one key, another carries two. installState is whatever a restore must match on the way back in; carrying it here is the difference between a guard and a step someone has to remember.

Ordering inside stage() is load-bearing: databases, then extras (so a file referenced by a snapshotted row already exists), then the version token — which opens the snapshot and leaves -wal/-shm beside it — then the source's cleanup(), and only then the manifest walk. Read the version after the walk and those two files ride along unlisted and unchecksummed.

PostgreSQL (0.24.0 source, 0.25.0 restore)

An app picks ONE engine; nothing runs both. The Postgres source produces a logical archive into the same envelope, retention, destinations and restore-arm flow the SQLite source uses.

const source = backup.createPostgresSource({
  connectionString: config.databaseUrl,
  databases: [{ name: 'app', primary: true }],
  binDir: null                                  // where pg_dump lives, if not on PATH
});

const restore = backup.createPostgresRestore({ connectionString: maintenanceUrl });

It stages <db>.dump (pg_dump -Fc), globals.sql, and lineage.json. Each is there for a measured reason:

| file | why | |---|---| | <db>.dump | custom format: compressed, and pg_restore -t/-L gives selective recovery out of the same file a full restore reads | | globals.sql | a pg_dump contains zero CREATE ROLE — verified on 17 and again on 18. Without it a bare-metal restore produces grants pointing at users that do not exist | | lineage.json | versionToken() reads STAGED files, and a .dump cannot be queried without restoring it first — so the migration lineage is captured while a connection still exists |

pg_dump is an external binary, not a library dependency. preflight() fails at configure time if it is missing, and refuses a client OLDER than the server — a 17.11 client against an 18.6 server aborts with a version mismatch, which would have turned every scheduled backup into a silent gap in the archive.

Restore, and the pre-restore copy

DROP DATABASE has no undo, and the SQLite path never destroys what it replaces — it renames the live file to .pre-restore-<stamp>. moveAside() is the analogue, using ALTER DATABASE … RENAME. Pass it to applyPendingRestore as databaseRestore and it slots into the existing flow: everything is set aside first, then applied, and a failure rolls both files and databases back.

Three things about that rename were measured against a live 18.6 and none are guessable:

• RENAME has no WITH (FORCE) and dies on one stray session — while DROP … WITH (FORCE) succeeds against that same session. The safe step is more fragile than the destructive one, so connections are locked out, terminated, and only then renamed.
• An over-long identifier is silently truncated at 63 bytes (a NOTICE, not an error), so two asides can collide and roll the wrong database back. The length is checked here, and every name is computed up front rather than lazily.
• ALLOW_CONNECTIONS must be restored on the aside, or the rollback copy exists and cannot be connected to.

moveAside() throws rather than warns — with no rollback copy a failed restore has nothing to go back to — and unwinds its own partial work, so aborting cannot itself destroy an already-renamed database. The aside is kept, and reported with its size and the exact DROP command, because a renamed database is invisible in a directory listing.

Replication, WAL archiving and PITR are NOT in this package

See docs/POSTGRES-OPERATIONS.md. The boundary is not squeamishness: PITR restores an entire cluster and cannot restore one database, so a package claiming to drive it would be claiming something the tool does not do. That document covers what to configure, how to verify it, and the failure mode that actually takes servers down — a broken archive_command does not stop the server, it fills the disk.

Key ring (0.3.0)

Which password encrypted which archive, behind ports so it needs neither a database nor a keystore to run:

const keyring = backup.createKeyring({
  settings: { all: () => …, set: (patch) => … },   // non-secret LABELS
  secrets:  { load: () => …, save: (next) => … },  // the passwords
  crypto: core
});

await keyring.rotate(newPassword);        // verifies a round trip BEFORE activating
await keyring.listGenerations();          // metadata only — no reveal permission needed
await keyring.revealPassword(gen);        // caller must check the permission and audit it

The split between the two stores is the design. Generation number, id and rotation date are labels: they live in the plain store so an admin page can show "this install is on generation 3" without decrypting anything. Only the passwords sit behind the reveal permission.

The invariant is "never lose a password", and it has been broken before. An earlier version gated retirement on Number.isInteger(prevGen); on the most common upgrade path — a password already set, no generation pointer yet — that was NaN, retirement was skipped, and the save overwrote the only copy of the password protecting every existing archive. So the two decisions that can lose one are pure functions, exported unbound and tested with plain objects:

• nextGeneration(pointer, history, passwords) — max+1 across all three sources, because they can disagree after a database is restored from an older copy beside a newer keystore
• planRetirement({...}) — never drops the outgoing password; files it under a generation only when that does not contradict what is already stored, and unlabelled when it does. A mislabelled password is a wrong answer; an unlabelled one is an honest "try these".

rotate() verifies the new password can encrypt AND decrypt before it becomes the archive key — without that, a password that cannot round-trip becomes live silently and the failure surfaces at restore, months later. It is single-flight: a double-submitted form must not mint two different passwords under one generation label.

Orchestrator (0.4.0)

Capture once, fan out, prune, record — plus the scheduler and the once-per-boot history reconciliation:

const orch = backup.createOrchestrator({
  core, logger,
  runRecorder: { start, finish, failStale, reconcileCaptureRun },
  cron: require('node-cron')          // optional peer
});

await orch.runExclusive(() => orch.performBackup({ source, keys, extras, settings, secrets, password }));
await orch.reconcileHistoryAtBoot(dataDir);
await orch.startScheduler({ enabled, expression, run });

The archive is written once to a temp file, then streamed to each destination — a single readable cannot fan out to N writers — and the file just written is protected from retention on every destination. Pruning the backup you have this second taken is the one bug a retention policy must never have.

A retention failure does not fail the backup. The archive was written; refusing to report that because a prune failed afterwards would be a worse outcome than a slightly fuller disk. The failure is returned in the result, not swallowed.

reconcileHistoryAtBoot runs once per boot, deliberately not on the reschedule path. An app that reschedules on every settings save would otherwise mark a live, in-flight backup as interrupted. It settles the restored capture run BEFORE failing stale runs, and clears the restore marker only after the history write lands — clearing it first loses the only note to a transient failure.

A latent crash this surfaced

Node opens a ReadStream lazily. A destination that rejects before consuming — refused connection, bad credentials, full disk — leaves the stream to open after the caller's finally has deleted the temp file, and it emits an 'error' with no listener. An unhandled 'error' event terminates the process: the app dies while handling a backup failure it had otherwise reported perfectly well.

Reproduced against the consuming app before fixing it here. The stream is now destroyed on the failure path, with its errors swallowed — the destination's error is the cause, and an ENOENT on a file about to be deleted is noise on top of it.

UI (0.5.0)

Partials, client behaviour and a specification — so an app gets the same page rather than reinventing one, and an app that cannot use the partials can still reproduce the shape.

// the app's own views dir FIRST; both Express and EJS accept an array
app.set('views', [path.join(__dirname, 'views'), backup.viewsDir]);
app.use('/js/backup', express.static(backup.assetsDir));

<%- include('backup/key-generations-modal', {
  keyGenerations, canReveal: can('backup.reveal_password'),
  revealPath: '/admin/backup/password/reveal', formatDateTime
}) %>

Body partials only, never a whole page. The two apps this was built for mount at different paths, use different chrome and have different permission names. All three are injected; the package assumes none of them.

docs/UI-SPEC.md is the durable half. It states the section order, every state, and the rules that are not stylistic — pending restore states never behind a click, reveals cleared when their container closes so re-reading is audited again, a schedule switch that hides the builder but not the retention that manual runs still obey, a copy control that falls back to selecting text where navigator.clipboard does not exist.

assets/backup-ui.js and assets/backup-ui.css are plain files — no bundler — and are safe to include on pages containing none of these controls.