@aria-framework/backup
v0.33.1
Published
Aria App Framework — backup core. Streaming authenticated archive envelope (scrypt + AES-256-GCM, framed), filename contract, grandfather-father-son retention, local + SFTP destinations with host-key pinning, compatibility guards, and the boot-time restor
Readme
@aria-framework/backup
The storage-agnostic half of a backup system: the encrypted archive envelope, the filename contract, grandfather-father-son retention, the destinations that hold archives, the compatibility guards, and the boot-time restore mechanism.
const backup = require('@aria-framework/backup');
const core = backup.createCore({
magic: 'S101ARC', // 7 ascii bytes, in the archive header. REQUIRED — pick your own
prefix: 'support101-', // filename prefix
ext: '.abak' // extension, dot included
});
core.naming.filenameFor(new Date(), 3); // support101-20260812-201500Z-k3.abak (UTC)
const enc = await core.createEncryptStream(password);
const dests = core.buildDestinations(settings, secrets);
await core.applyRetention(dests[0], policy, justWrittenName);An archive says which app wrote it, and that is checked
checkCompatibility() requires the app's own name and refuses an archive whose manifest names
a different one. The magic already made a foreign archive fail to decrypt, so this is not a new
guarantee — it is the same one, stated where an operator can read it: "this archive was written
by acc101, not support101" rather than "bad magic".
It matters most on the upload path, which deliberately does not trust the client filename and so had nothing but the magic between a mis-picked file and one app's database being replaced by another's. The manifest is inside the authenticated envelope, so this is not the weaker check of the two.
The parameter is required rather than optional, because an optional safety check is one nobody passes.
Integrating this into an app
docs/INTEGRATION.md is the start-to-finish guide: the order to wire things in, what stays
the app's, and the traps. Everything in it is a mistake that actually shipped in one of the two
consuming apps — a magic named after this package, a credential store marked optional, a
stylesheet mounted but never linked, an optional peer never declared.
This README documents each subsystem's API. That document covers the seams BETWEEN them, which is where the time has actually gone.
What is not here
The orchestrator — scheduling, run history, the password key ring. It needs ports into an app's settings and secret storage, and arrives in a later version.
Capture IS here as of 0.2.0, but behind a source adapter rather than as one shared function — see Sources below for why that seam is not speculative.
Format identity is configured, not assumed
Two things carry an app's brand: the 7-byte magic in the archive header, and the filename prefix and extension. Both were literals in the code this came from.
All three are required, and the magic has no default. It used to fall back to ARIABAK —
this package's own name — so a consumer that forgot the parameter branded every archive it wrote
with the framework's name rather than its own. That is not cosmetic: the magic is written into
the file, there is no read-set, and it cannot be corrected afterwards. Two consumers that both
forgot would each open the other's archives as its own, which is the precise confusion a
per-consumer magic exists to prevent. Choose seven bytes that name YOUR app.
There is no read-set. An archive opens only under a reader configured with the same
magic — a clean cut taken while both consumers were pre-release. Adding one later means
making FORMAT_VERSION a supported set rather than a scalar, which crypto.js already
documents as the prerequisite for any format change.
The filename half fails more quietly and is worth stating plainly: isBackupName() gates
both retention and the restore list. A name it does not recognise is never pruned and
never offered — which presents as nothing happening rather than as an error. That is why
naming is injected into destinations and retention instead of imported by them.
Time is UTC, and the host's timezone is not an input (0.32.0)
A host may be in any timezone and may move. None of that reaches a backup:
- Names are stamped in UTC and say so:
<prefix>YYYYMMDD-HHMMSSZ[-kN][-m]<ext>. - Retention reads its periods off the digits in the name: a day is a UTC day, a week starts
at UTC midnight on
weekStartsOn. The plan for a listing is the same on every host. - The schedule is handed to the scheduler as
cron.schedule(expr, fn, { timezone: 'UTC' }).0 3 * * *is 03:00 UTC. The schedule builder's preview says "UTC". There is no setting for another zone.
naming.parseDate(name) returns a Date whose UTC fields are the name's digits — read it with
getUTC*. naming.isUtcName(name) tells a 0.32.0 name from an older one.
Upgrading an installation that already has backups
Archives written before 0.32.0 have no Z; their digits are the local time of the host that
wrote them. They keep parsing, and:
No old archive changes period. Old names are bucketed by their digits, as they always were. They are not converted with the process's current zone: that would be host-dependent again, and on a host west of UTC it would move an evening archive from the last day of a month into the next month, where it stops being its month's archive and is pruned early.
No overwrite. East of UTC, the first UTC archive can carry the same digits as the last local one (03:00 local, then 03:00Z two hours later). The
Zmakes them two files, and retention treats theZone as the newer.An old archive can be pruned sooner than 0.31.x would have pruned it — east of UTC only. The rule: an old-named archive gives up its day (and its week or month, if it held them) to a later
Zarchive whose UTC date equals the old name's digit date. Under 0.31.x the later archive would have been named for the next local day and both would have been kept. TheZarchive is the newer backup; what is lost is one step in the daily chain. Two ways to meet it:- Schedule left unchanged. The same-digits case above: the old twin goes at the run that
wrote the
Zone, as many hours old as the host was ahead of UTC, and one step in the chain is24h + offset. Only if the upgrade is deployed between the two runs. - Expression moved to keep the old instant (a host at +10 changing
0 3 * * *to0 17 * * *): the last old archive,…07-030000, and the first new one,…07-170000Z, share 7 Oct. The old one goes, 24 hours old, and one step in the chain is 48 hours.
West of UTC neither can happen. To avoid it, copy the last old archive aside before the first run after the upgrade.
- Schedule left unchanged. The same-digits case above: the old twin goes at the run that
wrote the
Ordering an old name against a new one compares digits, so it can be off by the old host's UTC offset. West of UTC a new archive always sorts newer. East of UTC, a new archive taken less than the offset after an old one sorts older than it: until the UTC digits pass the old ones, a day's archive may be an old one up to
offsethours older than the newest. The archive just written is never deleted by its own run.parseDateon an old name is not the instant it was taken. It returns the digits as UTC, which is off by the old host's offset. Do not render it as a time; the archive header'screatedAtis the record. And if you call the rawretention.planwith aparseDateof your own, it must follow the same contract — UTC fields are the digits — because the buckets are read with the UTC getters.The schedule moves. A stored
0 3 * * *used to fire at 03:00 host time and now fires at 03:00 UTC. Nothing is rewritten; change the expression if the old instant mattered.Going back to 0.31.x leaves
Zarchives on the destination that the old version does not recognise: it neither prunes nor lists them.
If your app has its own test of the filename contract, it will need the Z, Date.UTC(...) for
its fixtures and getUTC* on whatever parseDate returns.
Envelope
MAGIC(7) + version(1) + headerLen(4) + header(JSON, plaintext) + frame*
Each frame is nonce(12) | flag(1) | ctLen(4) | ciphertext | tag(16), AES-256-GCM, with the
header bytes plus the frame index and final-flag as AAD. Framing keeps verification
incremental and makes truncation and reordering detectable; the key comes from scrypt with
the parameters recorded in the header, so decryption stays symmetric if they are ever tuned.
The header is readable without the password — deliberately. It carries the key generation and key id, so an operator can tell which password opens an archive before trying one.
Destinations
LocalDestination and SftpDestination share an interface: write, list, remove,
test. The SFTP one pins the host key (TOFU on first connect), verifies delete as well
as write in test() — a destination that accepts uploads but refuses deletes passes a
write-only check and breaks retention months later — and reports failures with the cause
rather than the symptom.
ssh2-sftp-client is an optional peer, resolved on first connect rather than at import,
so an app that only writes to a local directory does not have to carry it or its native
install scripts.
Testing
npm test covers the envelope (round trip, multi-frame, tampering, truncation, wrong
password, cross-brand rejection), the filename contract (including two apps not seeing each
other's archives), retention under two namings, and the local destination end to end.
compat.js and restore-apply.js are carried across unchanged and are not yet covered
here — their behaviour is pinned by the consuming app's suites, which exercise a real
staged restore including rollback. They should grow tests here when the orchestrator lands
and they stop being pass-through copies.
Sources (0.2.0)
The capture step, behind an adapter — because the two apps this was built for genuinely differ, and the difference was measured rather than assumed:
const source = backup.createSqliteSource({
driver: require('better-sqlite3'), // the CONSUMER's driver
databases: [
{ name: 'main.db', path: config.dbPath, primary: true },
{ name: 'credentials.db', path: config.credentialsDbPath, optional: true }
],
key: null, // a SQLCipher hex key selects VACUUM INTO
pragmas: ['kdf_iter = 256000']
});
const { stream, manifest, cleanup } = await core.createArchive({
source,
app: 'support101', appVersion, createdAt, password,
keys: [{ name: 'fields.json', value: { key: fieldsKeyHex } }],
extras: [{ to: 'uploads', from: uploadsDir, key: 'uploads' }],
installState: { credentialShape }
});| | |
|---|---|
| plain better-sqlite3 | db.backup(dest) — the online-backup API |
| SQLCipher | db.backup() throws backup is not supported with incompatible source and target databases. VACUUM INTO is the replacement, and keeps the snapshot encrypted. |
better-sqlite3 is an optional peer, never a dependency. In a SQLCipher app it is an
alias for better-sqlite3-multiple-ciphers; if this package declared it, that app would get
the plain driver and could not open its own database.
keys is what makes an archive portable — it can be restored on a machine that has never
seen the install. One consumer carries one key, another carries two. installState is
whatever a restore must match on the way back in; carrying it here is the difference between
a guard and a step someone has to remember.
Ordering inside stage() is load-bearing: databases, then extras (so a file referenced by a
snapshotted row already exists), then the version token — which opens the snapshot and
leaves -wal/-shm beside it — then the source's cleanup(), and only then the manifest
walk. Read the version after the walk and those two files ride along unlisted and
unchecksummed.
PostgreSQL (0.24.0 source, 0.25.0 restore)
An app picks ONE engine; nothing runs both. The Postgres source produces a logical archive into the same envelope, retention, destinations and restore-arm flow the SQLite source uses.
const source = backup.createPostgresSource({
connectionString: config.databaseUrl,
databases: [{ name: 'app', primary: true }],
binDir: null // where pg_dump lives, if not on PATH
});
const restore = backup.createPostgresRestore({ connectionString: maintenanceUrl });It stages <db>.dump (pg_dump -Fc), globals.sql, and lineage.json. Each is there for a
measured reason:
| file | why |
|---|---|
| <db>.dump | custom format: compressed, and pg_restore -t/-L gives selective recovery out of the same file a full restore reads |
| globals.sql | a pg_dump contains zero CREATE ROLE — verified on 17 and again on 18. Without it a bare-metal restore produces grants pointing at users that do not exist |
| lineage.json | versionToken() reads STAGED files, and a .dump cannot be queried without restoring it first — so the migration lineage is captured while a connection still exists |
pg_dump is an external binary, not a library dependency. preflight() fails at configure
time if it is missing, and refuses a client OLDER than the server — a 17.11 client against an
18.6 server aborts with a version mismatch, which would have turned every scheduled backup into
a silent gap in the archive.
Restore, and the pre-restore copy
DROP DATABASE has no undo, and the SQLite path never destroys what it replaces — it renames the
live file to .pre-restore-<stamp>. moveAside() is the analogue, using ALTER DATABASE … RENAME.
Pass it to applyPendingRestore as databaseRestore and it slots into the existing flow:
everything is set aside first, then applied, and a failure rolls both files and databases back.
Three things about that rename were measured against a live 18.6 and none are guessable:
RENAMEhas noWITH (FORCE)and dies on one stray session — whileDROP … WITH (FORCE)succeeds against that same session. The safe step is more fragile than the destructive one, so connections are locked out, terminated, and only then renamed.- An over-long identifier is silently truncated at 63 bytes (a NOTICE, not an error), so two asides can collide and roll the wrong database back. The length is checked here, and every name is computed up front rather than lazily.
ALLOW_CONNECTIONSmust be restored on the aside, or the rollback copy exists and cannot be connected to.
moveAside() throws rather than warns — with no rollback copy a failed restore has nothing to go
back to — and unwinds its own partial work, so aborting cannot itself destroy an already-renamed
database. The aside is kept, and reported with its size and the exact DROP command, because
a renamed database is invisible in a directory listing.
Extensions, and who may restore (0.33.0)
A site has an administrator create vector and pg_trgm, so the superuser owns them, while
the restore runs as the database's owner role. Replaying the dump's own extension entries as
that owner fails: must be owner of extension … (DROP and COMMENT ON EXTENSION), or
permission denied to create extension "vector" in a database made from template0. Measured on
18.6 (AISearch's backup bench). --single-transaction rolled that back cleanly, so the restore
was safe but could not succeed on such a site.
So applyDatabase lists the dump (pg_restore -l) and decides, for each extension it names,
which of the routes below provides it. It then replays with -L. For an extension provided by
route 2 or 4 it takes out only that extension's EXTENSION and COMMENT - EXTENSION
entries. Any other error still fails the restore. A TOC line in any other shape is left in, so
it fails rather than being skipped on a guess. The first route that works is used, and all of
it is decided before anything is dropped:
- Not on this server (not in
pg_available_extensions): refused. No route can create it. - In template1, and the database can be created from template1: inherited. The database is created from template1. That needs the captured encoding, collation, ctype and locale provider to equal template1's; otherwise only template0 could create it, and template0 has no extensions.
- The restoring role is a superuser, or the extension is trusted for its default version
(
pg_available_extension_versions.trusted; nothing counts as trusted before PG13): its entries stay in the replay, which creates it as the restoring role.WITH SCHEMAis included, and the COMMENT succeeds because that role now owns the extension. This is exactly what 0.32.x did, so a restore that worked there still works. extensionsConnectionStringis set: created through that administrator connection.- Otherwise: refused, for example
extension vector is not in template1; create it there as an administrator, or pass extensionsConnectionString, orextension vector is in template1, but "app" (UTF8, C, …) cannot be created from template1. In the boot flow the aside is then rolled back.
const restore = backup.createPostgresRestore({
connectionString: ownerMaintenanceUrl,
extensionsConnectionString: adminUrl // optional: used ONLY for CREATE EXTENSION IF NOT EXISTS
});extensionsConnectionString is parsed when the restore is created. It is refused if it is not a
URL, or if its host or port differ from connectionString's. Its database part is ignored. Its
password is passed to psql through PGPASSWORD, not on the command line, and a failed command's
error no longer carries the command line (err.cmd / err.spawnargs).
Schema, on routes 2 and 4 only.
- An extension inherited from template1 is wherever template1 has it.
- One created through the administrator connection goes into the default schema (
public). - In both cases the dump's
WITH SCHEMAis not used. If the dump's objects expect the extension in another schema, the replay fails and rolls back. - Route 3 keeps the dump's own schema.
Before the DROP, the restore also checks that the restoring role may SET ROLE to the
database's original owner. That is pg_has_role(…, 'SET') on 16+ and 'MEMBER' before. A
superuser, or the owner itself, always may. CREATE DATABASE … OWNER needs it, and
inheriting the owner's privileges, which is all preflight() and moveAside() need, is not
enough.
The listing pass also reads the whole TOC first, so an archive whose TOC is unreadable is now refused before the database is dropped.
preflight(databases) checks rights before anything is touched. The restoring role must be
able to create databases (CREATEDB or superuser). It must also own each existing database to be
set aside, or inherit the owner's privileges, because RENAME needs both. applyPendingRestore
calls it, when the databaseRestore has one, after the staged tree is verified and before
beforeSwap, any file move or moveAside. A refusal names the missing right.
Also in 0.33.0, restorePostgres reports every pg_restore: error: line. The first one
used to be dropped, because the thrown message starts with <bin> failed: . AISearch saw 3 of 4
errors listed. By the same logic, a restore whose only error came first would have been
reported as a success, since pg_restore's closing "errors ignored" line is a warning.
The captured attributes now reach the recreate in the boot flow. applyPendingRestore hands
apply() the app's database list, which has no attributes. After the rename,
readAttributes(name) found nothing, so through 0.32.x a boot restore recreated the database
owned by the restoring role, with template1's encoding and collation. moveAside() now keeps
what it captured, by name, and apply() uses it. Owner, encoding, collation, ctype and, on 15+,
the locale provider and its locale (daticulocale on 15–16, datlocale on 17+) are carried
over.
Upgrading to 0.33.0
restorePostgresnow throws for a restore whose only error was on stderr's first line. 0.32.x let that through as a success.- A Postgres restore with extensions follows the routes above. A restore 0.32.x completed still completes: route 3 is what 0.32.x did. A restore that needs an untrusted extension the restoring role cannot create now stops before the DROP and names the fix. 0.32.x failed it inside the replay and rolled back.
- In the boot flow the database comes back with its original owner and collation, not the
restoring role and template1's. The restoring role must therefore be able to
SET ROLEto that owner (PG16+); the owner itself always can. If it cannot, the restore is refused before the DROP and names the GRANT. - An archive whose TOC cannot be read is refused before the DROP.
preflight()runs before anything moves, when thedatabaseRestorehas it.filterExtensionEntriesandplanExtensionsare exported from the package index.
Restore hardening (0.33.1)
Six findings from AISearch switching its boot restore to createPostgresRestore (its
docs/UPSTREAM-BACKUP.md items 7–12), plus a review round. Each was measured there or here on 18.6.
apply()takes the archive root ordb/.applyPendingRestorepasses the staged archive's root, while the source stages everything underdb/. Wired as this README showed, every boot restore set the database aside, failed on<db>.dump is not in the archiveand rolled back. If the given directory contains adb/directory, that is where the dumps are read from. Otherwise the directory itself is used, so a caller that already passesdb/is unchanged.Globals: a restore that needs a missing role is refused before anything moves; otherwise they are applied tolerantly, and no longer silently. Postgres checks permission before existence, so a non-superuser gets
permission denied to create role/to alter rolefor every line ofglobals.sqlon an intact cluster, not "already exists". Even CREATEROLE cannot alter the superuser's line.checkGlobals()compares the rolesglobals.sqlcreates withpg_roles:- All present: nothing to do.
- The restoring role is a superuser: nothing to do.
applyGlobalscreates the missing ones, as 0.33.0 did. - A missing role the DUMP refers to: refused, before anything moves. "Refers to" means it owns
an object, holds or granted a privilege, owns default privileges, or is a policy target; this
is read from
pg_restore --schema-only. The message names the role(s) and says "this is a bare-metal rebuild: restore globals.sql as a superuser first". - Missing roles the dump does not refer to (another app's roles, say): one warning, and the restore proceeds.
preflight(databases, stagedDir)runs this check, and so doesapply()before any DROP.- psql exits 0 on statement errors, so stderr is now read on success.
applyGlobals()returns everyERRORline, andapply()reports them as one warning listing them all.applyPendingRestorelogs it and keeps it inrestore.result. - A psql that could not start is still a failure.
- The error lines are found by matching
ERROR:, so the server'slc_messagesmust be English for them to be recognised. Translated messages are not matched. They are then missing from the warning, but the refusal above does not depend on them.
Passwords go in
PGPASSWORD, never on a command line, on both the capture and the restore side. That coverspg_dumpandpg_dumpallin the source,restorePostgres, and every psql and pg_restore ofcreatePostgresRestore(the administrator URL too). Any local user can read a child's argv for as long as it runs. A?password=parameter is moved the same way.sslpasswordcannot be: libpq has no environment variable for it. So a URL carrying one is refused when the source or restore is created. Use an unencrypted client key readable only by the service account.restore.inprogressis kept whenever a database is not confirmed back under its name. This covers two cases:rollbackAside()put back fewer databases than were set aside, or threw.moveAside()renamed one, a later step failed, and its own unwind could not rename it back. In that casemoveAside()now records a database the moment its RENAME succeeds and throws witherr.stranded.
The marker is rewritten with the stranded names and the recovery statements. The next start refuses and prints them one per line, instead of booting on the half-restored database, or on none. A full rollback still removes the marker.
ANALYZEafter the replay, run as the owner (SET ROLE). A dump carries no statistics, and under--single-transactionautovacuum cannot analyze anything until the replay commits. An app measured its first queries after a large restore running into its statement timeout. TheANALYZEis bounded bystatement_timeout:analyzeTimeoutMs(a positive integer, default 15 minutes). A failure is a warning, not a failed restore.analyze: falseskips it. An app that runs its ownANALYZEafter a restore can drop it.Database-level properties survive the recreate.
moveAside()captures them beside owner and locale, asattributes.properties. A caller that passes attributes without them still gets the captured ones.- The ACL and the comment are re-applied after the CREATE and before the replay,
fail-closed. A database restored without a REVOKE its operator relied on would be open to
every role on the server. The ACL is re-granted exactly; a non-owner grantor is kept with
GRANTED BYon PG14+, and before 14 the grant is made plainly, so its grantor becomes the owner. ALTER DATABASE … SETandALTER ROLE … IN DATABASE … SETare re-applied after the replay and the ANALYZE. A setting changes every later session, the replay's included:default_transaction_read_only = onbroke it. A setting that cannot be re-applied, such as a superuser-only parameter issued by the owner, or another role's setting without CREATEROLE + ADMIN, is a warning carrying the exact statement for an administrator to run. It never fails the restore. List settings such assearch_pathare written element by element, as pg_dump does.
- The ACL and the comment are re-applied after the CREATE and before the replay,
fail-closed. A database restored without a REVOKE its operator relied on would be open to
every role on the server. The ACL is re-granted exactly; a non-owner grantor is kept with
Upgrading to 0.33.1
- The boot restore works when wired as shown:
databaseRestore: createPostgresRestore(…)with no wrapper. An app that wrappedapply()to passpath.join(stagedDir, 'db')can keep doing so, or drop the wrapper. - An archive that refers to a role this server lacks is now refused before anything moves,
unless a superuser restores it. 0.33.0 went on and the replay failed or lost the grant. Restore
globals.sqlas a superuser first, then retry. A wrapper'spreflightshould forward its second argument (stagedDir) so the refusal comes beforemoveAside. Without it, the refusal comes inapply(), which is still before the DROP, and the aside is rolled back. - A connection URL with
sslpasswordis refused bycreatePostgresSourceandcreatePostgresRestore. applyGlobals()returnserrors(the tolerated ERROR lines).apply()returnswarnings, andapplyDatabase()returnsanalyzedandwarnings.restore.resulton success haswarnings.- A failed restore that leaves a database under its aside name now blocks every start until
an operator follows the recovery steps in
restore.inprogressand deletes it. An app that re-created the marker itself finds it already there and does nothing. AISearch'srefuseAfterIncompleteRollbackis an example. - Restores take longer by the
ANALYZE(analyzeTimeoutMs,analyze: false). - A database ACL or comment the restoring role cannot re-apply fails the restore, which is rolled
back. Through 0.33.0 they were silently dropped. Per-database settings never fail it: read
restore.result.warnings.
Replication, WAL archiving and PITR are NOT in this package
See docs/POSTGRES-OPERATIONS.md. The boundary is not squeamishness:
PITR restores an entire cluster and cannot restore one database, so a package claiming to drive
it would be claiming something the tool does not do. That document covers what to configure, how to
verify it, and the failure mode that actually takes servers down — a broken archive_command does
not stop the server, it fills the disk.
Key ring (0.3.0)
Which password encrypted which archive, behind ports so it needs neither a database nor a keystore to run:
const keyring = backup.createKeyring({
settings: { all: () => …, set: (patch) => … }, // non-secret LABELS
secrets: { load: () => …, save: (next) => … }, // the passwords
crypto: core
});
await keyring.rotate(newPassword); // verifies a round trip BEFORE activating
await keyring.listGenerations(); // metadata only — no reveal permission needed
await keyring.revealPassword(gen); // caller must check the permission and audit itThe split between the two stores is the design. Generation number, id and rotation date are labels: they live in the plain store so an admin page can show "this install is on generation 3" without decrypting anything. Only the passwords sit behind the reveal permission.
The invariant is "never lose a password", and it has been broken before. An earlier
version gated retirement on Number.isInteger(prevGen); on the most common upgrade path — a
password already set, no generation pointer yet — that was NaN, retirement was skipped, and
the save overwrote the only copy of the password protecting every existing archive. So the
two decisions that can lose one are pure functions, exported unbound and tested with plain
objects:
nextGeneration(pointer, history, passwords)— max+1 across all three sources, because they can disagree after a database is restored from an older copy beside a newer keystoreplanRetirement({...})— never drops the outgoing password; files it under a generation only when that does not contradict what is already stored, and unlabelled when it does. A mislabelled password is a wrong answer; an unlabelled one is an honest "try these".
rotate() verifies the new password can encrypt AND decrypt before it becomes the archive
key — without that, a password that cannot round-trip becomes live silently and the failure
surfaces at restore, months later. It is single-flight: a double-submitted form must not mint
two different passwords under one generation label.
Orchestrator (0.4.0)
Capture once, fan out, prune, record — plus the scheduler and the once-per-boot history reconciliation:
const orch = backup.createOrchestrator({
core, logger,
runRecorder: { start, finish, failStale, reconcileCaptureRun },
cron: require('node-cron') // optional peer
});
await orch.runExclusive(() => orch.performBackup({ source, keys, extras, settings, secrets, password }));
await orch.reconcileHistoryAtBoot(dataDir);
await orch.startScheduler({ enabled, expression, run }); // expression is evaluated in UTCThe schedule is UTC. startScheduler calls cron.schedule(expression, fn, { timezone: 'UTC' }),
so a cron port that is not node-cron must honour that third argument.
a/n is read as "from a, every n" (0.32.1). 0 0/6 * * * is scheduled as 0 0-23/6 * * *.
node-cron 4.6 refuses the short form, so an install that had stored one stopped taking scheduled
backups on upgrade; node-cron 4.2 accepted it and ran it once a day. Upgrading to 0.32.1 therefore
makes such a schedule run as often as it says, which is more often than 4.2 ran it.
backup.normalizeCron(expression) is exported: validate and store its result on a settings page,
so what is shown is what runs. An expression that still cannot be scheduled is logged as an error.
The archive is written once to a temp file, then streamed to each destination — a single readable cannot fan out to N writers — and the file just written is protected from retention on every destination. Pruning the backup you have this second taken is the one bug a retention policy must never have.
A capture that fails leaves no temp archive (0.32.2). An error on the archive stream after
createArchive returned (tar meeting a missing entry, a source read error) now destroys both
streams and rejects only once the temp file is closed, so performBackup's cleanup always finds
and removes it. Before, the write stream was left running: it could create the file after the
cleanup had looked, or hold it open where Windows would not delete it, leaving a partial
<name>.<hex>.tmp in the temp directory about one failed run in a hundred. A temp file that
still cannot be removed is now logged as an error instead of ignored. No change for an app.
A retention failure does not fail the backup. The archive was written; refusing to report that because a prune failed afterwards would be a worse outcome than a slightly fuller disk. The failure is returned in the result, not swallowed.
reconcileHistoryAtBoot runs once per boot, deliberately not on the reschedule path. An
app that reschedules on every settings save would otherwise mark a live, in-flight backup as
interrupted. It settles the restored capture run BEFORE failing stale runs, and clears the
restore marker only after the history write lands — clearing it first loses the only note to
a transient failure.
A latent crash this surfaced
Node opens a ReadStream lazily. A destination that rejects before consuming — refused
connection, bad credentials, full disk — leaves the stream to open after the caller's
finally has deleted the temp file, and it emits an 'error' with no listener. An unhandled
'error' event terminates the process: the app dies while handling a backup failure it had
otherwise reported perfectly well.
Reproduced against the consuming app before fixing it here. The stream is now destroyed on the failure path, with its errors swallowed — the destination's error is the cause, and an ENOENT on a file about to be deleted is noise on top of it.
UI (0.5.0)
Partials, client behaviour and a specification — so an app gets the same page rather than reinventing one, and an app that cannot use the partials can still reproduce the shape.
// the app's own views dir FIRST; both Express and EJS accept an array
app.set('views', [path.join(__dirname, 'views'), backup.viewsDir]);
app.use('/js/backup', express.static(backup.assetsDir));<%- include('backup/key-generations-modal', {
keyGenerations, canReveal: can('backup.reveal_password'),
revealPath: '/admin/backup/password/reveal', formatDateTime
}) %>Body partials only, never a whole page. The two apps this was built for mount at different paths, use different chrome and have different permission names. All three are injected; the package assumes none of them.
docs/UI-SPEC.md is the durable half. It states the section order, every state, and the
rules that are not stylistic — pending restore states never behind a click, reveals cleared
when their container closes so re-reading is audited again, a schedule switch that hides the
builder but not the retention that manual runs still obey, a copy control that falls back to
selecting text where navigator.clipboard does not exist.
assets/backup-ui.js and assets/backup-ui.css are plain files — no bundler — and are safe
to include on pages containing none of these controls.
