mongodb-backup-service

Reusable Node.js service for automated MongoDB backups with scheduling, startup recovery, Excel export, retention, email notifications, and backup locking.

Key Features

• MongoDB Atlas Backup: Automated mongodump execution supporting local and Atlas clusters.
• Gzip Archive: Compresses database dumps natively via mongodump --gzip.
• Excel Export: Secondary human-readable .xlsx export, streaming large collections efficiently.
• Collection Exclusion: Skip unnecessary collections (e.g., sessions, temporary logs) from the Excel export.
• Advanced Scheduling: Supports daily, weekly, and monthly CRON scheduling.
• Configurable Timezone: Run backups according to your local timezone (e.g., Asia/Kolkata, America/New_York).
• Startup Missed-Backup Recovery: Automatically detects and recovers missed scheduled backups if the Node.js service was offline.
• Backup Locking: Ensures exactly-once execution and prevents concurrent backup collisions.
• Retention Cleanup: Automatically prunes old backups to save storage space (keeps the N most recent backups).
• Email Notifications: Detailed success/failure reports via SMTP, separating MongoDB and Excel statuses.
• Local Storage Abstraction: Clean, organized date-based directory structure for all artefacts.
• CLI Tool: Built-in mongodb-backup command for manual triggers and status checks.
• Metadata Tracking: Detailed metadata.json for every backup run.
• Graceful Shutdown: Safe SIGINT/SIGTERM handling to stop the scheduler cleanly.

Why this package exists

Managing automated database backups in Node.js applications often leads to scattered shell scripts and cron jobs that lack proper error handling, monitoring, and idempotency. mongodb-backup-service provides a robust, in-process solution that you can seamlessly integrate into your existing Express or Node.js backend. It treats backups as a first-class feature with strong guarantees against duplicate runs and missed schedules.

Architecture

graph TD
    A[Express / Node Backend] -->|backupService.start| B(DatabaseBackupService)
    B --> C{Scheduler}
    C -->|Trigger| D(BackupManager)
    B -->|Startup Recovery| D
    
    D -->|1. Acquire Lock| E[BackupLock]
    D -->|2. Dump| F[MongoDump]
    D -->|3. Export| G[ExcelExporter]
    D -->|4. Clean| H[RetentionManager]
    D -->|5. Notify| I[EmailNotifier]
    
    F --> J[(Storage: .gz)]
    G --> K[(Storage: .xlsx)]
    
    subgraph Storage Directory
    J
    K
    L[metadata.json]
    end
    
    D -->|Writes| L

Requirements

• Node.js: >=18.0.0
• MongoDB Database Tools: mongodump must be installed on the host system and available in the system PATH.
• MongoDB Connection: A valid MongoDB URI (supports MongoDB Atlas).
• SMTP Server: Required only if email notifications are enabled.

Installation

npm install mongodb-backup-service

Configuration

The service is configured using a JavaScript object, typically populated via environment variables (.env). Copy .env.example to .env in your project root.

| Variable | Required | Description | Example Safe Value | |---|---|---|---| | MONGO_URI | Yes | MongoDB connection string. | mongodb+srv://<user>:<pwd>@cluster... | | BACKUP_PROJECT_NAME | No | Identifier for logs and emails. | My Project | | BACKUP_STORAGE_PATH | No | Local directory to save backups. | ./backups | | BACKUP_SCHEDULE_TYPE | No | daily, weekly, or monthly. | daily | | BACKUP_SCHEDULE_TIME | No | Time to run backup (HH:MM). | 02:00 | | BACKUP_TIMEZONE | No | Valid IANA Timezone. | UTC | | BACKUP_EXCEL_ENABLED | No | Enable/disable Excel export. | true | | BACKUP_EXCLUDE_COLLECTIONS | No | Comma-separated collections to skip in Excel. | sessions,temporaryLogs | | BACKUP_RETENTION_ENABLED | No | Enable/disable cleanup of old backups. | true | | BACKUP_RETENTION_DAYS | No | Number of most recent backups to keep. | 30 | | BACKUP_EMAIL_ENABLED | No | Enable/disable SMTP notifications. | true | | BACKUP_EMAIL_TO | No | Alert recipient address. | [email protected] | | BACKUP_SMTP_HOST | No | SMTP server hostname. | smtp.gmail.com | | BACKUP_SMTP_PORT | No | SMTP port (e.g., 587, 465). | 587 | | BACKUP_SMTP_SECURE | No | Use TLS (true for 465, false for 587). | false | | BACKUP_SMTP_USER | No | SMTP username. | [email protected] | | BACKUP_SMTP_PASSWORD | No | SMTP password/App Password. | xxxx-xxxx-xxxx-xxxx | | BACKUP_EMAIL_FROM | No | Sender address for alerts. | [email protected] |

Security Note: Never commit your real .env file containing database credentials or SMTP passwords to version control.

Express Backend Integration

Integrate the backup service into your existing Node.js or Express application lifecycle:

require('dotenv').config();
const express = require('express');
const backupService = require('mongodb-backup-service');

const app = express();

// 1. Configure and start the backup service
backupService.start({
    mongoUri: process.env.MONGO_URI,
    projectName: process.env.BACKUP_PROJECT_NAME || 'My App',
    storage: { path: process.env.BACKUP_STORAGE_PATH || './backups' },
    schedule: {
        type: process.env.BACKUP_SCHEDULE_TYPE || 'daily',
        time: process.env.BACKUP_SCHEDULE_TIME || '02:00',
        timezone: process.env.BACKUP_TIMEZONE || 'UTC'
    },
    // ... include excel, retention, and email configs ...
});

const server = app.listen(3000, () => {
    console.log('Server running on port 3000');
});

// 2. Ensure graceful shutdown
const shutdown = () => {
    console.log('Shutting down server...');
    backupService.stop(); // Stops the scheduler cleanly
    server.close(() => {
        process.exit(0);
    });
};

process.on('SIGINT', shutdown);
process.on('SIGTERM', shutdown);

Scheduling

The internal scheduler uses standard cron expressions based on your configuration:

• daily: Runs every day at the specified time.
• weekly: Requires a day (e.g., sunday) and time.
• monthly: Requires a day (e.g., 1 for the 1st of the month) and time.

The timezone parameter ensures that Daylight Saving Time and regional offsets are respected.

Startup Missed-Backup Recovery

If your Node.js application is stopped or crashes during the scheduled backup window, the backup will not run while the process is offline.

To prevent data gaps, mongodb-backup-service performs a Startup Recovery Check when you call backupService.start(). It calculates the date of the most recently expected scheduled backup. If the backup directory for that date is missing, or its metadata indicates it did not complete successfully, the service will immediately trigger a recovery backup in the background.

Backup Locking / Duplicate Protection

To prevent overlapping backups (e.g., a manual trigger coinciding with a scheduled run, or a recovery run overlapping with a cron tick), the service utilizes a file-based lock (.backup.lock).

This ensures exactly-once execution. If a backup is already in progress, any subsequent trigger (cron, recovery, or manual) will be safely rejected, logging a warning rather than corrupting the archive.

Backup Directory Structure

Backups are neatly organized by date (in the configured timezone) within the storage path.

backups/
└── 2026-09-10/
    ├── mongodb.archive.gz   # Compressed mongodump output
    ├── database.xlsx        # Human-readable export
    └── metadata.json        # Execution report

Metadata

The metadata.json file tracks the execution status without exposing any sensitive credentials. It contains:

• projectName and backupDate
• startedAt, completedAt, and durationMs
• status: Overall backup status (success or failed)
• mongoDump: Specific status, filename, and error (if any)
• excel: Specific status, filename, and error (if any)

Excel Export

When excel.enabled is true, the service exports your database to database.xlsx:

• One Sheet per Collection: Each MongoDB collection becomes a separate worksheet.
• Dynamic Columns: Columns are discovered dynamically as documents are streamed.
• Data Types: Dates are preserved. ObjectIds are converted to strings. Nested objects and arrays are flattened into JSON strings to fit in Excel cells.
• Exclusions: Use excludeCollections to skip large or irrelevant collections (like sessions).

Retention

When retention.enabled is true, the RetentionManager runs at the end of every successful backup. It scans the storage directory and keeps the N most recent backup directories (where N is retention.days), deleting older directories to prevent disk exhaustion.

Email Notifications

When email.enabled is true, an SMTP email is dispatched upon backup completion. The email clearly delineates the overall status, the MongoDB dump status, and the Excel export status.

If the email notification itself fails to send (e.g., invalid SMTP credentials), the error is logged, but it does not mark the backup as failed in metadata.json, ensuring the actual data archiving process remains resilient.

CLI

The package includes a command-line interface for manual administration. Ensure your .env is present in the directory where you run the commands.

# Manually trigger a backup right now
npx mongodb-backup backup

# Check the status of today's backup
npx mongodb-backup status

# List all available backup dates on disk
npx mongodb-backup list

# Manually trigger retention cleanup
npx mongodb-backup cleanup

# View help
npx mongodb-backup help

Security

• .env: Always add .env and backups/ to your .gitignore.
• Metadata: No MongoDB URIs, passwords, or SMTP credentials are ever written to metadata.json or log files.
• Storage: Ensure the host environment has appropriate file permissions for the backups/ directory.

Testing

The package includes a comprehensive suite of unit and integration tests covering scheduling, locking, database operations, and failure scenarios.

npm test

(Currently 60/60 tests passing)

V1 Scope

Version 1 is designed specifically for single-node or replica-set MongoDB deployments (including Atlas) backing Node.js applications, utilizing local disk storage for artefacts and standard SMTP for notifications.

Limitations

• System Dependency: Requires mongodump binary to be installed on the host machine.
• Storage: Currently only supports local filesystem storage.
• Restore: Does not currently include automated mongorestore functionality; archives must be restored manually.

Roadmap

Future considerations (Not yet implemented):

• Support for Cloud Storage (AWS S3, Google Drive, Azure Blob).
• Automated restore workflows.
• Webhook integrations (Slack/Discord alerts).

License

MIT