@nowbackup/compression

v1.0.3

Published

5 days ago

Gzip compression module for NowBackup database backups

0High
0Medium
0Low

jd-ashish

nowbackup compression gzip backup

🚀 NowBackup SDK

Universal Database Auto Backup SDK and CLI for Node.js and Enterprise Environments.

⚡ Powered by NowMail

NowBackup is a production-grade, enterprise-ready backup solution that automatically detects your database, creates non-blocking backups, compresses, encrypts, and uploads them to your preferred cloud storage provider.

📖 Table of Contents

⚡ Key Features

Auto-Detection: Zero-config setup. Detects database types from environment variables or active package dependencies automatically.
Non-Blocking Streaming: Direct pipeline stream ([DB Driver] -> [Gzip] -> [AES-256] -> [Storage]) keeps memory footprint under 50MB even for databases larger than 100GB.
High-Fidelity fallbacks: Fallback JS engines generate standard backup scripts if system binaries (pg_dump, mysqldump) are not locally installed.
Multi-Database Scheduling: Support for backing up multiple databases in a single Node.js instance.
Built-in Rotation: Retention engine automatically rotates and purges old backups according to your policy.

📦 Installation

1. Unified Package (Recommended)

Install the self-contained package from npmjs (or your local packaged tarball):

npm install nowbackup

🚀 Quick Start (Zero-Config Auto-Detection)

If you have database connection details set up in standard environment variables, NowBackup will automatically detect your database type, construct connection pools, and schedule a cron job:

import { NowBackup } from 'nowbackup';

await NowBackup.init({
  database: 'auto',          // 👈 Automatically scans and schedules backups for Postgres, MySQL, Mongo, Redis, SQLite, MS SQL, Cassandra, or Neo4j!
  schedule: '0 0 * * *',     // Every night at midnight
  storage: {
    provider: 'local',
    options: {
      path: './backups'
    }
  },
  compression: true,         // Gzip compression
  retention: {
    keepLast: 10             // Automatically retains the 10 most recent backups
  }
});

🔄 Handling Multiple Auto-Detected Databases (Array return)

When you set database: 'auto', NowBackup dynamically scans your environments and dependencies for database details.

If multiple databases are detected (e.g. your app uses both a Postgres database and a Redis cache), NowBackup.init() will return an Array of backup instances.
If only a single database is detected (or you specify a database type explicitly, like database: 'postgres'), it returns a single backup instance.

To handle both cases safely in your code, use the standard Array.isArray() check:

import { NowBackup } from 'nowbackup';

const backup = await NowBackup.init({
  database: 'auto',
  schedule: '0 0 * * *', // Daily at midnight
  storage: { provider: 'local', options: { path: './backups' } }
});

if (Array.isArray(backup)) {
  console.log(`📡 Auto-detected and scheduled ${backup.length} databases!`);

  // (Optional) Trigger all backups instantly:
  for (const instance of backup) {
    await instance.run();
  }
} else {
  console.log(`📡 Auto-detected and scheduled a single database!`);

  // (Optional) Trigger backup instantly:
  await backup.run();
}

[!NOTE] If you provided a schedule cron configuration, the background scheduler automatically activates and runs your backups in the background immediately upon calling .init(). Calling await backup.run() manually is only required if you want to trigger a one-off, immediate backup in addition to the schedule.

📝 Programmatic Multi-Database Configuration (No Environment Variables)

If you do not use environment variables for your database connection strings, you can still easily backup multiple databases concurrently by passing an array of configuration objects directly into NowBackup.init().

This approach is completely self-contained and allows you to configure completely different connection URLs, cron schedules, storage destinations, and compression/retention policies for each database separately:

import { NowBackup } from 'nowbackup';

// Pass an array of database configurations directly to the SDK
const backupInstances = await NowBackup.init([
  {
    database: 'postgres',
    databaseUrl: 'postgres://user:pass@localhost:5432/my_production_db',
    schedule: '0 0 * * *', // Every night at midnight
    storage: {
      provider: 'local',
      options: { path: './backups/postgres' }
    },
    compression: true,
    retention: { keepLast: 15 }
  },
  {
    database: 'mysql',
    databaseUrl: 'mysql://root:secret@localhost:3306/another_db',
    schedule: '0 2 * * *', // Daily at 2 AM
    storage: {
      provider: 's3',
      options: {
        bucket: 'my-mysql-s3-backups',
        region: 'us-east-1',
        credentials: {
          accessKeyId: 'AWS_ACCESS_KEY_ID',
          secretAccessKey: 'AWS_SECRET_ACCESS_KEY'
        }
      }
    },
    compression: true
  }
]);

console.log(`🚀 Successfully initialized and scheduled ${backupInstances.length} database backups programmatically!`);

⚙️ Programmatic Configuration Reference

The NowBackup.init(config) method takes a BackupConfig structure:

📢 Smart Logging Controls

NowBackup features an intelligent logger designed to keep your server console quiet, clean, and perfectly aligned with your preferences:

1. Default (Perfect Silence)

By default, the SDK runs completely silently. No initialization notices, scheduler details, or retry logs will clutter your process.

await NowBackup.init({
  database: 'auto',
  storage: { provider: 'local', options: { path: './backups' } }
  // log_enable defaults to false
  // log defaults to false
});

2. Success Summary Only (Log Enable: `true`, Log: `false`)

Enable logs but keep them non-verbose. Ideal for production servers where you want a clean audit trail without driver execution noise:

await NowBackup.init({
  database: 'auto',
  storage: { provider: 'local', options: { path: './backups' } },
  log_enable: true,
  log: false // 👈 Show ONLY a clean summary line
});

Console output when a backup runs:

[18/05/2026, 16:44:01] Backup done for sqlite database.
[18/05/2026, 16:44:02] Backup done for mysql database.

3. Full Verbose Logging (Log Enable: `true`, Log: `true`)

Log every detailed lifecycle event "as it is". Ideal for debugging or development environments:

await NowBackup.init({
  database: 'auto',
  storage: { provider: 'local', options: { path: './backups' } },
  log_enable: true,
  log: true // 👈 Verbose logging active
});

Console output when a backup runs:

⚡ NowBackup - Powered by NowMail (https://nowmail.in)
[NowBackup] Running multi-database auto-detection...
[NowBackup] Auto-detected databases: mysql, sqlite
[NowBackup] Initializing instances for 2 detected databases.
[NowBackup] Scheduling backup: */10 * * * * * for sqlite
[NowBackup] [2026-05-18T11-15-40-165Z] Starting backup sequence for sqlite...
[SqliteDriver] Starting backup: ./database.sqlite
[NowBackup] Applying Gzip compression...
[NowBackup] Uploading to local...
[NowBackup] SUCCESS: Backup stored at backups\backup-sqlite-2026-05-18T11-15-40-165Z.sql.gz
[18/05/2026, 16:45:40] Backup done for sqlite database.

🗄️ Database-Specific Setups

For each database, you can pass the connection string programmatically via the databaseUrl key, or let the SDK resolve it from environment variables:

1. PostgreSQL

Programmatic URL: postgres://user:pass@localhost:5432/mydb
Auto-Detect Env: DATABASE_URL, POSTGRES_URL, POSTGRES_DATABASE_URL, PGHOST (with pg in dependencies).
Backup Output: Standard plain SQL dump file (.sql). Uses pg_dump if installed on host, or falls back to our robust internal JS table exporter.

2. MySQL

Programmatic URL: mysql://user:pass@localhost:3306/mydb
Auto-Detect Env: MYSQL_URL, MYSQL_DATABASE_URL, MYSQL_HOST (with mysql or mysql2 in dependencies).
Backup Output: Standard SQL dump file (.sql). Uses mysqldump if installed on host, or falls back to our robust internal JS table exporter.

3. SQLite

Programmatic URL: file://relative/path/to/database.sqlite (or absolute path)
Auto-Detect Env: SQLITE_URL, SQLITE_FILE. Defaults to ./database.sqlite if no environment variable is provided.
Backup Output: Binary database copy file (.sqlite).

4. MongoDB

Programmatic URL: mongodb://user:pass@localhost:27017/mydb or mongodb+srv://...
Auto-Detect Env: MONGO_URI, MONGODB_URL.
Backup Output: BSON archive dump file. Uses native mongodump tool.

5. Redis

Programmatic URL: redis://user:pass@localhost:6379
Auto-Detect Env: REDIS_URL.
Backup Output: Standard Redis dump dataset.

6. MS SQL Server

Programmatic URL: mssql://username:password@localhost:1433/database_name or sqlserver://...
Auto-Detect Env: MSSQL_URL, SQLSERVER_URL, MSSQL_CONNECTION_STRING, MSSQL_HOST.
Backup Output: Transact-SQL schema and insert statements script. Automatically detects identity columns and injects SET IDENTITY_INSERT ... ON/OFF rules to ensure clean restores.

7. Apache Cassandra

Programmatic URL: cassandra://username:password@localhost:9042/keyspace_name
Auto-Detect Env: CASSANDRA_URL, CASSANDRA_HOST, CASSANDRA_CONTACT_POINTS.
Backup Output: Cassandra Query Language (CQL) dump script (.sql). Resolves complete column family configurations and exports all row datasets.

8. Neo4j Graph Database

Programmatic URL: neo4j://username:password@localhost:7687 or bolt://...
Auto-Detect Env: NEO4J_URL, NEO4J_URI, NEO4J_HOST.
Backup Output: Standard Cypher script file (.cypher). Streams all graph elements (nodes, properties, and relationships) and binds them dynamically via temporary internal identifiers (_nowbackup_id).

💾 Storage Providers

1. Local Storage

Stores files locally in the specified path:

storage: {
  provider: 'local',
  options: {
    path: './backups' // Folder where backups will be stored
  }
}

2. AWS S3 (and S3-Compatible providers like MinIO or Cloudflare R2)

Uploads streams directly into an S3 bucket:

storage: {
  provider: 's3',
  options: {
    bucket: 'my-production-backups',
    region: 'us-east-1',
    credentials: {
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
    },
    // Optional endpoint for S3 compatibles (e.g. MinIO, Cloudflare R2)
    // endpoint: 'https://<accountid>.r2.cloudflarestorage.com'
  }
}

🔒 Built-in Encryption & Compression

Streaming Gzip Compression

To compress backups on-the-fly and save up to 80% storage space, enable compression:

compression: true // Appends '.gz' to the output filename

Streaming AES-256-GCM Encryption

To protect your production backups, you can enable military-grade streaming encryption. Make sure to specify a 32-character secret key:

encryption: {
  enabled: true,
  algorithm: 'aes-256-gcm',
  key: 'my-super-secret-secure-32-char-key!!' // 👈 MUST be exactly 32 characters
} // Appends '.enc' to the output filename

💻 CLI Tool Usage

NowBackup includes a command-line interface allowing you to run one-off backups instantly from your terminal without writing any code.

Command Syntax:

npx nowbackup run --database <type> --url <connection-string> --storage <local|s3> [options]

Available Parameter Flags:

-d, --database <type>: Database type (postgres, mysql, mongodb, redis, sqlite, mssql, cassandra, neo4j, auto)
-u, --url <url>: Database connection string URL (falls back to environmental variables if not set)
-s, --storage <provider>: Storage provider (local or s3)
-p, --path <path>: Local storage target directory path (for local storage, defaults to ./backups)
-b, --bucket <bucket>: AWS S3 bucket name
-c, --compress: Apply Gzip compression (enabled by default)
-e, --encrypt <key>: Enable AES-256-GCM encryption with the provided 32-character key

Example CLI Backup Command:

npx nowbackup run -d mysql -u "mysql://root:password@localhost:3306/db" -s local -p "./backups" --encrypt "my-32-character-secret-key-123"

📦 Local Distribution & Tarball Installation

The monorepo contains a dedicated, cross-platform build and package utility. You can generate a single unified, self-contained NPM package tarball that includes the programmatic SDK and the terminal CLI runner with zero internal monorepo package dependencies at runtime:

1. Build and Generate Tarball

From your nodejs root monorepo folder, run:

npm run pack

This generates exactly one unified tarball file inside the packs directory:

packs/nowbackup-1.0.0.tgz

2. Install and Use locally

Copy the .tgz file to any project and install it:

npm install /path/to/packs/nowbackup-1.0.0.tgz

You can now import and use it instantly in your project:

import { NowBackup } from 'nowbackup';