CDK Rsync Backup

AWS CDK L3 construct for cloud backup system with rsync.

Create a full system backup of an on-premise server as EBS snapshots. When you send files with rsync, the system will

• Create an EBS volume (from snapshot if exists)
• Attach and mount the volume to the EC2 instance
• Receive rsync data
• Unmount and detach the volume
• Create a EBS snapshot from the volume
• Delete the volume
• Upload log file into S3 bucket

Installation

$ npm install @anyakichi/cdk-rsync-backup

Usage

Backup

import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import { RsyncBackup } from "@anyakichi/cdk-rsync-backup";

export class CdkDemoStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create `hostname` rsync module with 100 GiB EBS.
    // SSH public key is required per module.
    const rsyncBackup = new RsyncBackup(this, "RsyncBackup", {
      modules: [
        {
          name: "hostname",
          sshKey: "ssh-rsa AAAAB3Nza...",
          size: 100,
        },
      ],
    });
  }
}

If no EBS snapshot for hostname does not exist, a new EBS volume is created on rsync execution with specified size.

If an EBS snapshot exists, a new EBS volume is created from the latest EBS snapshot. When the size parameter differs from the snapshot size:

• Growing: the volume is created with the new size and the filesystem is expanded automatically before receiving data.
• Shrinking: the backup is received on a volume of the previous size first, so the client never has to wait. After the transfer completes, the data is copied to a new smaller volume in the background and the snapshot is taken from the new volume. If the data does not fit into the new size, shrinking is skipped and the previous size is kept.

Note that size is embedded in the instance's authorized_keys at first boot. To apply a size change to an existing deployment, replace the instance by setting or bumping the instanceVersion property.

When you create a new backup, simply execute rsync.

# rsync -azAXHS -e 'ssh -i id_rsa' --delete --numeric-ids --inplace \
    --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"}
    / [email protected]::hostname/

You can get the host SSH private key to login your instance from AWS System Manager Parameter Sotre.

$ aws ssm get-parameter \
    --name /ec2/keypair/$(aws ec2 describe-key-pairs \
      --key-names rsync-backup-keypair --query "KeyPairs[].KeyPairId" --output text) \
    --with-decryption --query Parameter.Value --output text

Restore

Create a new EC2 instance and attach a volume created from a required snapshot, then copy the backup data.

Reading backup data from the backup instance is disabled by default.

Swap file

Small instance types (e.g. t4g.nano) have limited memory and may fail under heavy rsync workloads. You can optionally enable a swap file on the instance by specifying swapSize in GiB. When set, the instance creates /swapfile on first boot and registers it in /etc/fstab for automatic activation on subsequent boots.

const rsyncBackup = new RsyncBackup(this, "RsyncBackup", {
  swapSize: 1, // 1 GiB swap file at /swapfile
  modules: [
    /* ... */
  ],
});

Recommended workflow

Backup data transfer on Internet is not always reliable. Two-way backup is recommended for reliability.

• Take a snapshot on local machine. rsnapshot with LVM snapshot is easy and convenient. This should be finished in predictable time.

• Send a local backup to EC2 instance.

First level backups can be used for recovery from operation mistakes. You can easily copy the files from local backups.

Second level backups are for disaster recovery. Its recovery cost is relatively high.