s3-forklift

aws-sdk based S3 uploader with support for AWS S3, DigitalOcean Spaces, and Cloudflare R2. It wraps upload method of aws-sdk. For file upload, ContentType automatically added. Also, it can remove uploaded file from the file system, after upload completed successfully.

Install

npm install s3-forklift --save

Test

Rename secret.example.json to secret.json with valid credentials.

Then, run test via npm run test

Requirements

• Node.js 10+

Initialize

AWS S3

const Forklift = require("s3-forklift");

const forklift = new Forklift({
  provider: "aws",
  accessKey: "<YOUR_ACCESS_KEY>",
  secretKey: "<YOUR_SECRET_KEY>",
  bucket: "<BUCKET_NAME>",
  region: "<REGION>", // required for aws
  s3params: {ACL: "bucket-owner-read"} // optional, default: {ACL: "public-read"}
});

DigitalOcean Spaces

const Forklift = require("s3-forklift");

const forklift = new Forklift({
  provider: "do",
  accessKey: "<YOUR_ACCESS_KEY>",
  secretKey: "<YOUR_SECRET_KEY>",
  bucket: "<BUCKET_NAME>",
  region: "fra1", // required for do (e.g., nyc3, sfo2, fra1)
  s3params: {ACL: "public-read"}
});

Cloudflare R2

const Forklift = require("s3-forklift");

const forklift = new Forklift({
  provider: "r2",
  accessKey: "<YOUR_R2_ACCESS_KEY_ID>",
  secretKey: "<YOUR_R2_SECRET_ACCESS_KEY>",
  bucket: "<BUCKET_NAME>",
  endpoint: "https://<ACCOUNT_ID>.r2.cloudflarestorage.com", // required for r2
  // region is optional for r2, defaults to "auto"
  s3params: {ACL: "public-read"}
});

Custom S3-Compatible Endpoint

You can also use any S3-compatible storage by providing a custom endpoint:

const forklift = new Forklift({
  provider: "aws", // or "do"
  accessKey: "<YOUR_ACCESS_KEY>",
  secretKey: "<YOUR_SECRET_KEY>",
  bucket: "<BUCKET_NAME>",
  region: "us-east-1",
  endpoint: "https://your-custom-s3-endpoint.com" // optional custom endpoint
});

Upload

• source should be string (file path) or readable stream.
• remotePath s3 path.
• options Besides the all the options of original S3.upload you can pass
  • {remove: true} to remove the source after upload completed successfully
  • {timestamp: true} to add timestamp at the end of url

const url = await forklift.upload({source, remotePath, options});

Example

// To upload file
// ContentType automatically retrieved from file name and passed to S3.upload method
// if you want to override it, you should pass {ContentType:"<CONTENT_TYPE>"} as options.
const url = await forklift.upload({
  source: "test.jpg", 
  remotePath: "test/test.jpg"
});

// To upload and then remove the file with callback
const url = await forklift.upload({
  source: "test.jpg", 
  remotePath: "test/test.jpg",
  options: {remove: true}
});

// To upload a stream without ContentType
const url = await forklift.upload({
  source: fs.createReadStream("test.jpg"), 
  remotePath: "test/test.jpg"
});

// To upload a stream with ContentType
const url = await forklift.upload({
  source: fs.createReadStream("test.jpg"), 
  remotePath: "test/test.jpg",
  options: {ContentType:"image/jpeg"}
});

Changelog

1.2.0

• Added Cloudflare R2 support (provider: "r2")
• Added custom endpoint support for any S3-compatible storage
• Increased accessKey max length to 128 characters (for R2 compatibility)
• Region is now optional for R2 (defaults to "auto")

1.1.0

• Added DigitalOcean Spaces support

1.0.0

• Initial release with AWS S3 support