xStorage

Fastify v5 plugin providing a simple, intuitive library of methods for S3-compatible storage.

Manage file uploads, downloads, and storage operations with Digital Ocean Spaces, AWS S3, Cloudflare R2, and any S3-compatible storage service. For image processing, use @xenterprises/fastify-ximagepipeline.

Requirements

• Fastify v5.0.0+
• Node.js v20+

Features

• 📦 S3-Compatible Storage - Works with Digital Ocean Spaces, AWS S3, Cloudflare R2
• 🔒 Secure by Default - Private ACL with signed URLs for access control
• 🔗 Signed URLs - Generate temporary access URLs for private files
• 📊 File Operations - Upload, download, delete, list, copy, and metadata retrieval
• ⚡ Concurrent Operations - Batch uploads and deletes for efficiency
• 🎯 Simple API - Intuitive methods decorated on Fastify instance

Installation

npm install @xenterprises/fastify-xstorage @aws-sdk/client-s3 @aws-sdk/s3-request-presigner fastify@5

For file uploads via HTTP, also install:

npm install @fastify/multipart@9

Quick Start

import Fastify from "fastify";
import xStorage from "@xenterprises/fastify-xstorage";

const fastify = Fastify({ logger: true });

// Register xStorage
await fastify.register(xStorage, {
  endpoint: "https://nyc3.digitaloceanspaces.com", // Digital Ocean Spaces
  region: "us-east-1",
  accessKeyId: process.env.STORAGE_ACCESS_KEY_ID,
  secretAccessKey: process.env.STORAGE_SECRET_ACCESS_KEY,
  bucket: "your-bucket-name",
  publicUrl: "https://your-bucket-name.nyc3.digitaloceanspaces.com",
  // Default ACL is "private" - use signed URLs for secure access
});

// Upload a file programmatically
const buffer = await fs.readFile("path/to/file.pdf");
const result = await fastify.xStorage.upload(buffer, "file.pdf", {
  folder: "documents",
});

console.log(result);
// {
//   key: "documents/file-a1b2c3d4.pdf",
//   url: "https://your-bucket-name.nyc3.digitaloceanspaces.com/documents/file-a1b2c3d4.pdf",
//   size: 123456,
//   contentType: "application/pdf"
// }

// Generate a signed URL for temporary private file access
const signedUrl = await fastify.xStorage.getSignedUrl(
  result.key,
  3600 // Expires in 1 hour
);

// Use in your application
await fastify.prisma.document.create({
  data: {
    filename: "file.pdf",
    storageKey: result.key,
    size: result.size,
  },
});

await fastify.listen({ port: 3000 });

Configuration

Digital Ocean Spaces

await fastify.register(xStorage, {
  endpoint: "https://nyc3.digitaloceanspaces.com",
  region: "nyc3",
  accessKeyId: process.env.DO_SPACES_KEY,
  secretAccessKey: process.env.DO_SPACES_SECRET,
  bucket: "your-bucket",
  publicUrl: "https://your-bucket.nyc3.digitaloceanspaces.com",
  forcePathStyle: true,
  // acl: "private", // Default - use signed URLs for access
});

AWS S3

await fastify.register(xStorage, {
  region: "us-east-1",
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  bucket: "your-bucket",
  publicUrl: "https://your-bucket.s3.us-east-1.amazonaws.com",
  forcePathStyle: false,
  // acl: "private", // Default - use signed URLs for access
});

Cloudflare R2

await fastify.register(xStorage, {
  endpoint: "https://your-account-id.r2.cloudflarestorage.com",
  region: "auto",
  accessKeyId: process.env.R2_ACCESS_KEY_ID,
  secretAccessKey: process.env.R2_SECRET_ACCESS_KEY,
  bucket: "your-bucket",
  publicUrl: "https://your-custom-domain.com",
  forcePathStyle: true,
  // acl: "private", // Default - use signed URLs for access
});

Core Storage API

All methods are available on the fastify.xStorage namespace.

fastify.xStorage.upload(file, filename, options)

Upload a file to storage.

const result = await fastify.xStorage.upload(buffer, "document.pdf", {
  folder: "documents",
  useRandomName: true,
});

console.log(result);
// {
//   key: "documents/document-a1b2c3d4.pdf",
//   url: "https://your-bucket.nyc3.digitaloceanspaces.com/documents/document-a1b2c3d4.pdf",
//   size: 245678,
//   contentType: "application/pdf"
// }

// Store in database
await db.documents.create({
  data: {
    filename: "document.pdf",
    storageKey: result.key,
    size: result.size,
  },
});

Options:

• folder - Folder path (e.g., "documents", "docs/2024")
• key - Custom storage key (overrides folder/filename)
• contentType - MIME type (auto-detected if not provided)
• metadata - Custom metadata object
• useRandomName - Add random ID to filename (default: true)
• acl - File ACL override (default: uses plugin's configured ACL)
  • "private" - Only accessible via signed URLs
  • "public-read" - Publicly accessible

Example with per-file ACL:

// Private file (default)
await fastify.xStorage.upload(buffer, "private.pdf", {
  folder: "documents",
});

// Public file
await fastify.xStorage.upload(buffer, "public.pdf", {
  folder: "documents",
  acl: "public-read",
});

fastify.xStorage.uploadMultiple(files, options)

Upload multiple files at once with optional per-file ACL control.

const files = [
  { file: buffer1, filename: "private.pdf" },
  { file: buffer2, filename: "public.pdf", acl: "public-read" },
];

const results = await fastify.xStorage.uploadMultiple(files, {
  folder: "documents",
  acl: "private", // Default ACL for all files
});

// Returns array of upload results
// Files can override batch ACL with per-file acl property

fastify.xStorage.delete(key)

Delete a file.

await fastify.xStorage.delete("documents/document-a1b2c3d4.pdf");

fastify.xStorage.deleteMultiple(keys)

Delete multiple files at once.

const keys = ["documents/doc1.pdf", "documents/doc2.pdf"];
await fastify.xStorage.deleteMultiple(keys);

fastify.xStorage.download(key)

Download a file as a buffer.

const buffer = await fastify.xStorage.download("documents/document-a1b2c3d4.pdf");

fastify.xStorage.list(prefix, maxKeys)

List files in a folder.

const files = await fastify.xStorage.list("documents/", 100);

console.log(files);
// [
//   {
//     key: "documents/doc1.pdf",
//     url: "https://...",
//     size: 123456,
//     lastModified: Date,
//     etag: "..."
//   }
// ]

fastify.xStorage.getSignedUrl(key, expiresIn)

Generate a temporary signed URL for private file access.

const url = await fastify.xStorage.getSignedUrl("documents/document-a1b2c3d4.pdf", 3600); // 1 hour

fastify.xStorage.getPublicUrl(key)

Get public URL for a file. Note: This only works if file ACL is set to public.

const url = fastify.xStorage.getPublicUrl("documents/document.pdf");
// Returns: "https://your-bucket.nyc3.digitaloceanspaces.com/documents/document.pdf"

fastify.xStorage.copy(sourceKey, destinationKey)

Copy a file to a new location.

await fastify.xStorage.copy("documents/doc1.pdf", "documents/backup/doc1.pdf");

fastify.xStorage.exists(key)

Check if a file exists.

const exists = await fastify.xStorage.exists("documents/document.pdf");

fastify.xStorage.getMetadata(key)

Get file metadata.

const metadata = await fastify.xStorage.getMetadata("documents/document.pdf");
// Returns: { size, contentType, lastModified, etag, etc }

Image Processing

For image processing, optimization, resizing, thumbnail generation, and format conversion, use @xenterprises/fastify-ximagepipeline.

xImagePipeline integrates with xStorage and provides:

• 🖼️ Image optimization and format conversion
• 🎯 Multiple variant generation (webp, avif, etc)
• 📐 Intelligent resizing and cropping
• 🔍 EXIF metadata extraction and stripping
• 💫 Blur hash generation for progressive loading
• 📊 Compressed original image storage

import Fastify from "fastify";
import xImagePipeline from "@xenterprises/fastify-ximagepipeline";
import xStorage from "@xenterprises/fastify-xstorage";

const fastify = Fastify();

// Register xStorage first
await fastify.register(xStorage, { /* config */ });

// Register xImagePipeline
await fastify.register(xImagePipeline, { /* config */ });

// Use for image processing
const result = await fastify.ximagepipeline.processImage(buffer, "photo.jpg", {
  sourceType: "avatar", // Uses configured variants for avatar
});

Helper Utilities

import { helpers } from "@xenterprises/fastify-xstorage";

// Format file size
helpers.formatFileSize(1234567); // "1.18 MB"

// Check file types
helpers.isImage("photo.jpg"); // true
helpers.isPdf("document.pdf"); // true
helpers.isVideo("movie.mp4"); // true

// Sanitize filename
helpers.sanitizeFilename("My File (2024).jpg"); // "my_file_2024.jpg"

// Calculate dimensions
helpers.calculateFitDimensions(4000, 3000, 1920, 1080);
// { width: 1440, height: 1080 }

// Generate responsive sizes
helpers.generateResponsiveSizes(1920, 1080);
// [
//   { width: 320, height: 180, name: "w320" },
//   { width: 640, height: 360, name: "w640" },
//   // ...
// ]

Usage Examples

Document Upload

import multipart from "@fastify/multipart";

// Register multipart for file uploads
await fastify.register(multipart);

// HTTP endpoint for document upload
fastify.post("/documents", async (request, reply) => {
  const data = await request.file();

  if (!data) {
    return reply.code(400).send({ error: "No file uploaded" });
  }

  const buffer = await data.toBuffer();

  // Upload file
  const result = await fastify.xStorage.upload(buffer, data.filename, {
    folder: "documents",
    useRandomName: true,
  });

  // Save to database
  await fastify.db.document.create({
    data: {
      filename: data.filename,
      storageKey: result.key,
      size: result.size,
      contentType: result.contentType,
    },
  });

  return { success: true, file: result };
});

// Download document with signed URL
fastify.get("/documents/:id", async (request, reply) => {
  const { id } = request.params;

  const document = await fastify.db.document.findUnique({ where: { id } });

  // Generate signed URL valid for 1 hour
  const signedUrl = await fastify.xStorage.getSignedUrl(document.storageKey, 3600);

  return { download: signedUrl };
});

Batch File Operations

// Upload multiple files
fastify.post("/batch-upload", async (request, reply) => {
  const parts = request.parts();
  const files = [];

  for await (const part of parts) {
    if (part.type === "file") {
      const buffer = await part.toBuffer();
      files.push({ file: buffer, filename: part.filename });
    }
  }

  const results = await fastify.xStorage.uploadMultiple(files, {
    folder: "batch-uploads",
  });

  return { success: true, files: results };
});

// Delete multiple files
fastify.post("/batch-delete", async (request, reply) => {
  const { keys } = request.body;

  await fastify.xStorage.deleteMultiple(keys);

  return { success: true, deleted: keys.length };
});

File Organization

// List files in a folder
fastify.get("/files/:folder", async (request, reply) => {
  const { folder } = request.params;

  const files = await fastify.xStorage.list(`${folder}/`, 100);

  return { folder, files };
});

// Move file (copy then delete)
fastify.post("/files/move", async (request, reply) => {
  const { sourceKey, destinationKey } = request.body;

  await fastify.xStorage.copy(sourceKey, destinationKey);
  await fastify.xStorage.delete(sourceKey);

  return { success: true, newLocation: destinationKey };
});

Best Practices

1. Use signed URLs by default - Default ACL is private; always use signed URLs for file access
2. Validate file types - Check file types before accepting uploads
3. Use random filenames - Prevents accidental overwrites of existing files
4. Store storage keys in database - Keep reference to the storage key, not just the URL
5. Organize with folders - Use logical folder structure (e.g., "documents/2024/january")
6. Set reasonable expiration times - Use appropriate TTL for signed URLs (shorter for sensitive data)
7. Handle errors gracefully - Files might be deleted externally; implement proper error handling
8. Batch operations for efficiency - Use uploadMultiple/deleteMultiple for better performance

Plugin Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | endpoint | string | - | S3 endpoint URL (required for non-AWS) | | region | string | "us-east-1" | AWS region | | accessKeyId | string | - | Access key ID (required) | | secretAccessKey | string | - | Secret access key (required) | | bucket | string | - | Bucket name (required) | | publicUrl | string | - | Public URL base (required) | | forcePathStyle | boolean | true | Use path-style URLs | | acl | string | "private" | Default ACL for uploads |

Testing

See TESTING.md for comprehensive testing guide.

Examples

See EXAMPLES.md for complete real-world examples.

License

ISC