express-s3-helper

v1.0.0

Published

8 days ago

Production-ready Node.js + Express package for easy Amazon S3 integration with pre-signed URLs, file management, and Express route templates

0High
0Medium
0Low

adilsagheer

express s3 aws amazon upload download presigned-url file-storage cloud-storage nodejs backend

Express S3 Helper

A production-ready Node.js + Express package for easy Amazon S3 integration. Generate pre-signed URLs, manage files, and integrate S3 into your Express app with minimal configuration.

Features

🚀 Easy Setup - Just add AWS credentials and start using
🔐 Pre-signed URLs - Secure upload and download URLs
📁 File Management - Upload, download, delete, copy, and list files
✅ Built-in Validation - File type and size validation
🛣️ Express Routes - Ready-to-use route templates
🛡️ Error Handling - Comprehensive error classes and middleware
📦 AWS SDK v3 - Modern, modular AWS SDK
🔧 TypeScript Friendly - ES Modules with JSDoc annotations
🎯 Zero Boilerplate - Works out of the box

Installation

npm install express-s3-helper

Or with yarn:

yarn add express-s3-helper

Quick Start

Install the package

npm install express-s3-helper

Create a .env file in your project root:

AWS_ACCESS_KEY_ID=your_access_key_here
AWS_SECRET_ACCESS_KEY=your_secret_key_here
S3_BUCKET=your_bucket_name
S3_REGION=us-east-1

Use in your Express app

import express from 'express';
import { createS3Routes, s3ErrorHandler } from 'express-s3-helper';

const app = express();
app.use(express.json());

// Mount S3 routes
app.use('/api/s3', createS3Routes());

// Error handler (place after routes)
app.use(s3ErrorHandler());

app.listen(3000, () => {
  console.log('Server running on port 3000');
});

That's it! You now have fully functional S3 endpoints.

Environment Setup

Create a .env file in your project root with the following variables:

# Required
AWS_ACCESS_KEY_ID=your_access_key_here
AWS_SECRET_ACCESS_KEY=your_secret_key_here
S3_BUCKET=your_bucket_name

# Optional (defaults to us-east-1)
S3_REGION=us-east-1

Getting AWS Credentials

Log in to the AWS Console
Go to IAM → Users → Add User
Create a user with programmatic access
Attach the AmazonS3FullAccess policy (or create a custom policy)
Copy the Access Key ID and Secret Access Key

S3 Bucket CORS Configuration

For browser uploads, configure CORS on your S3 bucket:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"],
    "AllowedOrigins": ["http://localhost:3000", "https://yourdomain.com"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }
]

Usage Examples

Basic Usage

import { S3Service, getS3Service } from 'express-s3-helper';

// Using the singleton instance (recommended)
const s3 = getS3Service();

// Or create a custom instance
const customS3 = new S3Service({
  accessKeyId: 'your_key',
  secretAccessKey: 'your_secret',
  bucket: 'your_bucket',
  region: 'us-west-2',
});

Generate Upload URL

Generate a pre-signed URL for uploading files directly to S3:

import { getS3Service } from 'express-s3-helper';

const s3 = getS3Service();

// Basic upload URL
const result = await s3.getUploadUrl({
  fileName: 'photo.jpg',
  contentType: 'image/jpeg',
});

console.log(result);
// {
//   uploadUrl: 'https://bucket.s3.region.amazonaws.com/...',
//   key: '1701234567890-abc123.jpg',
//   bucket: 'your-bucket',
//   region: 'us-east-1',
//   contentType: 'image/jpeg',
//   expiresIn: 900,
//   expiresAt: '2024-01-01T00:15:00.000Z'
// }

// Advanced upload URL with options
const advancedResult = await s3.getUploadUrl({
  fileName: 'document.pdf',
  contentType: 'application/pdf',
  contentLength: 1024000, // 1MB - for size validation
  folder: 'documents/invoices',
  prefix: 'user-123',
  preserveOriginalName: true,
  expiresIn: 3600, // 1 hour
  metadata: {
    uploadedBy: 'user-123',
    category: 'invoices',
  },
});

// Result key: 'documents/invoices/user-123-document-1701234567890-abc123.pdf'

Generate Download URL

Generate a pre-signed URL for downloading files:

import { getS3Service } from 'express-s3-helper';

const s3 = getS3Service();

// Basic download URL
const result = await s3.getDownloadUrl('documents/report.pdf');

console.log(result);
// {
//   downloadUrl: 'https://bucket.s3.region.amazonaws.com/...',
//   key: 'documents/report.pdf',
//   bucket: 'your-bucket',
//   expiresIn: 900,
//   expiresAt: '2024-01-01T00:15:00.000Z'
// }

// Force download with custom filename
const downloadResult = await s3.getDownloadUrl('documents/report.pdf', {
  expiresIn: 3600,
  responseContentDisposition: 'attachment; filename="Monthly-Report.pdf"',
});

Delete Files

import { getS3Service } from 'express-s3-helper';

const s3 = getS3Service();

// Delete single file
const result = await s3.deleteFile('uploads/old-file.jpg');
console.log(result);
// { success: true, key: 'uploads/old-file.jpg', bucket: '...', deletedAt: '...' }

// Delete multiple files
const bulkResult = await s3.deleteFiles([
  'uploads/file1.jpg',
  'uploads/file2.jpg',
  'uploads/file3.jpg',
]);
console.log(bulkResult);
// {
//   success: true,
//   deleted: ['uploads/file1.jpg', 'uploads/file2.jpg', 'uploads/file3.jpg'],
//   errors: [],
//   bucket: '...',
//   deletedAt: '...'
// }

List Files

import { getS3Service } from 'express-s3-helper';

const s3 = getS3Service();

// List all files in a folder
const result = await s3.listFiles({
  prefix: 'uploads/user-123/',
  maxKeys: 100,
});

console.log(result);
// {
//   files: [
//     { key: 'uploads/user-123/file1.jpg', size: 1024, lastModified: '...', etag: '...' },
//     { key: 'uploads/user-123/file2.jpg', size: 2048, lastModified: '...', etag: '...' },
//   ],
//   prefix: 'uploads/user-123/',
//   bucket: '...',
//   isTruncated: false,
//   keyCount: 2
// }

// Pagination
const page2 = await s3.listFiles({
  prefix: 'uploads/',
  maxKeys: 100,
  continuationToken: result.nextContinuationToken,
});

File Metadata

import { getS3Service } from 'express-s3-helper';

const s3 = getS3Service();

// Check if file exists
const exists = await s3.fileExists('uploads/photo.jpg');
console.log(exists); // true or false

// Get file metadata
const metadata = await s3.getFileMetadata('uploads/photo.jpg');
console.log(metadata);
// {
//   key: 'uploads/photo.jpg',
//   bucket: '...',
//   contentType: 'image/jpeg',
//   contentLength: 102400,
//   lastModified: '2024-01-01T00:00:00.000Z',
//   etag: '"abc123..."',
//   metadata: { uploadedBy: 'user-123' }
// }

// Copy file
const copyResult = await s3.copyFile(
  'uploads/original.jpg',
  'backups/original-backup.jpg'
);

// Get public URL (for public buckets)
const publicUrl = s3.getPublicUrl('uploads/photo.jpg');
// https://bucket.s3.region.amazonaws.com/uploads/photo.jpg

Express Routes

The package includes ready-to-use Express routes:

Basic Route Setup

import express from 'express';
import { createS3Routes, s3ErrorHandler } from 'express-s3-helper';

const app = express();
app.use(express.json());

// Mount routes at /api/s3
app.use('/api/s3', createS3Routes());

// Add error handler
app.use(s3ErrorHandler());

app.listen(3000);

Route Configuration Options

import { createS3Routes } from 'express-s3-helper';

const s3Routes = createS3Routes({
  // Restrict allowed file types
  allowedMimeTypes: ['image/jpeg', 'image/png', 'application/pdf'],

  // Limit file size (10MB)
  maxFileSize: 10 * 1024 * 1024,

  // URL expiration times
  uploadUrlExpiration: 900, // 15 minutes
  downloadUrlExpiration: 3600, // 1 hour

  // Add authentication middleware
  authMiddleware: requireAuth,

  // Enable/disable endpoints
  enableList: true,
  enableDelete: true,
  enableMetadata: true,
});

app.use('/api/s3', s3Routes);

Available Endpoints

| Method | Endpoint | Description | |--------|----------|-------------| | POST | /upload | Generate upload URL | | GET | /download/:key | Generate download URL | | GET | /download?key= | Generate download URL (query param) | | DELETE | /file/:key | Delete a single file | | DELETE | /files | Delete multiple files | | GET | /files | List files | | GET | /metadata/:key | Get file metadata | | HEAD | /file/:key | Check if file exists |

Route Usage Examples

Frontend - Get Upload URL and Upload File:

// 1. Get upload URL from your backend
const response = await fetch('/api/s3/upload', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    fileName: file.name,
    contentType: file.type,
    contentLength: file.size,
    folder: 'uploads',
  }),
});

const { data } = await response.json();

// 2. Upload directly to S3 using the pre-signed URL
await fetch(data.uploadUrl, {
  method: 'PUT',
  headers: { 'Content-Type': file.type },
  body: file,
});

// 3. Save the key for later reference
console.log('File uploaded:', data.key);

Frontend - Download File:

// Get download URL
const response = await fetch(`/api/s3/download/${encodeURIComponent(fileKey)}`);
const { data } = await response.json();

// Redirect to download
window.location.href = data.downloadUrl;

// Or open in new tab
window.open(data.downloadUrl, '_blank');

Helper Functions

File Utilities

import {
  generateFileName,
  getFileExtension,
  getBaseName,
  sanitizeFileName,
  getMimeType,
  validateFile,
  validateFileKey,
  formatFileSize,
} from 'express-s3-helper';

// Generate unique filename
const fileName = generateFileName('photo.jpg', {
  folder: 'uploads',
  prefix: 'user-123',
  preserveOriginalName: true,
});
// Result: 'uploads/user-123-photo-1701234567890-abc123.jpg'

// Get file extension
getFileExtension('document.pdf'); // '.pdf'

// Get base name
getBaseName('path/to/file.txt'); // 'file'

// Sanitize filename
sanitizeFileName('My File (1).jpg'); // 'my-file-1.jpg'

// Get MIME type
getMimeType('photo.jpg'); // 'image/jpeg'

// Validate file
validateFile({
  name: 'photo.jpg',
  mimeType: 'image/jpeg',
  size: 1024000,
}, {
  allowedMimeTypes: ['image/jpeg', 'image/png'],
  maxFileSize: 5 * 1024 * 1024,
}); // Returns true or throws S3ValidationError

// Format file size
formatFileSize(1536000); // '1.46 MB'

Default Values

import {
  DEFAULT_ALLOWED_MIME_TYPES,
  DEFAULT_MAX_FILE_SIZE,
  DEFAULT_URL_EXPIRATION,
} from 'express-s3-helper';

console.log(DEFAULT_ALLOWED_MIME_TYPES);
// ['image/jpeg', 'image/png', 'image/gif', ...]

console.log(DEFAULT_MAX_FILE_SIZE);
// 52428800 (50MB)

console.log(DEFAULT_URL_EXPIRATION);
// 900 (15 minutes)

Middleware

Error Handler

import { s3ErrorHandler } from 'express-s3-helper';

// Basic usage
app.use(s3ErrorHandler());

// With options
app.use(s3ErrorHandler({
  logErrors: true,
  logger: console.error,
  includeStack: process.env.NODE_ENV !== 'production',
}));

Async Handler

import { asyncHandler } from 'express-s3-helper';

// Wrap async routes to catch errors
router.get('/files/:key', asyncHandler(async (req, res) => {
  const s3 = getS3Service();
  const url = await s3.getDownloadUrl(req.params.key);
  res.json(url);
}));

Request Validators

import {
  validateUploadRequest,
  validateDownloadRequest,
  validateDeleteRequest,
} from 'express-s3-helper';

// Validate upload requests
router.post('/upload',
  validateUploadRequest({
    allowedMimeTypes: ['image/jpeg'],
    maxFileSize: 5 * 1024 * 1024,
  }),
  asyncHandler(async (req, res) => {
    // req.body is validated
  })
);

// Validate download requests
router.get('/download/:key',
  validateDownloadRequest(),
  asyncHandler(async (req, res) => {
    // req.s3Key contains the validated key
  })
);

// Validate delete requests
router.delete('/file/:key',
  validateDeleteRequest(),
  asyncHandler(async (req, res) => {
    // req.s3Key or req.s3Keys contains validated keys
  })
);

Error Handling

The package provides specific error classes for different scenarios:

import {
  S3Error,
  S3ConfigurationError,
  S3UploadError,
  S3DownloadError,
  S3DeleteError,
  S3ValidationError,
  S3NotFoundError,
  S3AccessDeniedError,
} from 'express-s3-helper';

try {
  await s3.getDownloadUrl('nonexistent-file.jpg');
} catch (error) {
  if (error instanceof S3NotFoundError) {
    console.log('File not found');
  } else if (error instanceof S3ValidationError) {
    console.log('Validation failed:', error.message);
  } else if (error instanceof S3AccessDeniedError) {
    console.log('Access denied');
  } else if (error instanceof S3Error) {
    console.log('S3 error:', error.code, error.message);
  }
}

Error Properties

All S3 errors have these properties:

name - Error class name
message - Human-readable message
code - Error code (e.g., 'NOT_FOUND', 'VALIDATION_ERROR')
originalError - Original AWS SDK error (if any)
stack - Stack trace

Configuration

Using Environment Variables (Recommended)

// .env file
AWS_ACCESS_KEY_ID=your_key
AWS_SECRET_ACCESS_KEY=your_secret
S3_BUCKET=your_bucket
S3_REGION=us-east-1

// Your code - configuration is automatic
import { getS3Service } from 'express-s3-helper';
const s3 = getS3Service();

Custom Configuration

import { S3Service, createS3Config } from 'express-s3-helper';

// Create custom instance
const s3 = new S3Service({
  accessKeyId: 'your_key',
  secretAccessKey: 'your_secret',
  bucket: 'your_bucket',
  region: 'eu-west-1',
});

// Or create config separately
const config = createS3Config({
  accessKeyId: 'your_key',
  secretAccessKey: 'your_secret',
  bucket: 'your_bucket',
  region: 'eu-west-1',
});

Multiple Buckets

import { S3Service } from 'express-s3-helper';

const publicBucket = new S3Service({
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  bucket: 'my-public-bucket',
  region: 'us-east-1',
});

const privateBucket = new S3Service({
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  bucket: 'my-private-bucket',
  region: 'us-east-1',
});

Best Practices

1. Security

// Always validate file types on upload
const result = await s3.getUploadUrl({
  fileName: 'photo.jpg',
  contentType: 'image/jpeg',
  validate: true,
  allowedMimeTypes: ['image/jpeg', 'image/png'],
  maxFileSize: 5 * 1024 * 1024,
});

// Use short expiration times for URLs
const downloadUrl = await s3.getDownloadUrl(key, {
  expiresIn: 300, // 5 minutes
});

// Add authentication to routes
app.use('/api/s3', requireAuth, createS3Routes());

2. File Organization

// Use folders for organization
const result = await s3.getUploadUrl({
  fileName: 'avatar.jpg',
  contentType: 'image/jpeg',
  folder: `users/${userId}/avatars`,
});

// Use prefixes for categorization
const result = await s3.getUploadUrl({
  fileName: 'document.pdf',
  contentType: 'application/pdf',
  folder: 'documents',
  prefix: `${year}-${month}`,
});

3. Error Handling

import { S3NotFoundError, S3ValidationError } from 'express-s3-helper';

try {
  const url = await s3.getDownloadUrl(key);
  res.json({ success: true, url });
} catch (error) {
  if (error instanceof S3NotFoundError) {
    res.status(404).json({ error: 'File not found' });
  } else if (error instanceof S3ValidationError) {
    res.status(400).json({ error: error.message });
  } else {
    res.status(500).json({ error: 'Internal server error' });
  }
}

4. Cleanup Old Files

// List and delete old files
const files = await s3.listFiles({ prefix: 'temp/' });
const oldFiles = files.files.filter(f => {
  const age = Date.now() - new Date(f.lastModified).getTime();
  return age > 24 * 60 * 60 * 1000; // Older than 24 hours
});

if (oldFiles.length > 0) {
  await s3.deleteFiles(oldFiles.map(f => f.key));
}

API Reference

S3Service Class

| Method | Description | Returns | |--------|-------------|---------| | getUploadUrl(options) | Generate pre-signed upload URL | Promise<UploadUrlResult> | | getDownloadUrl(key, options) | Generate pre-signed download URL | Promise<DownloadUrlResult> | | getUploadUrlForFile(fileName, options) | Auto-detect content type and generate upload URL | Promise<UploadUrlResult> | | deleteFile(key) | Delete a single file | Promise<DeleteResult> | | deleteFiles(keys) | Delete multiple files | Promise<BulkDeleteResult> | | fileExists(key) | Check if file exists | Promise<boolean> | | getFileMetadata(key) | Get file metadata | Promise<MetadataResult> | | listFiles(options) | List files in a folder | Promise<ListResult> | | copyFile(sourceKey, destKey) | Copy a file | Promise<CopyResult> | | getPublicUrl(key) | Get public URL (for public buckets) | string |

Utility Functions

| Function | Description | |----------|-------------| | generateFileName(name, options) | Generate unique filename | | getFileExtension(fileName) | Get file extension | | getBaseName(fileName) | Get filename without extension | | sanitizeFileName(fileName) | Sanitize filename | | getMimeType(fileName) | Get MIME type from filename | | getExtensionFromMimeType(mimeType) | Get extension from MIME type | | validateFile(file, options) | Validate file | | validateFileKey(key) | Validate S3 key | | formatFileSize(bytes) | Format bytes to human readable |

Complete Example

Here's a complete Express application using this package:

import express from 'express';
import cors from 'cors';
import {
  createS3Routes,
  s3ErrorHandler,
  getS3Service,
  asyncHandler,
} from 'express-s3-helper';

const app = express();

// Middleware
app.use(cors());
app.use(express.json());

// Mount S3 routes
app.use('/api/s3', createS3Routes({
  allowedMimeTypes: ['image/jpeg', 'image/png', 'application/pdf'],
  maxFileSize: 10 * 1024 * 1024,
}));

// Custom route example
app.post('/api/upload-avatar', asyncHandler(async (req, res) => {
  const s3 = getS3Service();
  const { userId, fileName, contentType } = req.body;

  const result = await s3.getUploadUrl({
    fileName,
    contentType,
    folder: `users/${userId}/avatars`,
    allowedMimeTypes: ['image/jpeg', 'image/png'],
    maxFileSize: 2 * 1024 * 1024,
  });

  res.json({ success: true, data: result });
}));

// Error handler
app.use(s3ErrorHandler({ logErrors: true }));

// Generic error handler
app.use((err, req, res, next) => {
  console.error(err);
  res.status(500).json({ error: 'Internal server error' });
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});