@softvence/s3

A powerful and flexible NestJS module for uploading files to AWS S3 with built-in caching, automatic folder organization, and hash-based deduplication.

Features

✨ Smart File Upload - Upload single or multiple files to AWS S3 with ease
🗂️ Auto Organization - Automatically organizes files into folders based on MIME type (images, videos, audio, documents)
⚡ Built-in Caching - Optional caching layer using node-cache to prevent duplicate uploads
🔒 Hash-based Deduplication - Uses SHA256 hashing to detect and prevent duplicate file uploads
📦 TypeScript Support - Fully typed with TypeScript for better developer experience
🎯 NestJS Integration - Seamlessly integrates with NestJS applications

Installation

npm install @softvence/s3
# or
yarn add @softvence/s3
# or
pnpm add @softvence/s3

Prerequisites

• Node.js >= 16
• NestJS >= 11.0.0
• AWS S3 bucket with appropriate credentials

Quick Start

1. Import the Module

Import S3Module in your application module and configure it with your AWS credentials:

import { Module } from "@nestjs/common";
import { S3Module } from "@softvence/s3";

@Module({
  imports: [
    S3Module.forRoot({
      region: "us-east-1",
      bucket: "my-bucket-name",
      accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
      cache: {
        isCache: true, // Enable caching
        options: {
          stdTTL: 86400, // Cache TTL in seconds (default: 1 day)
          checkperiod: 120, // Cache check period in seconds
        },
      },
    }),
  ],
})
export class AppModule {}

2. Use in Your Service/Controller

Inject S3Service and start uploading files:

import {
  Controller,
  Post,
  UploadedFile,
  UploadedFiles,
  UseInterceptors,
} from "@nestjs/common";
import { FileInterceptor, FilesInterceptor } from "@nestjs/platform-express";
import { S3Service } from "@softvence/s3";

@Controller("upload")
export class UploadController {
  constructor(private readonly s3Service: S3Service) {}

  @Post("single")
  @UseInterceptors(FileInterceptor("file"))
  async uploadSingle(@UploadedFile() file: Express.Multer.File) {
    const result = await this.s3Service.uploadFile(file);
    return result;
  }

  @Post("multiple")
  @UseInterceptors(FilesInterceptor("files", 10))
  async uploadMultiple(@UploadedFiles() files: Express.Multer.File[]) {
    const results = await this.s3Service.uploadFiles(files);
    return results;
  }
}

Configuration

S3ModuleOptions

| Option | Type | Required | Description | | ----------------- | ---------------- | -------- | ----------------------------------------------- | | region | string | ✅ | AWS region (e.g., 'us-east-1') | | bucket | string | ✅ | S3 bucket name | | accessKeyId | string | ✅ | AWS access key ID | | secretAccessKey | string | ✅ | AWS secret access key | | endpoint | string | ❌ | Custom S3 endpoint (for S3-compatible services) | | cache | S3CacheOptions | ❌ | Cache configuration |

S3CacheOptions

| Option | Type | Default | Description | | --------------------- | --------- | ------- | ----------------------------- | | isCache | boolean | false | Enable/disable caching | | options.stdTTL | number | 86400 | Cache TTL in seconds (1 day) | | options.checkperiod | number | 120 | Cache check period in seconds |

Metadata Options

| Option | Type | Default | Description | | ---------------- | -------- | ---------- | ---------------------------------- | | maxFileSize | number | 20971520 | Maximum file size in bytes (20 MB) | | maxFiles | number | 20 | Maximum number of files to upload | | actualFileName | string | "" | Custom file name (optional) | | fileHash | string | "" | Custom file hash (optional) |

Usage Examples

Basic Single File Upload

import { Injectable } from "@nestjs/common";
import { S3Service } from "@softvence/s3";

@Injectable()
export class FileService {
  constructor(private readonly s3Service: S3Service) {}

  async uploadProfilePicture(file: Express.Multer.File) {
    const result = await this.s3Service.uploadFile(file);

    console.log("File uploaded:", result.url);
    console.log("Bucket:", result.bucket);
    console.log("File size:", result.size);
    console.log("MIME type:", result.mimeType);

    return result;
  }
}

Upload with Custom Metadata

async uploadWithMetadata(file: Express.Multer.File) {
  const result = await this.s3Service.uploadFile(file, {
    maxFileSize: 10 * 1024 * 1024, // 10 MB limit
    actualFileName: 'custom-name',
  });

  return result;
}

Multiple Files Upload

async uploadGallery(files: Express.Multer.File[]) {
  const results = await this.s3Service.uploadFiles(files, {
    maxFiles: 5, // Limit to 5 files
    maxFileSize: 5 * 1024 * 1024, // 5 MB per file
  });

  return results.map(r => ({
    url: r.url,
    originalName: r.originalName,
    cached: r.cached,
  }));
}

Complete Controller Example

import {
  Controller,
  Post,
  UploadedFile,
  UploadedFiles,
  UseInterceptors,
  BadRequestException,
} from "@nestjs/common";
import { FileInterceptor, FilesInterceptor } from "@nestjs/platform-express";
import { S3Service } from "@softvence/s3";

@Controller("files")
export class FilesController {
  constructor(private readonly s3Service: S3Service) {}

  @Post("avatar")
  @UseInterceptors(FileInterceptor("avatar"))
  async uploadAvatar(@UploadedFile() file: Express.Multer.File) {
    if (!file) {
      throw new BadRequestException("No file uploaded");
    }

    const result = await this.s3Service.uploadFile(file, {
      maxFileSize: 2 * 1024 * 1024, // 2 MB
    });

    return {
      message: "Avatar uploaded successfully",
      url: result.url,
      cached: result.cached,
    };
  }

  @Post("documents")
  @UseInterceptors(FilesInterceptor("documents", 10))
  async uploadDocuments(@UploadedFiles() files: Express.Multer.File[]) {
    const results = await this.s3Service.uploadFiles(files, {
      maxFiles: 10,
      maxFileSize: 10 * 1024 * 1024, // 10 MB per file
    });

    return {
      message: `${results.length} documents uploaded successfully`,
      files: results.map((r) => ({
        url: r.url,
        name: r.originalName,
        size: r.size,
        folder: r.folder,
      })),
    };
  }
}

Using Environment Variables

Create a .env file:

AWS_REGION=us-east-1
AWS_BUCKET=my-s3-bucket
AWS_ACCESS_KEY_ID=your-access-key-id
AWS_SECRET_ACCESS_KEY=your-secret-access-key

Configure the module:

import { Module } from "@nestjs/common";
import { ConfigModule, ConfigService } from "@nestjs/config";
import { S3Module } from "@softvence/s3";

@Module({
  imports: [
    ConfigModule.forRoot(),
    S3Module.forRoot({
      region: process.env.AWS_REGION!,
      bucket: process.env.AWS_BUCKET!,
      accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
      cache: {
        isCache: true,
      },
    }),
  ],
})
export class AppModule {}

Async Configuration

For dynamic configuration using ConfigService:

import { Module } from "@nestjs/common";
import { ConfigModule, ConfigService } from "@nestjs/config";
import { S3Module } from "@softvence/s3";

@Module({
  imports: [
    ConfigModule.forRoot(),
    {
      module: S3Module,
      imports: [ConfigModule],
      useFactory: (configService: ConfigService) => ({
        region: configService.get("AWS_REGION")!,
        bucket: configService.get("AWS_BUCKET")!,
        accessKeyId: configService.get("AWS_ACCESS_KEY_ID")!,
        secretAccessKey: configService.get("AWS_SECRET_ACCESS_KEY")!,
        cache: {
          isCache: true,
          options: {
            stdTTL: 86400,
          },
        },
      }),
      inject: [ConfigService],
    },
  ],
})
export class AppModule {}

API Reference

S3Service

uploadFile(file: Express.Multer.File, metadata?: Metadata): Promise<S3Response>

Uploads a single file to S3.

Parameters:

• file - Multer file object
• metadata - Optional metadata configuration

Returns: Promise resolving to S3Response

Throws: BadRequestException if upload fails

uploadFiles(files: Express.Multer.File[], metadata?: Metadata): Promise<S3Response[]>

Uploads multiple files to S3.

Parameters:

• files - Array of Multer file objects
• metadata - Optional metadata configuration

Returns: Promise resolving to array of S3Response

Throws:

• BadRequestException if no files provided
• BadRequestException if file count exceeds maxFiles

S3Response Type

type S3Response = {
  url: string; // Full S3 URL
  bucket: string; // Bucket name
  region: string; // AWS region
  originalName: string; // Original file name
  size: number; // File size in bytes
  mimeType: string; // MIME type
  extension: string; // File extension
  folder: string; // Auto-assigned folder (images/videos/audio/documents)
  hash?: string; // SHA256 hash (for cached/small files)
  cached?: boolean; // Whether file was returned from cache
  cacheKey?: string; // Cache key if cached
  uploadedAt: Date; // Upload timestamp
  metadata?: Metadata; // User-provided metadata
};

Folder Organization

Files are automatically organized into folders based on their MIME type:

| MIME Type Pattern | Folder | | ----------------- | ------------ | | image/* | images/ | | video/* | videos/ | | audio/* | audio/ | | All others | documents/ |

Example URLs:

• Image: https://my-bucket.s3.us-east-1.amazonaws.com/images/abc123.jpg
• Video: https://my-bucket.s3.us-east-1.amazonaws.com/videos/xyz789.mp4
• PDF: https://my-bucket.s3.us-east-1.amazonaws.com/documents/doc456.pdf

Caching Behavior

When caching is enabled:

1. Small files (< maxFileSize) are hashed using SHA256
2. Hash is checked against the cache before uploading
3. If found in cache, the cached URL is returned immediately (no S3 upload)
4. If not found, file is uploaded and cached for future requests
5. Cache TTL determines how long the entry stays in cache

Benefits:

• Prevents duplicate uploads
• Reduces S3 costs
• Faster response times for duplicate files

Best Practices

1. Use Environment Variables

Never hardcode AWS credentials. Always use environment variables or a secure configuration service.

S3Module.forRoot({
  region: process.env.AWS_REGION!,
  bucket: process.env.AWS_BUCKET!,
  accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
});

2. Enable Caching for Performance

Enable caching to improve performance and reduce costs:

cache: {
  isCache: true,
  options: {
    stdTTL: 86400, // 1 day
  },
}

3. Set Appropriate File Size Limits

Protect your application from large file uploads:

await this.s3Service.uploadFile(file, {
  maxFileSize: 5 * 1024 * 1024, // 5 MB
});

4. Validate File Types

Use NestJS pipes to validate file types before upload:

import { ParseFilePipe, MaxFileSizeValidator, FileTypeValidator } from '@nestjs/common';

@Post('upload')
@UseInterceptors(FileInterceptor('file'))
async upload(
  @UploadedFile(
    new ParseFilePipe({
      validators: [
        new MaxFileSizeValidator({ maxSize: 5 * 1024 * 1024 }),
        new FileTypeValidator({ fileType: 'image/*' }),
      ],
    }),
  )
  file: Express.Multer.File,
) {
  return this.s3Service.uploadFile(file);
}

5. Handle Errors Gracefully

Always wrap uploads in try-catch blocks:

try {
  const result = await this.s3Service.uploadFile(file);
  return { success: true, url: result.url };
} catch (error) {
  console.error("Upload failed:", error);
  throw new BadRequestException("Failed to upload file");
}

6. Configure S3 Bucket CORS

If accessing files from a browser, configure CORS on your S3 bucket:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": []
  }
]

7. Use IAM Policies

Grant minimal required permissions to your AWS user/role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject"],
      "Resource": "arn:aws:s3:::your-bucket-name/*"
    }
  ]
}

Error Handling

The service throws BadRequestException in the following cases:

• No files uploaded
• File count exceeds maxFiles limit
• S3 upload fails

Example:

import { BadRequestException } from "@nestjs/common";

try {
  const result = await this.s3Service.uploadFile(file);
} catch (error) {
  if (error instanceof BadRequestException) {
    console.error("Upload error:", error.message);
  }
  throw error;
}

TypeScript Support

All types are exported and can be imported:

import {
  S3Service,
  S3ModuleOptions,
  S3Response,
  Metadata,
  S3CacheOptions,
} from "@softvence/s3";

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT © Sabbir Hossain Shuvo

Repository

• GitHub: softvence-agency/softvence
• Issues: Report a bug

Support

For questions and support, please open an issue on GitHub.

Made with ❤️ by the Softvence Team