📦 @ashotsiroyan/file-manager

A modular file management system for NestJS supporting Local, AWS S3, and Google Cloud Storage (GCS) backends.
It provides a unified API for uploading, downloading, listing, deleting, and signing files — all with a clean, injectable design.

🚀 Features

• ⚙️ Plug-and-play NestJS module
• 📁 Supports multiple storage engines: Local, S3, GCS
• 🚀 Streaming uploads/downloads (no buffering)
• 🔐 Signed URL generation for direct-to-cloud uploads
• 🧠 Extensible interface (StorageEngine) for custom backends
• 🔒 Security-first design — control access and serve privately

📥 Installation

# Base package
npm install --save @ashotsiroyan/file-manager

# Optional: install for S3
npm install --save @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

# Optional: install for Google Cloud Storage
npm install --save @google-cloud/storage

🧱 Quick Start (Local Storage)

app.module.ts

import { Module } from '@nestjs/common';
import {
  FileManagerModule,
  LocalStorageEngine,
} from '@ashotsiroyan/file-manager';

@Module({
  imports: [
    FileManagerModule.forRoot({
      engine: new LocalStorageEngine({
        baseDir: './uploads',
        publicBaseUrl: 'http://localhost:3000/static',
      }),
      defaultPrefix: 'uploads',
      publicReadByDefault: true,
    }),
  ],
})
export class AppModule {}

forRoot receives a single configuration object:

• engine (required): any StorageEngine
• defaultPrefix / publicReadByDefault (optional service defaults)
• maxConcurrentOps (optional): cap simultaneous storage calls to avoid throttling
• global (optional): register the module globally across your app

main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { NestExpressApplication } from '@nestjs/platform-express';
import { join } from 'path';

async function bootstrap() {
  const app = await NestFactory.create<NestExpressApplication>(AppModule);
  app.useStaticAssets(join(process.cwd(), 'uploads'), { prefix: '/static' });
  await app.listen(3000);
  console.log('🚀 Running at http://localhost:3000');
}

bootstrap();

🧠 Using the Service

import {
  Controller,
  Post,
  Get,
  Delete,
  Param,
  Query,
  UploadedFile,
  UseInterceptors,
  Res,
} from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { Readable } from 'stream';
import { Response } from 'express';
import { FileManagerService } from '@ashotsiroyan/file-manager';

@Controller('avatars')
export class AvatarController {
  constructor(private readonly files: FileManagerService) {}
  
  /**
   * Upload an avatar
   * Works for both memoryStorage (buffer) and diskStorage (stream/path)
   */
  @Post('upload')
  @UseInterceptors(FileInterceptor('file'))
  async upload(@UploadedFile() file: Express.Multer.File) {
    const stream =
      file.stream ??
      Readable.from(file.buffer ?? Buffer.alloc(0));
    
    const res = await this.files.put({
      prefix: 'avatars',
      originalName: file.originalname,
      contentType: file.mimetype,
      body: stream,
    });
    
    return { key: res.key, url: this.files.publicUrl(res.key) };
  }
  
  /**
   * Get (download) an avatar
   */
  @Get(':key(*)')
  async get(@Param('key') key: string, @Res() res: Response) {
    const file = await this.files.get(`avatars/${ key }`);
    
    if (file.contentType) res.setHeader('Content-Type', file.contentType);
    if (file.size) res.setHeader('Content-Length', String(file.size));
    
    file.stream.pipe(res);
  }
  
  /**
   * Remove (delete) an avatar
   */
  @Delete(':key(*)')
  async remove(@Param('key') key: string) {
    await this.files.delete(`avatars/${ key }`);
    return { success: true, deleted: key };
  }
  
  /**
   * List all avatars (optionally filter by prefix)
   */
  @Get()
  async list(@Query('prefix') prefix = 'avatars/') {
    const files = await this.files.list(prefix);
    return { prefix, files };
  }
}

✅ Tip:

For large files, use diskStorage() and fs.createReadStream(file.path) instead of buffers.

For small files, memoryStorage() + Readable.from(file.buffer) is fastest.

Need to remove every file inside a logical folder? Call await this.files.deleteDirectory('avatars/user123/') to wipe an entire prefix in a single operation, regardless of the underlying storage engine.

☁️ Using AWS S3 (forRootAsync)

import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { FileManagerModule, S3StorageEngine } from '@ashotsiroyan/file-manager';

@Module({
  imports: [
    FileManagerModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (config: ConfigService) => ({
        engine: new S3StorageEngine({
          bucket: config.get('S3_BUCKET'),
          region: config.get('AWS_REGION'),
          baseUrlPublic: config.get('S3_PUBLIC_URL'),
          credentials: {
            accessKeyId: config.get('S3_ACCESS_KEY_ID'),
            secretAccessKey: config.get('S3_SECRET_ACCESS_KEY'),
          },
        }),
        defaultPrefix: 'uploads',
        publicReadByDefault: true,
      }),
    }),
  ],
})
export class AppModule {}

When using forRootAsync, ensure the factory returns the same shape as forRoot (an object with at least an engine property, plus any optional defaults or global flag).

🌍 Using Google Cloud Storage (forRootAsync)

import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { FileManagerModule, GcsStorageEngine } from '@ashotsiroyan/file-manager';

@Module({
  imports: [
    FileManagerModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (config: ConfigService) => ({
        engine: new GcsStorageEngine({
          bucket: config.get('GCS_BUCKET'),
          baseUrlPublic: config.get('GCS_PUBLIC_BASE_URL'),
          auth: {
            keyFilename: config.get('GOOGLE_APPLICATION_CREDENTIALS'),
          },
        }),
        defaultPrefix: 'uploads',
        publicReadByDefault: true,
      }),
    }),
  ],
})
export class AppModule {}

The GcsStorageEngine can work with either a key file (GOOGLE_APPLICATION_CREDENTIALS), a JSON blob ( GOOGLE_APPLICATION_CREDENTIALS_JSON, raw or base64), or explicit clientEmail / privateKey pairs.

✅ You can also supply a preconfigured Storage instance (via storage) or additional storageOptions if you need advanced Google Cloud Storage setup.

✅ For AWS you can pass a ready-made S3Client through the client option or inject extra configuration with clientConfig.

🔐 Signed URLs (Direct Uploads)

const url = await this.files.signedUrl({
  key: 'uploads/user123/avatar.png',
  action: 'put',
  expiresInSeconds: 600,
  contentType: 'image/png',
});

console.log('Upload directly to:', url);

For downloads:

await this.files.signedUrl({ key, action: 'get', expiresInSeconds: 300 });

⚠️ Local engine doesn’t produce real signatures — it just returns publicBaseUrl + key.

🎧 Streaming via Backend

@Get('files/:key(*)')
async
download(@Param('key')
key: string, @Res()
res: Response
)
{
  const obj = await this.files.get(key);
  if (obj.contentType) res.setHeader('Content-Type', obj.contentType);
  if (obj.size) res.setHeader('Content-Length', String(obj.size));
  obj.stream.pipe(res);
}

✅ Why stream via backend?

• Authorization & ownership checks
• Metrics & audit logs
• Prevent hotlinking
• Unified CORS and headers
• Range support for media

🧩 Advanced Usage

Multiple storage backends

Import the module once per backend. Give every non-default storage a name and inject it via @InjectFileManager(name). If you omit name, that instance becomes the default FileManagerService.

import { Module } from '@nestjs/common';
import { Readable } from 'stream';
import {
  FileManagerModule,
  FileManagerService,
  InjectFileManager,
  S3StorageEngine,
  GcsStorageEngine,
} from '@ashotsiroyan/file-manager';

@Module({
  imports: [
    FileManagerModule.forRoot({
      name: 'publicUploads',
      engine: new S3StorageEngine({
        bucket: process.env.AWS_PUBLIC_BUCKET!,
      }),
      defaultPrefix: 'public',
      publicReadByDefault: true,
    }),
    FileManagerModule.forRoot({
      name: 'backups',
      engine: new GcsStorageEngine({
        bucket: process.env.GCS_BACKUP_BUCKET!,
      }),
      defaultPrefix: 'backups',
    }),
  ],
})
export class FilesModule {
  constructor(
    @InjectFileManager('publicUploads')
    private readonly publicFiles: FileManagerService,
    @InjectFileManager('backups')
    private readonly backups: FileManagerService,
  ) {}

  async uploadBackup(file: Express.Multer.File) {
    await this.backups.put({
      prefix: 'daily',
      originalName: file.originalname,
      body: file.stream ?? Readable.from(file.buffer ?? Buffer.alloc(0)),
    });
  }
}

Async setup follows the same pattern — specify name on the module options and return the engine from the factory:

@Module({
  imports: [
    FileManagerModule.forRootAsync({
      name: 'backups',
      inject: [ConfigService],
      useFactory: async (config: ConfigService) => ({
        engine: new GcsStorageEngine({
          bucket: config.getOrThrow('GCS_BACKUP_BUCKET'),
        }),
        defaultPrefix: 'backups',
      }),
    }),
  ],
})
export class FilesModule {}

Custom engine

class MyEngine implements StorageEngine {
  async putObject() {
    // ...
  }
  
  async getObject() {
    // ...
  }
  
  async deleteObject() {
    // ...
  }
}

Throttle storage calls

FileManagerModule.forRoot({
  engine: new S3StorageEngine({...}),
  maxConcurrentOps: 5,
});

Setting maxConcurrentOps wraps every service method in a lightweight semaphore, keeping only the specified number of simultaneous storage requests active — helpful for S3/GCS rate limits or local I/O pressure.

⚡ Troubleshooting

| Issue | Cause | Fix | |------------------------------------|--------------------|---------------------------| | Cannot find module '@aws-sdk/... | Missing dependency | Install optional deps | | Forbidden on S3 | Wrong IAM policy | Update bucket permissions | | ENOENT local | Missing folder | Check baseDir | | GCS invalid_grant | Bad credentials | Fix service account JSON |