File Storage

This package provides NestJS integration to interact with various file storage providers.

Supported Providers

• Amazon S3
• Azure Blob Storage

Installation

npm install @wisemen/nestjs-file-storage

Basic Usage

1. Import the FileStorageModule into your NestJS module.

import {
  FileStorageModule,
  FileStorageProvider,
} from "@wisemen/nestjs-file-storage";
import { Module } from "@nestjs/common";
import { ConfigModule, ConfigService } from "@nestjs/config";

@Module({
  imports: [
    FileStorageModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (configService: ConfigService) => ({
        provider: FileStorageProvider.AWS_S3,
        config: {
          bucketName: configService.getOrThrow("S3_BUCKET"),
          region: configService.get("S3_REGION", "nl-ams"),
          accessKeyId: configService.getOrThrow("S3_ACCESS_KEY"),
          secretAccessKey: configService.getOrThrow("S3_SECRET_KEY"),
          endpoint: configService.getOrThrow("S3_ENDPOINT"),
        },
      }),
    }),
  ],
  providers: [MyService],
})
export class SomeModule {}

2. Inject FileStorage into your service to interact with the file storage.

import { Injectable } from "@nestjs/common";
import { FileStorage } from "@wisemen/nestjs-file-storage";
import { Readable } from "stream";
import { pipeline } from "stream/promises";
@Injectable()
export class MyService {
  constructor(private fileStorage: FileStorage) {}
  async upload(key: string, content: Buffer): Promise<void> {
    await this.fileStorage.upload(key, content);
  }

  async uploadStream(key: string, stream: Readable): Promise<void> {
    await this.fileStorage.uploadStream(key, stream);
  }

  async pipelineToStorage(key: string, stream: Readable): Promise<void> {
    await pipeline(stream, this.fileStorage.createUploadWritable(key));
  }
}

Multiple Storages

Pass name during registration when the application needs more than one storage (for example a second S3 bucket for cold storage). Registrations without a name keep using the FileStorage class token; named ones are injected with @InjectFileStorage(name).

import { Injectable, Module } from "@nestjs/common";
import { ConfigModule, ConfigService } from "@nestjs/config";
import {
  FileStorage,
  FileStorageModule,
  FileStorageProvider,
  InjectFileStorage,
} from "@wisemen/nestjs-file-storage";

export enum FS {
  COLD_STORAGE = "cold-storage",
}

@Module({
  imports: [
    FileStorageModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (configService: ConfigService) => ({
        provider: FileStorageProvider.AWS_S3,
        config: { /* default bucket */ },
      }),
    }),
    FileStorageModule.forRootAsync({
      name: FS.COLD_STORAGE,
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (configService: ConfigService) => ({
        provider: FileStorageProvider.AWS_S3,
        config: { /* cold storage bucket */ },
      }),
    }),
  ],
  providers: [ArchiveService],
})
export class SomeModule {}

@Injectable()
export class ArchiveService {
  constructor(
    private readonly fileStorage: FileStorage,
    @InjectFileStorage(FS.COLD_STORAGE)
    private readonly coldStorage: FileStorage,
  ) {}
}

Use getFileStorageToken(name) when you need the raw token, for example to override a named storage with TestFileStorage in tests.

Features

• Create temporary upload/download URLs
• Create temporary preview URLs for inline browser rendering
• Upload file using buffers or streams
• Create a writable upload stream for pipeline(...)-based uploads
• Download file
• Check whether a file exists
• Delete file

URL Types

• createTemporaryPreviewUrl(...): creates a temporary signed read URL intended for inline preview in the browser.
• createTemporaryDownloadUrl(...): creates a temporary signed read URL that forces a file download.
• getPublicUrl(...): returns the public object URL and only works when the file is already publicly accessible.

Options

Required values are positional; everything optional goes in a trailing options object. Each options object has an exported type.

| Method | Options type | Options | | --- | --- | --- | | createTemporaryUploadUrl(key, mimeType, options?) | CreateTemporaryUploadUrlOptions | expiresInSeconds, isPublic, checksum | | createTemporaryPreviewUrl(key, options?) | CreateTemporaryPreviewUrlOptions | mimeType, expiresInSeconds | | createTemporaryDownloadUrl(key, name, options?) | CreateTemporaryDownloadUrlOptions | mimeType, expiresInSeconds | | upload(key, content, options?), uploadStream(key, stream, options?), createUploadWritable(key, options?) | UploadOptions | isPublic | | list(options?) | ListOptions | prefix, startAfter |

Verifying presigned uploads

Pass the file's checksum as checksum to sign it into the upload URL. Storage that verifies checksums rejects any upload whose content does not match, so a truncated or wrong file is never stored.

// Client: hash the file before asking for an upload URL.
const digest = await crypto.subtle.digest("SHA-256", await file.arrayBuffer());
const value = btoa(String.fromCharCode(...new Uint8Array(digest)));

// Server
const url = await this.fileStorage.createTemporaryUploadUrl(key, mimeType, {
  checksum: { algorithm: "sha256", value },
});

The value is always base64-encoded. Supported algorithms:

| Algorithm | Notes | | --- | --- | | sha256 | Built into browsers (crypto.subtle). The default choice. | | sha1 | Built into browsers, but no reason to prefer it over sha256. | | crc32, crc32c, crc64nvme | Much faster on large files; browsers need a library to compute them. | | md5 | Not carried in the URL: the client must also send the value as a Content-MD5 header on the upload. |

This guards against corruption, not against a dishonest client, which can hash whatever it uploads. It is only enforced where the storage verifies checksums (AWS S3, SeaweedFS, MinIO); providers that do not, and Azure Blob Storage SAS URLs, accept the upload regardless.