@devoven/storage

Multi-cloud storage abstraction for NestJS. S3, GCS, and local filesystem behind a single port interface.

Installation

npm install @devoven/storage
# or
pnpm add @devoven/storage

Peer Dependencies

Standard NestJS peer dependencies (@nestjs/common, @nestjs/core, rxjs, reflect-metadata) must be present in your project. Any NestJS application already has these.

Install the SDK for whichever provider(s) you use. All cloud SDKs are optional peer dependencies:

# Amazon S3
npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

# Google Cloud Storage
npm install @google-cloud/storage

LocalStorageProvider has no extra peer dependencies — no SDK install needed.

Quick Start

import { StorageModule, S3StorageProvider } from '@devoven/storage';

@Module({
  imports: [
    StorageModule.register({
      provider: new S3StorageProvider({
        bucket: 'my-bucket',
        region: 'us-east-1',
      }),
    }),
  ],
})
export class AppModule {}

Module Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | provider | StorageProviderPort instance | — | Required. The storage backend to use | | defaultSignedUrlTtl | number | 3600 | Default TTL in seconds for pre-signed URLs | | global | boolean | false | Register the module globally |

provider must be a pre-constructed instance (not a class). Instantiate the provider in your module or factory and pass the result.

Async Registration

Use registerAsync when the provider configuration comes from a config service:

import { StorageModule, S3StorageProvider } from '@devoven/storage';
import { ConfigService } from '@nestjs/config';

@Module({
  imports: [
    StorageModule.registerAsync({
      useFactory: (config: ConfigService) => ({
        provider: new S3StorageProvider({
          bucket: config.get('S3_BUCKET'),
          region: config.get('S3_REGION'),
          credentials: {
            accessKeyId: config.get('AWS_ACCESS_KEY_ID'),
            secretAccessKey: config.get('AWS_SECRET_ACCESS_KEY'),
          },
        }),
        defaultSignedUrlTtl: config.get('SIGNED_URL_TTL'),
      }),
      inject: [ConfigService],
    }),
  ],
})
export class AppModule {}

Injecting the Provider

The module exports two tokens from TOKENS (both are Symbols):

| Token | Type | Description | |-------|------|-------------| | TOKENS.StorageProviderPort | StorageProviderPort | The configured storage backend | | TOKENS.DefaultSignedUrlTtl | number | The default TTL for signed URLs |

import { Inject, Injectable } from '@nestjs/common';
import { TOKENS, StorageProviderPort } from '@devoven/storage';
import { Readable } from 'stream';

@Injectable()
export class FileService {
  constructor(
    @Inject(TOKENS.StorageProviderPort)
    private readonly storage: StorageProviderPort,
    @Inject(TOKENS.DefaultSignedUrlTtl)
    private readonly defaultTtl: number,
  ) {}

  async uploadFile(key: string, buffer: Buffer, contentType: string) {
    return this.storage.upload(key, buffer, { contentType });
  }

  async getDownloadUrl(key: string): Promise<string> {
    return this.storage.getSignedDownloadUrl(key, this.defaultTtl);
  }
}

StorageProviderPort Interface

All provider operations are available through the single StorageProviderPort interface.

Basic operations

| Method | Signature | Description | |--------|-----------|-------------| | upload | (key, body, options?) => Promise<{ key }> | Upload a file. If the key already exists the provider appends a datetime suffix to avoid overwriting | | download | (key) => Promise<{ body: Readable, metadata: StorageObjectValueObject }> | Download as a stream with metadata | | delete | (key) => Promise<void> | Delete a single object | | deleteMany | (keys) => Promise<void> | Delete multiple objects in one call | | list | (prefix, options?) => Promise<ListResult> | List objects under a prefix with optional pagination |

Pre-signed URLs

| Method | Signature | Description | |--------|-----------|-------------| | getSignedUploadUrl | (key, expiresInSeconds, options?) => Promise<{ key, url }> | Generate a URL for client-side upload | | getSignedDownloadUrl | (key, expiresInSeconds) => Promise<string> | Generate a URL for direct client download |

Multipart uploads

Use multipart uploads for large files (typically >5 MB).

| Method | Signature | Description | |--------|-----------|-------------| | initiateMultipartUpload | (key, options?) => Promise<{ uploadId, key }> | Start a multipart upload session | | uploadPart | (key, uploadId, partNumber, body) => Promise<UploadedPart> | Upload one part | | completeMultipartUpload | (key, uploadId, parts) => Promise<void> | Finalise the upload | | abortMultipartUpload | (key, uploadId) => Promise<void> | Discard an incomplete multipart upload |

Multipart upload example

const PART_SIZE = 5 * 1024 * 1024; // 5 MB minimum per part (S3 requirement)

// 1. Start the session
const { uploadId, key } = await storage.initiateMultipartUpload('videos/demo.mp4', {
  contentType: 'video/mp4',
});

// 2. Upload each part (parts are 1-indexed)
const parts: UploadedPart[] = [];
for (let i = 0; i < chunks.length; i++) {
  const part = await storage.uploadPart(key, uploadId, i + 1, chunks[i]);
  parts.push(part);
}

// 3. Finalise — provider assembles the parts into a single object
await storage.completeMultipartUpload(key, uploadId, parts);

If an error occurs mid-upload, call abortMultipartUpload(key, uploadId) to release the incomplete upload and avoid storage charges.

UploadOptions

interface UploadOptions {
  contentType?: string;          // MIME type to store with the object
  cacheControl?: string;         // Cache-Control header value
  metadata?: Record<string, string>; // Custom key/value metadata
}

ListOptions

interface ListOptions {
  maxKeys?: number;              // Maximum results to return
  continuationToken?: string;    // Resume a paginated listing
}

ListResult

interface ListResult {
  objects: StorageObjectValueObject[]; // Objects found
  isTruncated?: boolean;               // Whether more results exist
  nextContinuationToken?: string;      // Pass to the next list call to continue
}

Built-in Providers

S3StorageProvider

Backed by @aws-sdk/client-s3. Supports all StorageProviderPort operations.

new S3StorageProvider({
  bucket: 'my-bucket',
  region: 'us-east-1',

  // Optional — defaults to AWS environment credentials
  credentials: {
    accessKeyId: 'AKIA...',
    secretAccessKey: '...',
  },

  // Optional — custom endpoint for MinIO or other S3-compatible stores
  endpoint: 'http://localhost:9000',
  forcePathStyle: true,
})

| Config field | Type | Default | Description | |---|---|---|---| | bucket | string | — | Required | | region | string | — | Required | | endpoint | string | AWS default | Custom endpoint URL | | credentials | { accessKeyId, secretAccessKey } | Environment | Explicit credentials | | forcePathStyle | boolean | false | Use path-style URLs (required for MinIO) |

GCSStorageProvider

Backed by @google-cloud/storage. Supports all StorageProviderPort operations.

new GCSStorageProvider({
  bucket: 'my-bucket',

  // Option A: inline credentials
  credentials: {
    clientEmail: '[email protected]',
    privateKey: '-----BEGIN PRIVATE KEY-----\n...',
  },

  // Option B: path to a service account JSON file
  keyFilename: '/path/to/service-account.json',

  projectId: 'my-gcp-project',
})

| Config field | Type | Default | Description | |---|---|---|---| | bucket | string | — | Required | | credentials | { clientEmail, privateKey } | ADC | Inline service account credentials | | projectId | string | From credentials | GCP project ID | | keyFilename | string | — | Path to a service account JSON key file |

LocalStorageProvider

Stores files on the local filesystem. Intended for development and testing. Pre-signed URLs are not supported — calls to getSignedUploadUrl and getSignedDownloadUrl throw UnsupportedOperation.

new LocalStorageProvider({
  basePath: './uploads',
})

| Config field | Type | Description | |---|---|---| | basePath | string | Required. Directory where files will be stored |

StorageObjectValueObject

Returned by download and list. Immutable value object.

| Property | Type | Description | |----------|------|-------------| | key | string | Object key (path within the bucket) | | extension | string \| undefined | File extension parsed from the key | | size | number \| undefined | File size in bytes | | contentType | string \| undefined | MIME type | | lastModified | Date \| undefined | Last modification timestamp | | etag | string \| undefined | Entity tag |

Created via StorageObjectValueObject.create({ key, size?, contentType?, lastModified?, etag? }).

Custom Adapters

Implement StorageProviderPort to add your own backend:

import { StorageProviderPort, UploadOptions, ListOptions, ListResult, UploadedPart } from '@devoven/storage';
import { StorageObjectValueObject } from '@devoven/storage';
import { Readable } from 'stream';

export class AzureBlobStorageProvider implements StorageProviderPort {
  async upload(key: string, body: Buffer | Readable, options?: UploadOptions) {
    // upload to Azure Blob Storage...
    return { key };
  }

  async download(key: string) {
    // download from Azure...
    return { body: readable, metadata: StorageObjectValueObject.create({ key }) };
  }

  async delete(key: string): Promise<void> { /* ... */ }
  async deleteMany(keys: string[]): Promise<void> { /* ... */ }
  async list(prefix: string, options?: ListOptions): Promise<ListResult> { /* ... */ }

  async getSignedUploadUrl(key: string, expiresInSeconds: number, options?: UploadOptions) {
    return { key, url: 'https://...' };
  }

  async getSignedDownloadUrl(key: string, expiresInSeconds: number): Promise<string> {
    return 'https://...';
  }

  async initiateMultipartUpload(key: string, options?: UploadOptions) {
    return { uploadId: '...', key };
  }

  async uploadPart(key: string, uploadId: string, partNumber: number, body: Buffer): Promise<UploadedPart> {
    return { partNumber, etag: '...' };
  }

  async completeMultipartUpload(key: string, uploadId: string, parts: UploadedPart[]): Promise<void> { /* ... */ }
  async abortMultipartUpload(key: string, uploadId: string): Promise<void> { /* ... */ }
}

Pass the instance as the provider option when registering the module:

StorageModule.register({
  provider: new AzureBlobStorageProvider(connectionString),
})

If a particular operation is not supported by your backend, throw UnsupportedOperation (exported from @devoven/storage) to return a 501 Not Implemented HTTP response.

Error Handling

UnsupportedOperation

UnsupportedOperation is an HttpException that maps to HTTP 501 Not Implemented. It is thrown when a storage operation is called on a provider that does not support it.

The built-in case is LocalStorageProvider: calling getSignedUploadUrl or getSignedDownloadUrl on it throws UnsupportedOperation because the local filesystem has no concept of pre-signed URLs.

import { UnsupportedOperation } from '@devoven/storage';

// In a custom adapter, signal that an operation is not available:
async getSignedDownloadUrl(key: string, expiresInSeconds: number): Promise<string> {
  throw new UnsupportedOperation('getSignedDownloadUrl');
}

When UnsupportedOperation is thrown inside a request handler, NestJS's default (or your custom) exception filter catches it and returns:

{
  "statusCode": 501,
  "message": "getSignedDownloadUrl",
  "error": "Unsupported Operation"
}

If no operation name is passed to the constructor, the message defaults to "Operation is not supported".

Consumers that call signed-URL methods conditionally based on provider type should catch UnsupportedOperation and fall back gracefully rather than letting it propagate as a 501 to the client.