@egintegrations/storage

Provider-agnostic storage library with S3 and Supabase implementations. Supports file uploads, downloads, pre-signed URLs, and public URL generation with a unified interface.

Installation

npm install @egintegrations/storage

Peer Dependencies

Install the storage provider(s) you need:

# For S3 (AWS, DigitalOcean Spaces, MinIO, etc.)
npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

# For Supabase Storage
npm install @supabase/storage-js

Features

• Provider-Agnostic Interface: Single API for all storage providers
• S3 Compatible: Works with AWS S3, DigitalOcean Spaces, MinIO, and other S3-compatible services
• Supabase Storage: First-class support for Supabase Storage
• Pre-signed URLs: Generate secure upload URLs for direct client uploads
• Public URLs: Generate public access URLs for stored files
• TypeScript: Full type safety with comprehensive type definitions
• Factory Pattern: Easy provider switching

Quick Start

Using Factory Pattern (Recommended)

import { createStorage } from '@egintegrations/storage';

// S3 (AWS, DigitalOcean Spaces, MinIO, etc.)
const s3Storage = createStorage({
  driver: 's3',
  bucket: 'my-bucket',
  region: 'us-east-1',
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
});

// Supabase Storage
const supabaseStorage = createStorage({
  driver: 'supabase',
  bucket: 'my-bucket',
  supabaseUrl: process.env.SUPABASE_URL,
  supabaseServiceKey: process.env.SUPABASE_SERVICE_KEY,
});

Using Environment Variables

import { createStorageFromEnv } from '@egintegrations/storage';

// Reads from: STORAGE_BUCKET, AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
const storage = createStorageFromEnv('s3');

// Reads from: STORAGE_BUCKET, SUPABASE_URL, SUPABASE_SERVICE_KEY
const storage = createStorageFromEnv('supabase');

Direct Provider Usage

import { S3StorageClient, SupabaseStorageClient } from '@egintegrations/storage';

// S3
const s3 = new S3StorageClient({
  driver: 's3',
  bucket: 'my-bucket',
  region: 'us-east-1',
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
});

// Supabase
const supabase = new SupabaseStorageClient({
  driver: 'supabase',
  bucket: 'my-bucket',
  supabaseUrl: process.env.SUPABASE_URL,
  supabaseServiceKey: process.env.SUPABASE_SERVICE_KEY,
});

Usage Examples

Upload a File

const fileBuffer = await fs.readFile('./image.jpg');

const url = await storage.putObject({
  key: 'uploads/image.jpg',
  contentType: 'image/jpeg',
  data: fileBuffer,
});

console.log(`File uploaded: ${url}`);
// https://my-bucket.s3.amazonaws.com/uploads/image.jpg

Delete a File

await storage.deleteObject({
  key: 'uploads/image.jpg',
});

console.log('File deleted');

Get Public URL

const url = storage.getPublicUrl('uploads/image.jpg');
console.log(`Public URL: ${url}`);

Generate Pre-signed Upload URL

Pre-signed URLs allow clients to upload files directly to storage without going through your server.

S3

const signedUrl = await storage.getSignedPutUrl({
  key: 'uploads/new-file.jpg',
  contentType: 'image/jpeg',
  expiresIn: 600, // 10 minutes
});

// Client can now upload directly to this URL
fetch(signedUrl, {
  method: 'PUT',
  body: fileData,
  headers: {
    'Content-Type': 'image/jpeg',
  },
});

Supabase

const { url, token } = await storage.getSignedPutUrl({
  key: 'uploads/new-file.jpg',
  contentType: 'image/jpeg',
});

// Client must include token in header
fetch(url, {
  method: 'PUT',
  body: fileData,
  headers: {
    'Content-Type': 'image/jpeg',
    'x-upsert-signature': token, // Required for Supabase
  },
});

Configuration

S3 Configuration

interface S3Config {
  driver: 's3';
  bucket: string;
  region?: string; // e.g., 'us-east-1'
  accessKeyId?: string;
  secretAccessKey?: string;
  endpoint?: string; // For S3-compatible services (MinIO, DigitalOcean Spaces)
  publicUrlBase?: string; // Custom CDN URL
}

Examples:

// AWS S3
const awsS3 = createStorage({
  driver: 's3',
  bucket: 'my-bucket',
  region: 'us-east-1',
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
});

// DigitalOcean Spaces
const doSpaces = createStorage({
  driver: 's3',
  bucket: 'my-space',
  region: 'nyc3',
  endpoint: 'https://nyc3.digitaloceanspaces.com',
  accessKeyId: process.env.DO_SPACES_KEY,
  secretAccessKey: process.env.DO_SPACES_SECRET,
});

// MinIO
const minio = createStorage({
  driver: 's3',
  bucket: 'my-bucket',
  endpoint: 'https://minio.example.com',
  accessKeyId: process.env.MINIO_ACCESS_KEY,
  secretAccessKey: process.env.MINIO_SECRET_KEY,
});

// With CloudFront CDN
const s3WithCDN = createStorage({
  driver: 's3',
  bucket: 'my-bucket',
  region: 'us-east-1',
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  publicUrlBase: 'https://cdn.example.com',
});

Supabase Configuration

interface SupabaseConfig {
  driver: 'supabase';
  bucket: string;
  supabaseUrl: string; // e.g., 'https://abc123.supabase.co'
  supabaseServiceKey: string; // Service role key
  publicUrlBase?: string; // Custom CDN URL
}

Example:

const supabase = createStorage({
  driver: 'supabase',
  bucket: 'avatars',
  supabaseUrl: process.env.SUPABASE_URL,
  supabaseServiceKey: process.env.SUPABASE_SERVICE_KEY,
});

Environment Variables

When using createStorageFromEnv():

S3

• STORAGE_BUCKET - Bucket name (required)
• AWS_REGION - AWS region (optional)
• AWS_ACCESS_KEY_ID - Access key (optional)
• AWS_SECRET_ACCESS_KEY - Secret key (optional)
• STORAGE_ENDPOINT - Custom endpoint for S3-compatible services (optional)
• STORAGE_PUBLIC_URL_BASE - Custom CDN URL (optional)

Supabase

• STORAGE_BUCKET - Bucket name (required)
• SUPABASE_URL - Supabase project URL (required)
• SUPABASE_SERVICE_KEY - Service role key (required)
• STORAGE_PUBLIC_URL_BASE - Custom CDN URL (optional)

API Reference

StorageClient Interface

All storage providers implement this interface:

interface StorageClient {
  putObject(params: PutObjectParams): Promise<string>;
  deleteObject(params: DeleteObjectParams): Promise<void>;
  getPublicUrl(key: string): string;
  getSignedPutUrl?(params: SignedUploadUrlParams): Promise<SignedUploadUrlResult>;
}

Methods

putObject(params)

Upload a file to storage.

Parameters:

• key (string): Object key / file path
• contentType (string): MIME type
• data (Buffer | Uint8Array | ArrayBuffer): File data

Returns: Promise - Public URL of uploaded file

deleteObject(params)

Delete a file from storage.

Parameters:

• key (string): Object key / file path

Returns: Promise

getPublicUrl(key)

Get public URL for a file.

Parameters:

• key (string): Object key / file path

Returns: string - Public URL

getSignedPutUrl(params) (optional)

Generate a pre-signed URL for direct upload.

Parameters:

• key (string): Object key / file path
• contentType (string): MIME type
• expiresIn (number, optional): Expiration time in seconds (default: 600)

Returns:

• S3: Promise - Pre-signed URL
• Supabase: Promise<{ url: string; token: string }> - URL and signature token

Error Handling

try {
  await storage.putObject({
    key: 'uploads/file.txt',
    contentType: 'text/plain',
    data: Buffer.from('content'),
  });
} catch (error) {
  if (error.message.includes('No such bucket')) {
    console.error('Bucket does not exist');
  } else if (error.message.includes('Access Denied')) {
    console.error('Invalid credentials or insufficient permissions');
  } else {
    console.error('Upload failed:', error);
  }
}

TypeScript Types

import type {
  StorageClient,
  StorageConfig,
  StorageDriver,
  PutObjectParams,
  DeleteObjectParams,
  SignedUploadUrlParams,
  SignedUploadUrlResult,
} from '@egintegrations/storage';

Testing

The package includes comprehensive tests with mocked providers:

npm test
npm test -- --coverage

Migration from Other Libraries

From aws-sdk v2

// Before (aws-sdk v2)
const s3 = new AWS.S3({
  accessKeyId: '...',
  secretAccessKey: '...',
});

await s3.putObject({
  Bucket: 'my-bucket',
  Key: 'file.txt',
  Body: buffer,
}).promise();

// After (@egintegrations/storage)
const storage = createStorage({
  driver: 's3',
  bucket: 'my-bucket',
  accessKeyId: '...',
  secretAccessKey: '...',
});

await storage.putObject({
  key: 'file.txt',
  contentType: 'text/plain',
  data: buffer,
});

From @supabase/supabase-js

// Before (@supabase/supabase-js)
import { createClient } from '@supabase/supabase-js';

const supabase = createClient(url, key);
await supabase.storage
  .from('bucket')
  .upload('file.txt', file);

// After (@egintegrations/storage)
const storage = createStorage({
  driver: 'supabase',
  bucket: 'bucket',
  supabaseUrl: url,
  supabaseServiceKey: key,
});

await storage.putObject({
  key: 'file.txt',
  contentType: 'text/plain',
  data: file,
});

License

MIT

Credits

Extracted from HS-C (Health Sciences Company) core packages with S3 and Supabase implementations.

Contributing

This package is maintained by EGI Integrations. For bugs or feature requests, please open an issue on the egi-comp-library repository.