@novahelm/storage

S3-compatible storage package for NovaHelm -- provides the storage client factory, presigned URL generation, multipart uploads, file scanning, and a full set of object management helpers. Works with MinIO (local dev) and any S3-compatible provider in production.

Quick Start

pnpm add @novahelm/storage

import { createStorage, createStorageHelpers } from "@novahelm/storage/server";
import { initNova } from "@novahelm/core";

// 1. Create the S3 client
const storage = createStorage({
  endpoint: env.S3_ENDPOINT,
  accessKeyId: env.S3_ACCESS_KEY,
  secretAccessKey: env.S3_SECRET_KEY,
  bucket: env.S3_BUCKET,
});

// 2. Register with the Nova registry
initNova({ db, redis, auth, storage, logger, config });

// 3. Create helpers for convenient operations
const helpers = createStorageHelpers(storage.s3, storage.bucket);

createStorage Options

import { createStorage } from "@novahelm/storage/server";

const storage = createStorage({
  endpoint: "https://s3.us-east-1.amazonaws.com",
  accessKeyId: "AKIAIOSFODNN7EXAMPLE",
  secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  region: "us-east-1",          // default: "us-east-1"
  bucket: "my-app-uploads",

  // Optional: ClamAV virus scanner
  scanner: { host: "clamav", port: 3310 },

  // Allow local MinIO (auto-enabled in non-production)
  allowPrivateEndpoint: true,
});

| Option | Type | Default | Description | |--------|------|---------|-------------| | endpoint | string | -- | S3-compatible endpoint URL (required) | | accessKeyId | string | -- | AWS access key (required) | | secretAccessKey | string | -- | AWS secret key (required) | | region | string | "us-east-1" | AWS region | | bucket | string | -- | Default bucket name (required) | | scanner | { host, port? } | -- | ClamAV scanner config (optional) | | allowPrivateEndpoint | boolean | !production | Allow private/loopback endpoints |

The factory includes SSRF protection -- private/loopback endpoints are blocked in production unless explicitly allowed.

Storage Helpers

createStorageHelpers() returns a complete set of bucket-bound operations:

import { createStorageHelpers } from "@novahelm/storage/server";

const helpers = createStorageHelpers(storage.s3, storage.bucket);

Upload & Download

// Upload a buffer
await helpers.upload("avatars/user-123.png", imageBuffer, "image/png");

// Upload a stream (large files)
await helpers.uploadStream("videos/intro.mp4", readStream, "video/mp4", fileSize);

// Download for server-side processing
const obj = await helpers.getObject("avatars/user-123.png");
const body = obj.Body; // ReadableStream

// Generate a unique storage key
const key = helpers.generateKey("uploads", "photo.jpg");
// => "uploads/1709234567890-a1b2c3d4-photo.jpg"

Presigned URLs

// Presigned POST for browser-direct upload
const { url, fields } = await helpers.createPresignedPost({
  key: "uploads/photo.jpg",
  contentType: "image/jpeg",
  maxSizeMb: 10,
});
// Use url + fields in an HTML form or fetch

// Signed download URL (time-limited)
const downloadUrl = await helpers.getSignedDownloadUrl("files/report.pdf", 3600);

Object Management

// Check existence
const exists = await helpers.objectExists("avatars/user-123.png");

// Get metadata without downloading
const meta = await helpers.getObjectMetadata("avatars/user-123.png");
// => { contentType, contentLength, lastModified }

// List objects in a prefix
const files = await helpers.listObjects("uploads/user-123/", 100);
// => [{ key, size, lastModified }]

// Copy within bucket
await helpers.copyObject("temp/photo.jpg", "avatars/user-123.jpg");

// Delete single
await helpers.deleteObject("temp/photo.jpg");

// Delete batch
const { deleted, errors } = await helpers.deleteObjects([
  "temp/a.jpg", "temp/b.jpg", "temp/c.jpg",
]);

// Public URL
const url = helpers.getPublicUrl("avatars/user-123.png", "https://cdn.example.com");

// Health check
await helpers.headBucket(); // throws if unreachable

Multipart Upload

For large files, use multipart uploads (server-side or browser-direct):

Server-Side (High-Level)

// Automatic chunking, upload, and completion
const result = await helpers.uploadMultipart("backups/db.sql.gz", largeBuffer, {
  contentType: "application/gzip",
  partSizeMb: 10,           // default 10, min 5
  onProgress: ({ uploaded, total, partNumber }) => {
    console.log(`Part ${partNumber}: ${uploaded}/${total}`);
  },
});

Server-Side (Low-Level)

// 1. Initiate
const handle = await helpers.initiateMultipartUpload({
  key: "backups/db.sql.gz",
  contentType: "application/gzip",
});

// 2. Upload parts (min 5MB each, except last)
const part1 = await helpers.uploadPart(handle, 1, chunk1);
const part2 = await helpers.uploadPart(handle, 2, chunk2);

// 3. Complete
await helpers.completeMultipartUpload(handle, [part1, part2]);

// Or abort on failure
await helpers.abortMultipartUpload(handle);

Browser-Direct (Presigned)

Generate presigned URLs so browsers upload chunks directly to S3:

// Server: generate presigned URLs
const upload = await helpers.createPresignedMultipartUpload({
  key: "videos/intro.mp4",
  contentType: "video/mp4",
  partCount: 5,           // number of chunks
  expiresIn: 3600,        // URL expiration in seconds
});
// => { uploadId, key, partUrls: string[] }

// Browser: PUT each chunk to its presigned URL
// Collect ETags from response headers

// Server: complete the upload
await helpers.completePresignedMultipartUpload({
  key: upload.key,
  uploadId: upload.uploadId,
  parts: [
    { partNumber: 1, etag: "abc123" },
    { partNumber: 2, etag: "def456" },
    // ...
  ],
});

// Or abort if cancelled
await helpers.abortPresignedMultipartUpload({
  key: upload.key,
  uploadId: upload.uploadId,
});

Bucket Initialization

Create required buckets and configure CORS on startup (idempotent):

import { initBuckets } from "@novahelm/storage/server";

await initBuckets(storage.s3, {
  bucket: "my-app",
  corsOrigins: ["http://localhost:3000", "https://myapp.com"],
});
// Creates: "my-app" (private) and "my-app-public" (public)

Virus Scanning

Optional ClamAV integration for uploaded files:

import { scanBuffer } from "@novahelm/storage/server";

const result = await scanBuffer(fileBuffer, {
  host: "clamav",
  port: 3310,
  timeoutMs: 30_000,
});

if (!result.clean) {
  console.log(`Virus detected: ${result.virus}`);
}
// result: { clean: boolean, virus?: string, skipped?: boolean }

API Reference

| Export | Description | |--------|-------------| | createStorage(config) | Create an S3 client bound to a bucket | | StorageConfig | Configuration type | | NovaStorage | Return type of createStorage() | | createStorageHelpers(s3, bucket) | Create bucket-bound helper methods | | NovaStorageHelpers | Return type of createStorageHelpers() | | initBuckets(s3, config) | Create buckets and configure CORS | | initiateMultipartUpload(s3, bucket, opts) | Start multipart upload | | uploadPart(s3, handle, partNumber, body) | Upload a single part | | completeMultipartUpload(s3, handle, parts) | Finalize multipart upload | | abortMultipartUpload(s3, handle) | Cancel multipart upload | | listParts(s3, handle) | List uploaded parts | | uploadMultipart(s3, bucket, key, body, opts) | High-level multipart upload | | scanBuffer(buffer, options) | Scan file with ClamAV | | parseClamAVResponse(response) | Parse raw ClamAV output |