@onlineapps/conn-base-storage

Unified MinIO client for all OA Drive services. Provides direct S3-compatible access to object storage with fingerprint-based immutable content handling.

Why Direct Access (Not MQ Wrapper)?

1. Performance - MinIO is optimized for direct streaming
2. Standard - S3 API is industry standard
3. Efficiency - Direct streaming for large files
4. Best Practice - Object storage always accessed directly

Installation

npm install @onlineapps/conn-base-storage

Quick Start

const StorageConnector = require('@onlineapps/conn-base-storage');

const storage = new StorageConnector({
  endpoint: process.env.MINIO_HOST || 'api_services_storage',
  port: 9000,
  accessKey: process.env.MINIO_ACCESS_KEY,
  secretKey: process.env.MINIO_SECRET_KEY
});

// Upload with fingerprint
const result = await storage.uploadWithFingerprint(
  'registry',
  content,
  'specs/invoicing'
);
// Returns: { path: 'specs/invoicing/abc123.json', fingerprint: 'abc123', size: 1234 }

// Download with verification
const content = await storage.downloadWithVerification(
  'registry',
  'specs/invoicing/abc123.json',
  'abc123'  // expected fingerprint
);

// Get pre-signed URL for external access
const url = await storage.getPresignedUrl(
  'registry',
  'specs/invoicing/abc123.json',
  3600  // expiration in seconds
);

Core Methods

uploadWithFingerprint(bucket, content, basePath)

Uploads content with automatic fingerprint in filename.

downloadWithVerification(bucket, path, expectedFingerprint)

Downloads file and verifies fingerprint.

exists(bucket, path)

Checks if file exists.

getPresignedUrl(bucket, path, expiry)

Creates temporary URL for external access.

calculateFingerprint(content)

Calculates SHA256 fingerprint.

listByPrefix(bucket, prefix)

Lists all objects with given prefix.

Usage in Registry

const StorageConnector = require('@onlineapps/conn-base-storage');

class SpecPublisher {
  constructor() {
    this.storage = new StorageConnector();
  }

  async publishSpec(serviceName, spec) {
    // Upload spec to MinIO
    const result = await this.storage.uploadWithFingerprint(
      'registry',
      JSON.stringify(spec),
      `specs/${serviceName}`
    );

    // Publish event with reference
    await this.mq.publish('registry.changes', {
      type: 'SPEC_PUBLISHED',
      service: serviceName,
      fingerprint: result.fingerprint,
      path: result.path,
      bucket: result.bucket
    });

    return result;
  }
}

Usage in Services

const StorageConnector = require('@onlineapps/conn-base-storage');

class InvoicingService {
  constructor() {
    this.storage = new StorageConnector();
  }

  async saveInvoice(invoice) {
    // Save invoice PDF to storage
    const result = await this.storage.uploadWithFingerprint(
      'services',
      invoice.pdfBuffer,
      'invoicing/invoices'
    );

    // Return reference, not content
    return {
      invoiceId: invoice.id,
      pdfUrl: result.path,
      fingerprint: result.fingerprint,
      size: result.size
    };
  }

  async getInvoice(path, fingerprint) {
    // Download and verify
    return await this.storage.downloadWithVerification(
      'services',
      path,
      fingerprint
    );
  }
}

Security

Bucket Access Rights

• registry/ - Read-only for services, write for Registry
• workflow/ - Read/write for workflow components
• services/ - Read/write for services (each has own prefix)

Network Isolation

• MinIO runs only in internal Docker network
• External access only via pre-signed URLs

Testing

const StorageConnector = require('@onlineapps/conn-base-storage');

describe('StorageConnector', () => {
  let storage;

  beforeAll(() => {
    storage = new StorageConnector({
      endpoint: 'localhost',
      port: 33025  // Host port
    });
  });

  test('upload and download with fingerprint', async () => {
    const content = JSON.stringify({ test: 'data' });

    const upload = await storage.uploadWithFingerprint(
      'test',
      content,
      'specs/test'
    );

    expect(upload.fingerprint).toBeDefined();

    const downloaded = await storage.downloadWithVerification(
      'test',
      upload.path,
      upload.fingerprint
    );

    expect(downloaded).toBe(content);
  });
});

Environment Variables

# MinIO Configuration
MINIO_HOST=api_services_storage
MINIO_PORT=9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_USE_SSL=false

# From host (for testing)
MINIO_HOST=localhost
MINIO_PORT=33025

Benefits

1. Unified API - All services use same library
2. Built-in Fingerprints - Automatic immutable content management
3. Verification - Integrity check on download
4. Pre-signed URLs - Secure external access
5. Efficient - Direct streaming, no MQ overhead
6. Standard - S3 API is industry standard

This is the correct way to handle object storage in microservices architecture.

📚 Documentation

• Complete Connectors Documentation
• MinIO Infrastructure