@x12i/xronox-storage-interface

Base storage interface and key generation utilities for unified storage operations across S3, Azure, and local filesystem adapters.

📦 Installation

npm install @x12i/xronox-storage-interface

Or add to your package.json:

{
  "dependencies": {
    "@x12i/xronox-storage-interface": "^2.2.0"
  }
}

🎯 Purpose

This package provides:

1. StorageAdapter Interface - Unified storage interface for S3, Azure, and local filesystem operations
2. Key Generation Utilities - Functions for generating standardized storage keys
3. Key Parsing Utilities - Functions for parsing and extracting information from storage keys

Why This Package Exists

Modern applications need to store data across different storage providers (S3, Azure Blob Storage, local filesystem) without coupling code to specific implementations. This package provides:

• ✅ Storage-agnostic code - Write once, run on any storage backend
• ✅ Standardized key formats - Consistent naming across all storage providers
• ✅ Type safety - Full TypeScript support with detailed type definitions
• ✅ Zero runtime dependencies - Pure TypeScript interfaces and utilities

🚀 Quick Start

Basic Interface Usage

import { StorageAdapter } from '@x12i/xronox-storage-interface';

// Implement the interface for your storage provider
class S3StorageAdapter implements StorageAdapter {
  async putJSON(bucket: string, key: string, data: any) {
    // Your S3 implementation
    return { size: 1024, sha256: 'abc123...' };
  }
  
  async getJSON(bucket: string, key: string) {
    // Your S3 implementation
    return { /* data */ };
  }
  
  // ... implement other methods
}

// Use the adapter
const storage: StorageAdapter = new S3StorageAdapter();
await storage.putJSON('my-bucket', 'data/item.json', { name: 'Test' });

Key Generation

import { jsonKey, propBlobKey, manifestKey } from '@x12i/xronox-storage-interface';

// Generate a JSON key for versioned storage
const key1 = jsonKey('users', 'abc123', 1);
// Result: 'users/abc123/v1/item.json'

// Generate a blob key for binary properties
const key2 = propBlobKey('documents', 'content', 'def456', 2);
// Result: 'documents/content/def456/v2/blob.bin'

// Generate a manifest key
const key3 = manifestKey('products', 2024, 10, 5);
// Result: '__manifests__/products/2024/10/snapshot-5.json.gz'

Key Parsing

import { parseJsonKey, parseBlobKey } from '@x12i/xronox-storage-interface';

// Parse a JSON key
const parsed1 = parseJsonKey('users/abc123/v1/item.json');
// Result: { collection: 'users', id: 'abc123', ov: 1 }

// Parse a blob key
const parsed2 = parseBlobKey('documents/content/def456/v2/blob.bin');
// Result: { collection: 'documents', prop: 'content', id: 'def456', ov: 2 }

📚 API Reference

StorageAdapter Interface

The main storage interface that all adapters must implement:

Methods

putJSON(bucket, key, data)

Store JSON data.

Parameters:

• bucket: string - Storage bucket/container name
• key: string - Object key/path
• data: any - Data to store (will be JSON serialized)

Returns: Promise<{ size: number | null; sha256: string | null }>

Example:

await storage.putJSON('my-bucket', 'users/123.json', { name: 'John' });

putRaw(bucket, key, body, contentType?)

Store raw binary data.

Parameters:

• bucket: string - Storage bucket/container name
• key: string - Object key/path
• body: Buffer | Uint8Array - Raw data to store
• contentType?: string - Optional MIME type

Returns: Promise<{ size: number | null; sha256: string | null }>

Example:

const buffer = Buffer.from('Hello World');
await storage.putRaw('my-bucket', 'files/hello.txt', buffer, 'text/plain');

getJSON(bucket, key)

Retrieve and parse JSON data.

Parameters:

• bucket: string - Storage bucket/container name
• key: string - Object key/path

Returns: Promise<Record<string, unknown>>

Example:

const data = await storage.getJSON('my-bucket', 'users/123.json');
console.log(data.name); // 'John'

getRaw(bucket, key)

Retrieve raw binary data.

Parameters:

• bucket: string - Storage bucket/container name
• key: string - Object key/path

Returns: Promise<Buffer>

Example:

const buffer = await storage.getRaw('my-bucket', 'files/hello.txt');
console.log(buffer.toString()); // 'Hello World'

head(bucket, key)

Check if object exists and get metadata.

Parameters:

• bucket: string - Storage bucket/container name
• key: string - Object key/path

Returns: Promise<{ exists: boolean; size?: number }>

Example:

const { exists, size } = await storage.head('my-bucket', 'users/123.json');
if (exists) {
  console.log(`File size: ${size} bytes`);
}

del(bucket, key)

Delete an object.

Parameters:

• bucket: string - Storage bucket/container name
• key: string - Object key/path

Returns: Promise<void>

Example:

await storage.del('my-bucket', 'users/123.json');

presignGet(bucket, key, ttlSeconds)

Generate a presigned URL for temporary access.

Parameters:

• bucket: string - Storage bucket/container name
• key: string - Object key/path
• ttlSeconds: number - URL expiration time in seconds

Returns: Promise<string>

Example:

const url = await storage.presignGet('my-bucket', 'files/document.pdf', 3600);
console.log(`Download link: ${url}`);

list(bucket, prefix, opts?)

List objects with a given prefix.

Parameters:

• bucket: string - Storage bucket/container name
• prefix: string - Key prefix to filter by
• opts?: { maxKeys?: number; continuationToken?: string } - Pagination options

Returns: Promise<{ keys: string[]; nextToken?: string }>

Example:

const { keys, nextToken } = await storage.list('my-bucket', 'users/', { maxKeys: 100 });
console.log(`Found ${keys.length} objects`);

// Pagination
if (nextToken) {
  const nextPage = await storage.list('my-bucket', 'users/', { 
    maxKeys: 100, 
    continuationToken: nextToken 
  });
}

copy(sourceBucket, sourceKey, destBucket, destKey)

Copy an object to a new location.

Parameters:

• sourceBucket: string - Source bucket name
• sourceKey: string - Source object key
• destBucket: string - Destination bucket name
• destKey: string - Destination object key

Returns: Promise<void>

Example:

await storage.copy('bucket-1', 'users/123.json', 'bucket-2', 'backup/users/123.json');

Key Generation Functions

jsonKey(collection, idHex, ov)

Generate a key for versioned JSON storage.

Format: {collection}/{id}/v{version}/item.json

Parameters:

• collection: string - Collection name (lowercased automatically)
• idHex: string - Item ID as hex string (lowercased automatically)
• ov: number - Object version (non-negative integer)

Returns: string

Example:

jsonKey('Users', 'ABC123', 1);
// Result: 'users/abc123/v1/item.json'

propBlobKey(collection, prop, idHex, ov)

Generate a key for externalized binary properties.

Format: {collection}/{property}/{id}/v{version}/blob.bin

Parameters:

• collection: string - Collection name (lowercased)
• prop: string - Property name (lowercased)
• idHex: string - Item ID (lowercased)
• ov: number - Object version (non-negative integer)

Returns: string

Example:

propBlobKey('Documents', 'Content', 'DEF456', 2);
// Result: 'documents/content/def456/v2/blob.bin'

propTextKey(collection, prop, idHex, ov)

Generate a key for optional text renditions.

Format: {collection}/{property}/{id}/v{version}/text.txt

Parameters: Same as propBlobKey

Returns: string

manifestKey(collection, year, month, cv)

Generate a key for manifest snapshots.

Format: __manifests__/{collection}/{YYYY}/{MM}/snapshot-{cv}.json.gz

Parameters:

• collection: string - Collection name (lowercased)
• year: number - Year (4-digit, 1000-9999)
• month: number - Month (1-12)
• cv: number - Collection version (non-negative integer)

Returns: string

Example:

manifestKey('Products', 2024, 10, 5);
// Result: '__manifests__/products/2024/10/snapshot-5.json.gz'

Prefix Functions

itemPrefix(collection, idHex)

Get prefix for all versions of an item.

Returns: string - Format: {collection}/{id}/

collectionPrefix(collection)

Get prefix for all items in a collection.

Returns: string - Format: {collection}/

manifestPrefix(collection)

Get prefix for all manifests of a collection.

Returns: string - Format: __manifests__/{collection}/

manifestPeriodPrefix(collection, year, month)

Get prefix for manifests in a specific time period.

Returns: string - Format: __manifests__/{collection}/{YYYY}/{MM}/

Key Parsing Functions

parseJsonKey(key)

Parse a JSON key into its components.

Returns: { collection: string; id: string; ov: number } | null

Example:

parseJsonKey('users/abc123/v1/item.json');
// Result: { collection: 'users', id: 'abc123', ov: 1 }

parseJsonKey('invalid-key');
// Result: null

parseBlobKey(key)

Parse a blob key into its components.

Returns: { collection: string; prop: string; id: string; ov: number } | null

parseManifestKey(key)

Parse a manifest key into its components.

Returns: { collection: string; year: number; month: number; cv: number } | null

🏗️ Architecture

This package follows Poiesis Development Methodology principles:

Component Structure

xronox-storage-interface/
├── src/
│   ├── interface.ts     # StorageAdapter interface definitions
│   ├── keys.ts          # Key generation and parsing utilities
│   └── index.ts         # Main entry point with re-exports
├── docs/                # Documentation and guidelines
├── templates/           # Templates for development
└── package.json         # Package configuration

Design Principles

1. Single Responsibility - Each function/interface has one clear purpose
2. Type Safety - Full TypeScript coverage with strict mode
3. Zero Dependencies - No runtime dependencies (only dev dependencies)
4. Immutability - Pure functions with no side effects
5. Validation - Input validation with clear error messages

💡 Usage Examples

Example 1: S3 Adapter Implementation

import { StorageAdapter } from '@x12i/xronox-storage-interface';
import { S3Client, PutObjectCommand, GetObjectCommand } from '@aws-sdk/client-s3';

export class S3Adapter implements StorageAdapter {
  private client: S3Client;

  constructor(region: string) {
    this.client = new S3Client({ region });
  }

  async putJSON(bucket: string, key: string, data: any) {
    const body = JSON.stringify(data);
    const command = new PutObjectCommand({
      Bucket: bucket,
      Key: key,
      Body: body,
      ContentType: 'application/json',
    });
    
    await this.client.send(command);
    
    return {
      size: Buffer.byteLength(body),
      sha256: null, // Could calculate hash here
    };
  }

  async getJSON(bucket: string, key: string): Promise<Record<string, unknown>> {
    const command = new GetObjectCommand({ Bucket: bucket, Key: key });
    const response = await this.client.send(command);
    const body = await response.Body?.transformToString();
    return JSON.parse(body || '{}');
  }

  // ... implement other methods
}

Example 2: Local Filesystem Adapter

import { StorageAdapter } from '@x12i/xronox-storage-interface';
import * as fs from 'fs/promises';
import * as path from 'path';

export class LocalAdapter implements StorageAdapter {
  constructor(private baseDir: string) {}

  async putJSON(bucket: string, key: string, data: any) {
    const filePath = path.join(this.baseDir, bucket, key);
    await fs.mkdir(path.dirname(filePath), { recursive: true });
    const json = JSON.stringify(data, null, 2);
    await fs.writeFile(filePath, json, 'utf-8');
    
    return {
      size: Buffer.byteLength(json),
      sha256: null,
    };
  }

  async getJSON(bucket: string, key: string): Promise<Record<string, unknown>> {
    const filePath = path.join(this.baseDir, bucket, key);
    const content = await fs.readFile(filePath, 'utf-8');
    return JSON.parse(content);
  }

  // ... implement other methods
}

Example 3: Complete Storage Service

import {
  StorageAdapter,
  jsonKey,
  parseJsonKey,
  itemPrefix,
} from '@x12i/xronox-storage-interface';

export class DocumentService {
  constructor(private storage: StorageAdapter) {}

  async saveDocument(collection: string, id: string, version: number, data: any) {
    const key = jsonKey(collection, id, version);
    await this.storage.putJSON('documents', key, data);
    console.log(`Saved document: ${key}`);
  }

  async loadDocument(collection: string, id: string, version: number) {
    const key = jsonKey(collection, id, version);
    return await this.storage.getJSON('documents', key);
  }

  async listVersions(collection: string, id: string) {
    const prefix = itemPrefix(collection, id);
    const { keys } = await this.storage.list('documents', prefix);
    
    return keys
      .map(key => parseJsonKey(key))
      .filter(parsed => parsed !== null)
      .sort((a, b) => b!.ov - a!.ov);
  }
}

// Usage
const service = new DocumentService(new S3Adapter('us-east-1'));
await service.saveDocument('invoices', 'inv-001', 1, { amount: 100 });
const versions = await service.listVersions('invoices', 'inv-001');

🧪 Testing

Unit Testing Key Functions

import { jsonKey, parseJsonKey } from '@x12i/xronox-storage-interface';

describe('Key Generation', () => {
  test('jsonKey generates correct format', () => {
    const key = jsonKey('users', 'abc123', 1);
    expect(key).toBe('users/abc123/v1/item.json');
  });

  test('jsonKey normalizes case', () => {
    const key = jsonKey('USERS', 'ABC123', 1);
    expect(key).toBe('users/abc123/v1/item.json');
  });

  test('jsonKey validates inputs', () => {
    expect(() => jsonKey('', 'abc', 1)).toThrow('Collection and ID must be non-empty');
    expect(() => jsonKey('users', '', 1)).toThrow('Collection and ID must be non-empty');
    expect(() => jsonKey('users', 'abc', -1)).toThrow('Object version must be a non-negative integer');
  });

  test('parseJsonKey extracts components', () => {
    const parsed = parseJsonKey('users/abc123/v1/item.json');
    expect(parsed).toEqual({ collection: 'users', id: 'abc123', ov: 1 });
  });

  test('parseJsonKey returns null for invalid keys', () => {
    expect(parseJsonKey('invalid-key')).toBeNull();
  });
});

🔗 Integration with Xronox Ecosystem

This package is part of the Xronox ecosystem and works seamlessly with:

• @nx-intelligence/nxconfig - Configuration management
• logs-gateway - Structured logging
• xronox-s3-adapter - S3 implementation
• xronox-azure-adapter - Azure Blob Storage implementation
• xronox-local-adapter - Local filesystem implementation

🛡️ Type Safety

Full TypeScript support with strict type checking:

import { StorageAdapter } from '@x12i/xronox-storage-interface';

// TypeScript will enforce all methods are implemented
class MyAdapter implements StorageAdapter {
  // TypeScript error if any method is missing
  // TypeScript error if method signatures don't match
  // Full autocomplete support in IDEs
}

📄 License

MIT License - see LICENSE file for details.

🤝 Contributing

This package follows Poiesis Development Methodology. Please read:

• docs/poiesis.md - Core principles
• docs/developer-principles.md - Development guidelines
• docs/testing-principles.md - Testing requirements

📞 Support

• Issues: GitHub Issues
• Documentation: See docs/ directory
• Examples: See examples/ directory

🔄 Version History

1.0.0 (Current)

✅ Initial release ✅ StorageAdapter interface ✅ Key generation utilities ✅ Key parsing utilities ✅ Full TypeScript support ✅ Comprehensive documentation ✅ Zero runtime dependencies

🎯 Roadmap

Future enhancements (post-1.0.0):

• Validation utilities for storage operations
• Advanced key pattern matching
• Performance benchmarking utilities
• Additional adapter examples

Built by x12i