@statedelta/gateway

I/O gateway for StateDelta - HTTP, filesystem, and custom providers with partial fetch and integrity verification.

Note: Provider and Middleware interfaces are defined in @statedelta/core for consistency across packages.

Installation

pnpm add @statedelta/gateway

Quick Start

import { createGateway, HttpProvider, FileProvider } from '@statedelta/gateway';

const gateway = createGateway({
  providers: [new HttpProvider(), new FileProvider()],
});

// Load from filesystem
const config = await gateway.load('./config.json');

// Load from HTTP
const data = await gateway.load('https://api.example.com/data.json');

Features

• Partial Fetch - Load only headers (~4KB) via HTTP Range / fs.read
• Integrity Verification - Dual hash (header + body) via ?integrity=
• Security Modes - normal, safe, strict validation
• Early Return - Get headers immediately while body downloads
• Multiple Providers - HTTP, File, Memory, or custom
• Middleware Support - Cache, logging, mocking, or custom
• Type-safe Events - Load lifecycle and integrity events
• Cancellation - AbortController support
• Security - Path traversal protection, size limits, timeouts
• Zero Dependencies - Uses only native APIs

Partial Fetch (loadHeader)

Load only the header fields without downloading the entire file:

// Only downloads ~4KB instead of full file
const header = await gateway.loadHeader('./large-manifest.json');
console.log(header.header.extends);  // Parent reference
console.log(header.header.deps);     // Dependencies
console.log(header.hash);            // Header hash

Early Return (loadWithEarlyReturn)

Get headers immediately while body continues downloading:

const result = await gateway.loadWithEarlyReturn('./large.json');

// Available immediately
console.log(result.header.extends);
console.log(result.headerHash);

// Wait for full content when needed
const full = await result.complete;
console.log(full.content);

// Or cancel if not needed
result.cancel();

Integrity Verification

Verify both header and body hashes:

// Via URL query parameter
const result = await gateway.load('./config.json?integrity=headerHash,bodyHash');

// Via options
const result = await gateway.load('./config.json', {
  integrity: {
    header: 'sha256:abc123...',
    body: 'sha256:def456...',
  },
});

// Check verification result
console.log(result.integrity.headerVerified);  // true/false
console.log(result.integrity.bodyVerified);    // true/false

Security Modes

Configure validation strictness:

// Normal - no requirements (default)
const gateway = createGateway({ security: { mode: 'normal' } });

// Safe - requires body hash
const gateway = createGateway({ security: { mode: 'safe' } });

// Strict - requires both header and body hash
const gateway = createSecureGateway();
// or
const gateway = createGateway({ security: { mode: 'strict' } });

Providers

| Provider | Resolves | Partial Fetch | |----------|----------|---------------| | HttpProvider | http://*, https://* | HTTP Range | | FileProvider | file://*, ./*, ../*, /* | fs.read | | MemoryProvider | memory://*, keys in store | slice |

FileProvider Options

const provider = new FileProvider({
  basePath: './data',           // Base path for relative sources
  maxSize: 10 * 1024 * 1024,    // Max file size (default: 10MB)
  allowAbsolutePaths: true,     // Allow absolute paths outside basePath
});

| Option | Type | Default | Description | |--------|------|---------|-------------| | basePath | string | process.cwd() | Base path for relative sources | | maxSize | number | 10MB | Maximum file size in bytes | | allowAbsolutePaths | boolean | false | Allow absolute paths outside basePath |

Allowing Absolute Paths

For CLI tools that need to load files from anywhere:

const provider = new FileProvider({ allowAbsolutePaths: true });

// Now accepts absolute paths
await provider.fetch('/home/user/project/data.json');

Security Warning: Only use allowAbsolutePaths: true when the source of paths is trusted.

Relative Path Resolution with Referrer

FileProvider resolves relative paths (./, ../) using the referrer option:

// When loading a chain with extends
await gateway.load('./delta.json', {
  referrer: '/fixtures/head.json'  // Resolve relative to this file
});
// Resolves to: /fixtures/delta.json

This is used internally by Chain to resolve extends paths correctly. See ARCHITECTURE.md for details.

Custom Providers

Providers implement IProvider from @statedelta/core:

import type { IProvider, ProviderFetchResult } from '@statedelta/core';

class S3Provider implements IProvider {
  readonly name = 's3';

  canResolve(source: string): boolean {
    return source.startsWith('s3://');
  }

  async fetch(source: string): Promise<ProviderFetchResult> {
    const key = source.replace('s3://', '');
    const content = await this.s3Client.getObject(key);
    return { content };
  }
}

Provider Capabilities

All built-in providers support partial fetch:

const provider = new HttpProvider();
console.log(provider.capabilities);
// { supportsPartialFetch: true, supportsStreaming: true, supportsRangeRequests: true }

// Manual partial fetch
const partial = await provider.fetchPartial('https://example.com/large.json', {
  maxBytes: 4096,
});
console.log(partial.content);     // First 4KB
console.log(partial.bytesRead);   // Actual bytes read
console.log(partial.totalSize);   // Total file size (if known)
console.log(partial.truncated);   // true if file is larger

Middlewares

import { createCacheMiddleware, createLogMiddleware, MemoryCache } from '@statedelta/gateway';

const gateway = createGateway({
  providers: [new HttpProvider()],
  middlewares: [
    createCacheMiddleware({ store: new MemoryCache({ maxSize: 100, ttl: 60000 }) }),
    createLogMiddleware({ level: 'info' }),
  ],
});

Custom Middlewares

Middlewares implement IMiddleware from @statedelta/core:

import type { IMiddleware } from '@statedelta/core';
import type { MiddlewareContext, MiddlewareResult, LoadResult } from '@statedelta/gateway';

const metricsMiddleware: IMiddleware<MiddlewareContext, LoadResult> = {
  name: 'metrics',

  async before(ctx) {
    metrics.increment('requests');
    return { action: 'continue' };
  },

  async after(ctx, result) {
    metrics.timing('duration', result.metadata.duration);
    return result;
  },
};

Integration with StateDeltaEnvironment

Gateway providers and middlewares can be registered in the @statedelta/core StateDeltaEnvironment so the launcher/CLI can consume them through the launcher pipeline:

import { createEnvironment } from '@statedelta/core';
import { createLauncher } from '@statedelta/launcher';
import {
  HttpProvider,
  FileProvider,
  createCacheMiddleware,
  MemoryCache,
} from '@statedelta/gateway';

const env = createEnvironment({
  engines: {
    "[email protected]": () => createMyAdapter(),
  },
  providers: [
    new HttpProvider({ timeout: 5000 }),
    new FileProvider({ basePath: './data' }),
  ],
  middlewares: [
    createCacheMiddleware({ store: new MemoryCache() }),
  ],
}).freeze();

const resolver = createLauncher({ environment: env });

Events

// Load lifecycle
gateway.on('load:start', (source) => console.log(`Loading: ${source}`));
gateway.on('load:complete', (source, result) => console.log(`Done: ${source}`));
gateway.on('load:error', (source, error) => console.error(`Error: ${source}`));
gateway.on('load:cancelled', (source) => console.log(`Cancelled: ${source}`));

// Cache
gateway.on('cache:hit', (source) => console.log(`Cache hit: ${source}`));
gateway.on('cache:miss', (source) => console.log(`Cache miss: ${source}`));

// Integrity
gateway.on('header:extracted', (source, header) => console.log(`Header: ${header.name}`));
gateway.on('header:verified', (source, hash) => console.log(`Header OK: ${hash}`));
gateway.on('header:mismatch', (source, expected, actual) => console.error(`Header mismatch!`));
gateway.on('body:verified', (source, hash) => console.log(`Body OK: ${hash}`));
gateway.on('body:mismatch', (source, expected, actual) => console.error(`Body mismatch!`));
gateway.on('partial:fetch', (source, bytesRead, totalSize) => console.log(`Partial: ${bytesRead}/${totalSize}`));

Errors

import {
  NotFoundError,
  HttpError,
  HashMismatchError,
  TimeoutError,
  AbortError,
  HeaderExtractionError,
  HeaderIntegrityError,
  BodyIntegrityError,
  SecurityModeError,
  isIntegrityError,
} from '@statedelta/gateway';

try {
  await gateway.load('./config.json', { integrity: { body: 'sha256:wrong' } });
} catch (error) {
  if (isIntegrityError(error)) {
    console.error('Integrity check failed:', error.expected, error.actual);
  }
}

Hash Utilities

import {
  calculateHash,
  calculateHeaderHash,
  calculateIntegrityHashes,
  verifyHash,
} from '@statedelta/gateway';

// Body hash
const bodyHash = await calculateHash('{"foo": "bar"}');
// => "sha256:7a38bf..."

// Header hash (deterministic - sorted fields)
const headerHash = await calculateHeaderHash({
  name: 'test',
  version: '1.0',
  extends: './base.json',
});

// Both hashes at once
const hashes = await calculateIntegrityHashes(jsonContent);
console.log(hashes.header, hashes.body);

Integrity URL Utilities

import {
  parseSourceUrl,
  buildSourceUrl,
  stripIntegrity,
  hasIntegrity,
} from '@statedelta/gateway';

// Parse URL with integrity
const parsed = parseSourceUrl('./file.json?integrity=abc,def');
// { path: './file.json', integrity: { header: 'abc', body: 'def', algorithm: 'sha256' } }

// Build URL with integrity
const url = buildSourceUrl('./file.json', { header: 'abc', body: 'def' });
// './file.json?integrity=abc,def'

// Check and strip
console.log(hasIntegrity('./file.json?integrity=abc'));  // true
console.log(stripIntegrity('./file.json?integrity=abc')); // './file.json'

Testing

import { createTestGateway } from '@statedelta/gateway';

const gateway = createTestGateway({
  'config.json': { name: 'test', version: '1.0' },
  'users.json': [{ id: 1, name: 'Alice' }],
});

const config = await gateway.load('config.json');
const header = await gateway.loadHeader('config.json');

Documentation

• docs/ARCHITECTURE.md - Technical architecture
• API.md - Full API reference

License

MIT