S3 Model Support

S3 backing for the travetto model module.

Install: @travetto/model-s3

npm install @travetto/model-s3

# or

yarn add @travetto/model-s3

This module provides an s3-based implementation for the Data Modeling Support. This source allows the Data Modeling Support module to read, write and stream against s3.

Supported features:

• CRUD
• Streaming
• Expiry Out of the box, by installing the module, everything should be wired up by default.If you need to customize any aspect of the source or config, you can override and register it with the Dependency Injection module.

Code: Wiring up a custom Model Source

import { InjectableFactory } from '@travetto/di';
import { S3ModelConfig, S3ModelService } from '@travetto/model-s3';

export class Init {
  @InjectableFactory({
    primary: true
  })
  static getModelSource(conf: S3ModelConfig) {
    return new S3ModelService(conf);
  }
}

where the S3ModelConfig is defined by:

Code: Structure of S3ModelConfig

import { fromIni } from '@aws-sdk/credential-provider-ini';
import type s3 from '@aws-sdk/client-s3';

import { Config, EnvVar } from '@travetto/config';
import { Field, Required } from '@travetto/schema';
import { Env } from '@travetto/base';

/**
 * S3 Support as an Asset Source
 */
@Config('model.s3')
export class S3ModelConfig {
  region = 'us-east-1'; // AWS Region
  namespace = ''; // S3 Bucket folder
  bucket = ''; // S3 bucket
  endpoint = ''; // Endpoint url

  @EnvVar('AWS_ACCESS_KEY_ID')
  accessKeyId: string = '';
  @EnvVar('AWS_SECRET_ACCESS_KEY')
  secretAccessKey: string = '';
  @EnvVar('AWS_PROFILE')
  profile?: string;

  @Field(Object)
  @Required(false)
  config: s3.S3ClientConfig; // Additional s3 config

  chunkSize = 5 * 2 ** 20; // Chunk size in bytes

  autoCreate?: boolean;

  /**
   * Provide host to bucket
   */
  get hostName(): string {
    return `${this.bucket}.s3.amazonaws.com`;
  }

  /**
   * Produces the s3 config from the provide details, post construction
   */
  async postConstruct(): Promise<void> {
    if (!this.accessKeyId && !this.secretAccessKey) {
      const creds = await fromIni({ profile: this.profile })();
      this.accessKeyId = creds.accessKeyId;
      this.secretAccessKey = creds.secretAccessKey;
    }

    this.config = {
      ...(this.config ?? {}),
      region: this.region,
      endpoint: this.endpoint,
      credentials: {
        accessKeyId: this.accessKeyId,
        secretAccessKey: this.secretAccessKey
      }
    };

    // We are in localhost and not in prod, turn on forcePathStyle
    if (!Env.production && this.endpoint.includes('localhost')) {
      this.config.forcePathStyle ??= true;
    }
  }
}

Additionally, you can see that the class is registered with the @Config annotation, and so these values can be overridden using the standard Configuration resolution paths.

Note: Do not commit your accessKeyId or secretAccessKey values to your source repository, especially if it is public facing. Not only is it a security risk, but Amazon will scan public repos, looking for keys, and if found will react swiftly.