@nx/s3-cache

This plugin provides a Nx cache which can be self hosted on Amazon S3 or S3 compatible storage providers.

Usage

Use of this package is governed by the following LICENSE. Please be sure to read through the license carefully before using this plugin. This license is also included in the package in a LICENSE file.

Use Cases

This plugin is useful when Nx Cloud is not a viable solution. Nx Cloud is the ideal recommended method of sharing a cache between multiple machines using Nx.

This plugin uploads and downloads artifacts and metadata to and from an S3 bucket of your choice. This allows Nx to run faster in CI environments by reusing computation done in other runs.

Adding this plugin to your Nx workspace

Use nx add @nx/s3-cache to install and setup this plugin in your Nx workspace.

Configuration

This cache can be configured in an s3 property in nx.json. The following properties are configurable:

| Property | Description | Default | Required | | ----------------- | --------------------------------------------------------------------------------------------- | ------------ | -------- | | bucket | The name of the S3 bucket to use for the cache. | N/a | Yes | | region | The AWS region where the s3 bucket lives. | N/a | Yes | | ssoProfile | The name of the SSO profile to use from your AWS CLI SSO Configuration. | N/a | No | | endpoint | The endpoint where AWS can be reached. | N/a | No | | encryptionKey | The key used to encrypt and decrypt artifacts from the cache. | N/a | No | | accessKeyId | The access key ID for AWS. | N/a | No | | secretAccessKey | The secret access key for AWS. | N/a | No | | forcePathStyle | Use path style URLs for S3 objects. | false | No | | localMode | The mode that the cache behaves as during local development: read, read-write, no-cache | 'read-write' | No | | ciMode | The mode that the cache behaves as during CI runs: read, read-write, no-cache | 'read-write' | No | | disableChecksum | This disables AWS' checksum validation for cache entries. | false | No | | cacheKeyPrefix | This adds a prefix to your cache key. | N/a | No |

S3 Compatible Providers

To use @nx/s3-cache with S3 compatible providers (MinIO, LocalStack, DigitalOcean Spaces, Cloudflare, etc..), endpoint will need to be provided. Some providers also need to have forcePathStyle set to true in the configuration.

Below is an example on how to connect to MinIO:

{
  "s3": {
    "region": "us-east-1",
    "bucket": "my-bucket",
    "endpoint": "https://play.min.io",
    "forcePathStyle": true,
    "accessKeyId": "abc1234",
    "secretAccessKey": "4321cba",
    "disableChecksum": true
  }
}

| Property | Description | | ------------------- | ---------------------------------------------------------------------------------------------------------------- | | region | The id of the S3 compatible storage region to use | | bucket | The name of the S3 compatible storage bucket to use | | forcePathStyle | Changes the way artifacts are uploaded. Usually used for S3 compatible providers (MinIO, LocalStack, etc) | | endpoint | The custom endpoint to upload artifacts to. If endpoint is not defined, the default AWS endpoint is used | | accessKeyId | AWS Access Key ID (optional if AWS_ACCESS_KEY_ID is set in the environment) | | secretAccessKey | AWS secret access key (optional if AWS_SECRET_ACCESS_KEY is set in the environment) | | disableChecksum | Disable checksum validation (optional but encouraged if you are using MinIO earlier than 2024-07-04T14-25-45Z) | | cacheKeyPrefix | Adds a prefix to your cache key, this is useful when you want to use the same bucket for multiple projects |