@cumulus/move-granules

This lambda function is responsible for moving granule files from a file staging location to their final location.

Message Configuration

For more information on configuring a Cumulus Message Adapter task, see the Cumulus workflow input/output documentation.

Config

Config object fields:

| field name | type | default | values | description | ---------- | ---- | ------- | ------ | ----------- | bucket | string | (required) | | Name of the source AWS S3 bucket that contains the granule files | buckets | object | (required) | | Object specifying AWS S3 buckets used by this task | collection | object | (required) | | The cumulus-api collection object | distribution_endpoint | string | (required) | | The API distribution endpoint | duplicateHandling | string | error | error - Throws an error on duplicatesreplace - Replaces the existing fileskip - Skips the duplicate fileversion - Adds a suffix to the existing filename to avoid a clash | Specifies how duplicate filenames should be handled. In cases where an object collision results in duplicate evaluation, the task will check the database via API for a granuleId collision in another collection and error regardless of this configuration if one is found. | moveStagedFiles | boolean | true | | Can set to false to skip moving files from the staging location. Defaults to true. | s3MultipartChunksizeMb | number | | | S3 multipart upload chunk size in MB. If none is specified, the default default_s3_multipart_chunksize_mb is used. | checkCrossCollectionCollisions | boolean | true | true/false | Default behavior configures MoveGranules to use the API to determine if a file collision is the result of a file registered to a granule in another collection, and if it is, fail (ignoring duplicateHandling configuration). Setting this value to false disables that check and MoveGranules will not use database checks and assume all collisions are due to same-collection collisions. | collectionCheckRetryCount | number | 3 | | "Number of times to retry checking for existing files in the collection when checkCrossCollectionCollisions is true. Default is 3. | crossCollectionThrowOnObjectNotFound | boolean | false | true/false | If checkCrossCollectionCollisions is set to true, when checking for cross-collection collisions, throw an error if a file is not found in the database. If false, files not found in the database will assume an orphan that can be overwritten. Default is false.

Input

Input object fields:

| field name | type | default | description | ---------- | ---- | ------- | ----------- | granules | array<object> | (required) | List of granule objects

Output

Output object fields:

| field name | type | default | description | ---------- | ---- | ------- | ----------- | granules | array<object> | N/A | List of granule objects with updated S3 location information

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Contributing

To make a contribution, please see our contributing guidelines.