node-red-contrib-bucket-collection

Node-RED nodes for object storage bucket operations.

Supports S3-compatible providers (MinIO, AWS S3, Oracle Object Storage) and Google Cloud Storage.

A single bucket node handles all four operations: upload, get, delete, list.

Nodes

| Node | Type | Purpose | |------|------|---------| | bucket-config | config | Shared connection and credentials | | bucket | operation | Choose upload, get, delete, or list via the operation field |

Config Node (bucket-config)

Create one config node per storage backend. The bucket node references it.

Provider: S3-compatible (MinIO, AWS S3, Oracle)

| Field | Description | |-------|-------------| | Endpoint | Server hostname or IP. Leave blank for AWS S3 | | Port | Server port (e.g. 9000 for MinIO; leave blank for AWS S3) | | Use SSL | Enable HTTPS | | Region | Required for AWS S3 (e.g. us-east-1); optional for MinIO | | Access Key | Stored as a credential | | Secret Key | Stored as a credential (password field) |

Provider: Google Cloud Storage

| Field | Description | |-------|-------------| | Service Account JSON | Full contents of a GCS service account key file. Stored encrypted. Must have Storage Object Admin (or equivalent) permissions on the target bucket |

The storage client is instantiated once at deploy time and reused across all messages.

The bucket node

Pick the operation in the node configuration (or pass msg.operation). The UI hides Object Name for list since it is not used.

| Config | Description | Override via | |--------|-------------|--------------| | Connection | bucket-config node | — | | Operation | upload | get | delete | list | msg.operation | | Bucket | Target bucket name | msg.bucket | | Object Name | Object path (not used for list) | msg.objectName | | Input Source | payload (default) or file — upload only | — | | File Path | Local file path — upload + file only | msg.filePath | | Output Target | payload (default) or file — get only | — | | Output File Path | Local file path — get + file only | msg.outputFilePath | | Overwrite | Allow replacing the destination — get + file only | — |

Upload input sources

For upload, the data source is selected explicitly with Input Source:

• payload (default, recommended) — uses msg.payload (stream or Buffer). msg.filePath is ignored.
• file — reads from the configured File Path (or msg.filePath). msg.payload is ignored.

The two sources are strictly exclusive: there is no auto-detection and no fallback.

Warning: File path mode reads from the local filesystem at runtime and may not work across environments (e.g., Docker, cloud deployments).

Get output targets

For get, the destination is selected explicitly with Output Target:

• payload (default) — msg.payload is the readable stream returned by the provider. Useful for in-flow processing.
• file — streams the object directly to the local filesystem at the configured path (or msg.outputFilePath). Memory-safe for large files: nothing is buffered. The output is msg.payload = { filePath, bucket, objectName }, with msg.filePath, msg.bucket, msg.objectName also set.

msg.outputFilePath only takes effect when Output Target is file. Setting it does not switch modes.

Overwrite behavior — by default the node refuses to overwrite an existing destination. Enable Overwrite to allow replacement.

Parent directory — must already exist. The node never creates directories silently.

Warning: File output mode writes to the local filesystem at runtime. Ensure the Node-RED process has permission to write to the selected path, and do not rely on local paths when moving flows across environments.

Validation

Required inputs are checked before any provider call:

• bucket is required for every operation.
• objectName is required for upload, get, delete (not for list).
• For upload with inputSource: payload: msg.payload must be present.
• For upload with inputSource: file: a file path must be set on the node or via msg.filePath.
• For get with outputTarget: file: an output file path must be set on the node or via msg.outputFilePath; the parent directory must already exist; if the destination file exists and Overwrite is off, the node fails before downloading.

A missing or unsatisfied precondition surfaces a clear error and the node status turns red (invalid input) before any provider call is made.

Status indicators

• Yellow dot — operation in progress
• Green dot — success
• Red ring — error or invalid input

Operation contracts

| Operation | msg.payload in | msg.payload out | |-----------|-------------------|--------------------| | upload | readable stream or Buffer (or read from File Path) | provider result | | get (payload mode) | — | readable stream of the object | | get (file mode) | — | { filePath, bucket, objectName } | | delete | — | { success: true } | | list | — | array of { name, size, lastModified, etag } |

Known Limitations

• get in payload mode returns a stream — the user is responsible for consuming it downstream
• upload payload mode expects msg.payload to be a stream or Buffer
• list does not support prefix filtering — always lists from the bucket root
• File modes (upload from file, get to file) require local filesystem access and are not portable across environments
• No retry logic

Design Principles

• One node, many operations — the bucket node selects its behavior via a single operation field, with field visibility adapting to the choice
• Config node as single source of truth — credentials live in one place
• Provider-agnostic interface — the node UI exposes no provider-specific options

Tests

npm test

Runs the unit suite under Node's built-in test runner (node --test). Tests cover operation routing, validation, upload/get input/output modes, file handling, message overrides, and error propagation. The storage client is mocked — no network or cloud access is required.

Requires Node 18+.

License

Apache-2.0