Kuzzle Plugin S3

S3 has a right system to limit who can upload files to buckets.

Presigned URLs are special disposable URLs generated by S3. It is possible to upload a file directly to one of these URLs so that it can be stored into the bucket.

These URLs must be generated on the server side, this plugin includes among other things the generation of these URLs so that developers can then send their files directly to S3 from a client application.

Compatibility Matrix

| Kuzzle Version | Plugin Version | | -------------- | -------------- | | 1.x.x | 1.x.x | | 2.x.x | 2.x.x | | 2.x.x | 3.x.x |

Configuration

Your access key must have the following rights: GetObject , PutObject and DeleteObject.

Then in your kuzzlerc file, you can change the following configuration variable:

{
  "plugins": {
    "s3": {
      "endpoints": {
        "eu-west-3": {
          "endpoint": "https://s3.eu-west-3.amazonaws.com",
          "forcePathStyle": false,
          "accessKeyIdPath": "foo.s3.eu-west-3.accessKeyId",
          "secretAccessKeyPath": "foo.s3.eu-west-3.secretAccessKey",
          "isMinio": false
        },
        "us-east-1": {
          "endpoint": "https://s3.us-east-1.amazonaws.com",
          "forcePathStyle": false,
          "accessKeyIdPath": "foo.s3.us-east-1.accessKeyId",
          "secretAccessKeyPath": "foo.s3.us-east-1.secretAccessKey",
          "isMinio": false
        },
        "custom-minio": {
          "endpoint": "https://minio.example.com",
          "forcePathStyle": true,
          "accessKeyIdPath": "foo.s3.minio.accessKeyId",
          "secretAccessKeyPath": "foo.s3.minio.secretAccessKey",
          "isMinio": true
        }
      },
      "signedUrlTTL": 1200000
    }
  }
}

Updated Configuration Format

• Endpoints: Defines region-specific configurations, including endpoint URL and access keys.
• Signed URL TTL: Time-to-live for presigned URLs generated by the plugin.

In addition to Amazon AWS S3, this plugin allows you to use any S3-API compatible service accessible through the AWS-S3 SDK. Any specific configuration option can be added to the s3ClientOptions configuration attribute. Please note that the parameters are translated directly, so refer to the SDK documentation for available options.

Usage

Get a Presigned URL:

// Kuzzle request
{
  "controller": "s3/upload",
  "action": "getUploadUrl",
  "filename": "headcrab.png",
  "uploadDir": "xen"
}

// Kuzzle response
{
  "fileKey": "xen/<uuid>-headcrab.png",
  "uploadUrl": "https://s3.eu-west-3.amazonaws.com/...",
  "fileUrl": "https://s3.eu-west-3.amazonaws.com/...",
  "ttl": 1200000
}

Then send a PUT request to the uploadUrl URL with the body set to the file's content and a Content-Type header corresponding to the file mime type.

Example using the JavaScript SDK

  // Get a Presigned URL
  const file = document.getElementById('uploadInput').files[0];
  const { result } = await kuzzle.query({
    controller: 's3/upload',
    action: 'getUploadUrl',
    uploadDir: 'xen',
    filename: file.name
  });

  // Upload the file directly to S3
  const axiosOptions = {
    headers: {
      'Content-Type': file.type
    }
  };
  await axios.put(result.uploadUrl, file, axiosOptions);

API

upload*:getUploadUrl*

Returns a Presigned URL to upload directly to S3.
The URL is only valid for a specified period of time. (Configurable in the kuzzlerc file)

File uploaded to the generated URL must be validated with upload:validate otherwise they will be deleted after the same TTL as for the URL expiration.

Request format:

{
  "controller": "s3/upload",
  "action": "getUploadUrl",
  "filename": "headcrab.png",
  "uploadDir": "xen",
  "bucketRegion": "foo",
  "bucketName": "bar"
}

Response result format:

{
  "fileKey": "xen/<uuid>-headcrab.png",
  "uploadUrl": "https://s3.eu-west-3.amazonaws.com/...",
  "fileUrl": "https://s3.eu-west-3.amazonaws.com/...",
  "ttl": 1200000
}

file*:getFileUrl*

Returns the public file URL.

Request format:

{
  "controller": "s3/file",
  "action": "getFileUrl",
  "fileKey": "xen/<uuid>-headcrab.png",
  "bucketRegion": "foo",
  "bucketName": "bar"
}

Response result format:

{
  "fileUrl": "https://s3.eu-west-3.amazonaws.com/..."
}

file*:delete*

Deletes an uploaded file from S3.

Request format:

{
  "controller": "s3/file",
  "action": "fileDelete",
  "fileKey": "xen/<uuid>-headcrab.png",
  "bucketRegion": "foo",
  "bucketName": "bar"
}

file*:getFilesKeys*

List the files keys uploaded to an S3 Bucket.

Request format:

{
  "controller": "s3/file",
  "action": "getFilesKeys",
  "bucketRegion": "foo",
  "bucketName": "bar"
}

Installation

Local setup

You can use the docker-compose.yml file provided in this repository to start a Kuzzle stack with this plugin pre-installed.

docker-compose -f docker-compose.yml up