NodeJS library to communicate with an objectstore

Table of Contents

• Features
• ETag Support
• Installing
• Configuration
• Implementation
• License

Features

• Fully integrated in CAP as a Service
• Support S3 objectstore and local objectstore for development testings
• Autodeletion if corresponding entity entry is been deleted (Entity which uses File)
• Autohandling streaming, if content is requested
• Testfiles possible
• Lazy deletion of objects and only executed, if transaction is successfull
• Auto configuration with cds plugin feature
• Automatic ETag from S3 for file integrity validation

ETag Support

Files entity includes an eTag field for file integrity validation.

Features:

• eTag field (MD5 hash from S3, nullable for legacy files)
• S3Client and LocalClient automatically return ETag after upload
• ETag is stored in database for integrity validation

Usage:

// API unchanged - works exactly as before
await objectstore.create(content, { filename: 'test.txt' })

// Now with automatic ETag storage
const file = await objectstore.read(ID)
console.log(file.eTag)  // MD5 hash from S3

Notes:

• Existing files will have NULL eTag (handled gracefully)
• New uploads automatically get MD5 hash from S3 (single-part uploads)
• No migration required

Installing

Using npm:

$ npm install @amag-ch/cds-objectstore

Using yarn:

$ yarn add @amag-ch/cds-objectstore

Configuration

{
    "cds": {
        "requires": {
            "objectstore": {
                "impl": "@amag-ch/cds-objectstore",
                "kind": "objectstore"
            }
        }
    }
}

The name and kind must be named objectstore, otherwise CAP cannot inject the credentials from environment

Variant with local objectstore for development testings

{
    "cds": {
        "requires": {
            "objectstore": {
                "impl": "@amag-ch/cds-objectstore",
                "kind": "local-objectstore",
                "[production]": {
                    "kind": "objectstore",
                }
            }
        }
    }
}

The local objectstore is saved in folder ~/.cds-objectstore

Standard configuration (cds-plugin)

{
    "cds": {
        "requires": {
            "objectstore": {
                "impl": "@amag-ch/cds-objectstore",
                "kind": "local-objectstore",
                "[production]": {
                    "kind": "objectstore"
                }
            }
        }
    }
}

Implementation

Database Schema

using {amag.common.objectstore.File as File} from '@amag-ch/cds-objectstore';

entity MyEntity {
    key ID      : UUID;
        file    : File
}

File is definied as Composition. Means if entry of MyEntity is been deleted, also the connected File would be deleted. File is per default defined as attachment (@Core.ContentDisposition.Type: 'attachment'), but could be overwriten with own annotations

Service for add or read files

const objectstore = await cds.connect.to('objectstore')

// Create file - ETag automatically obtained from S3
const ID = await objectstore.create('Any File Content', {
    filename: 'Testfile.txt',
    contentType: 'text/plain'
})

const file = await objectstore.read(ID)
console.log(file.modifiedAt)
console.log(file.eTag)  // MD5 hash from S3

const stream = await objectstore.readContent(ID)

Testing

Create a folder (e.g. test/objectstore) with all needed files named by ID, which is also used in test/data/amag.common.objectstore-Files.csv.

With cds-plugin mode this folder is automatically registered.

Otherwise add file test/init.js with following code snippet to register your folder.

module.exports = async () => {
    require('@amag-ch/cds-objectstore').testing.register(`${__dirname}/objectstore`)
}

License

MIT