EIP Cloud Services Module

Summary

The EIP Cloud Services Module is a comprehensive Node.js package that provides seamless integration with various cloud services, including Redis, AWS S3, CDN, AWS Lambda, AWS SQS, and MySQL. This module is designed to simplify the complexities of interacting with these cloud services, offering a range of functionalities from data caching and storage to content delivery and serverless computing.

Installation and Import

To use this module, first install it in your Node.js project. Then, import the required services as follows:

const { redis, cdn, mysql } = require('eip-cloud-services');

Config Example

Here is an example configuration for the EIP Cloud Services Module. Replace the placeholder values with your actual configuration details.

const packageJson = require('../package.json');

module.exports = {
    cdn: {
        ['cdnName']: { // e.g. "myCdn"
            ['envName1']: { //e.g. "production"
                type: 'google',
                urlMapName: 'YOUR_GCP_URL_MAP_NAME',
                projectId: 'YOUR_GCP_PROJECT_NAME',
            },
            ['envName2']: { // e.g. "staging"
                type: 'amazon',
                distributionId: 'YOUR_DISTRIBUTION_ID',
            }
        },
    },
    redis: { // For regular redis connections
        port: 'YOUR_REDIS_PORT',
        host: 'YOUR_REDIS_HOST',
        prefix: 'YOUR_REDIS_PREFIX',
    },
    redis: { // For redis cluster instances
        prefix: 'YOUR_REDIS_PREFIX',
        clusterEnabled: true,
        cluster: [
            {
                port: 'YOUR_REDIS_PORT',
                host: 'YOUR_REDIS_HOST',
            },
            {
                port: 'YOUR_REDIS_PORT_2',
                host: 'YOUR_REDIS_HOST_2',
            },
            ...
        ]
    },
    s3: {
        Bucket: 'YOUR_S3_BUCKET',
        logs: "verbose", // "verbose" or "outputs", any other value will not log. verbose will log all activity. outputs will only log when there is a file updated.
        logsFunction: ( message ) => { ... } // Optional, if nothing is provided console.log will be used.
    },
    sqs: {
        region: 'eu-west-1',
        endpoint: 'http://localhost:4566' // Optional (e.g. localstack)
    },
    mysql: {
        connectionLimit: 10, // Max connections
        host: 'my-database.domain.com',
        user: 'user',
        password: 'password',
        database: 'my-database', // defaults to undefined
        multipleStatements: false // defaults to true if not set
    }
};

Table of Contents

1. Redis Module
2. AWS S3 Module
3. CDN Module
4. MySQL Module
5. AWS Lambda Module
6. AWS SQS Module
7. AWS Secrets Manager Module

AWS Secrets Manager Module

Overview

This module provides helpers for reading and creating AWS Secrets Manager secrets. Secret reads use a local tmp-directory cache by default to avoid unnecessary AWS requests. Use bypassCache when the caller must force a fresh AWS lookup.

Usage

const { secrets } = require('eip-cloud-services');

const appKey = await secrets.get('my-service/apple-news-app-key');
const configSecret = await secrets.get('my-service/config', { parseJson: true });

Creating Secrets

Existing secrets are never overwritten.

await secrets.set('my-service/apple-news-app-key', 'secret-value');
await secrets.set('my-service/config', { apiKey: 'secret-value' });

CLI usage:

npm run secrets:set -- --name=my-service/apple-news-app-key --value='secret-value'
npm run secrets:set -- --name=my-service/config --json='{"apiKey":"secret-value"}'
npm run secrets:set -- --name=my-service/apple-news-app-key --file=./secret.txt

If --name is omitted, the script uses the current package.json name. If no package name is available, it prompts for one.

Cache Options

By default secrets.get() checks a local tmp cache first and writes AWS results back to that cache. There is no TTL. To force AWS:

await secrets.get('my-service/apple-news-app-key', { bypassCache: true });

Set EIP_SECRETS_CACHE_DIR or secrets.cacheDir to control cache location.

AWS Lambda Module

Overview

This module provides an interface for invoking AWS Lambda functions. It simplifies the process of triggering Lambda functions from your Node.js application, with support for both synchronous and asynchronous invocations.

Installation

Ensure this module is included in your Node.js project. Import the Lambda component as follows:

const lambda = require('eip-cloud-services/lambda');

Usage

Invoking a Lambda Function

You can invoke a Lambda function by specifying the function name and the event payload. The module supports both waiting for the function execution to complete and fire-and-forget invocations.

Synchronous Invocation (Wait for Execution to start)

const response = await lambda.invokeLambda('yourFunctionName', { key: 'value' }, true);
console.log(response);

Asynchronous Invocation (Fire-and-Forget)

await lambda.invokeLambda('yourFunctionName', { key: 'value' }, false);

Parameters

• functionName: The name of the Lambda function to invoke.
• eventPayload: The payload to pass to the function. This can be any valid JSON object.
• waitForExecution (optional): Set to true to wait for the execution to start and receive a response.
• context (optional): The client context for the function execution.
• invocationType (optional): Defaults to 'Event'. Can be set to 'RequestResponse' for synchronous execution.
• logType (optional): The type of logging for the function.

Configuration

Configure the module with your AWS credentials and Lambda settings.

Error Handling

The module includes error handling to manage issues related to Lambda invocation, such as network errors or configuration problems.

Logging

Logging is provided to track the start and completion of Lambda invocations, aiding in debugging and monitoring.

AWS SQS Module

Overview

This module provides helpers for sending, receiving, and deleting messages in AWS SQS queues.

Usage

const sqs = require('eip-cloud-services').sqs;

const queueUrl = await sqs.getQueueUrl('my-queue');
await sqs.sendMessage(queueUrl, { type: 'example', payload: 'hello' });

const messages = await sqs.receiveMessages(queueUrl, { maxMessages: 5, waitTimeSeconds: 10 });
if (messages.length > 0) {
    await sqs.deleteMessages(queueUrl, messages);
}

Configuration

Set sqs.region and optionally sqs.endpoint in your config. The module defaults to eu-west-1 and uses the AWS SDK credential chain.

MySQL Module

Overview

This MySQL module provides a simple and efficient interface for interacting with MySQL databases. It includes functionalities to manage database connections, execute queries, and handle connection pooling.

Installation

Ensure this module is included in your Node.js project. Import the MySQL component as follows:

const mysql = require('eip-cloud-services/mysql');

Usage

Executing Queries

You can execute queries using the query method. This method automatically handles connections and releases them after query execution.

const results = await mysql.query('SELECT * FROM your_table');
console.log(results);

Manually Managing Database Connections

It's not required for you to get connections manually, you can use the methods directly without having to setup connections. Getting a connection should only be used if you need to parse the connection to a third party module for example.

Get a Connection Pool

const pool = mysql.getPool();

Get a Single Connection

const connection = await mysql.getConnection();

Closing the Connection Pool

To gracefully close all connections in the pool:

await mysql.kill();

Configuration

Configure your MySQL connection settings (like host, user, password, etc.) in the config directory.

Error Handling

The module is designed to handle errors gracefully, providing clear error messages for database connection issues and query errors.

Pooling

• The module uses connection pooling for efficient database interaction.
• The pool is automatically created and managed by the module.

CDN Module

Overview

This module provides a shared client for CDN invalidation requests. CDN invalidation requests are sent through the shared Tools API invalidate endpoint, so callers use one centralized route while the backend implementation remains independently updateable inside the invalidation service.

Installation

Ensure this module is included in your Node.js project. Import the CDN component as follows:

const cdn = require('eip-cloud-services/cdn');

Usage

Create a CDN Invalidation

To invalidate cached content in a CDN, use the createInvalidation method. This sends the request to the shared Tools API invalidate endpoint, which executes the configured provider-specific invalidation in eip-cdn-invalidator.

Invalidate in Amazon CloudFront

await cdn.createInvalidation('amazon', 'path/to/your/file.jpg', 'production');

Invalidate in Google Cloud CDN

await cdn.createInvalidation('google', 'path/to/your/file.jpg', 'production');

You can also invalidate multiple paths by passing an array of keys:

await cdn.createInvalidation('amazon', ['file1.jpg', 'file2.jpg'], 'staging');

Configuration

Configure the module with your CDN settings in the config directory. The configuration should include details like project ID, distribution ID, and URL map names for the respective CDNs.

Error Handling

The module is equipped to handle errors gracefully, including validation of input parameters and handling CDN-specific errors.

Advanced Features

• Validates the key argument to ensure proper formatting.
• Constructs invalidation paths based on the provided keys.
• Centralizes invalidation routing through the shared Tools API endpoint.

AWS S3 Module

Overview

This module provides an interface for interacting with AWS S3, offering functionalities like object retrieval, storage, deletion, and more. It supports advanced features such as encryption and cache control directives, making it a versatile tool for managing S3 operations efficiently.

Installation

Ensure this module is included in your Node.js project. Import the S3 component as follows:

const s3 = require('eip-cloud-services/s3');

Usage

Check if an Object Exists

const exists = await s3.exists('myObjectKey');
console.log(exists); // true if exists, false otherwise

Retrieve an Object

const object = await s3.get('myObjectKey');
console.log(object);

Store an Object

await s3.set('myObjectKey', 'myObjectData', { /* options */ });

Delete an Object

await s3.del('myObjectKey');

Copy an Object

await s3.copy('sourceObjectKey', 'destinationObjectKey');

Move an Object

await s3.move('sourceObjectKey', 'destinationObjectKey');

Managing Cache-Control and Encryption

The module provides detailed control over cache behavior and supports encryption for stored objects, allowing you to optimize performance and security.

Configuration

Configure the module with your AWS credentials and preferred settings in the config directory.

Error Handling

The module includes comprehensive error handling, ensuring robust interaction with AWS S3 even in complex scenarios.

Redis Module

Overview

This Redis module provides a robust interface for interacting with Redis, supporting both standard Redis operations and advanced features like Redis Clusters. The module is designed to manage multiple Redis client instances efficiently, allowing seamless operation with different Redis configurations.

Installation

To use this Redis module, first ensure that it is included in your Node.js project. You can import the module as follows:

const redis = require('eip-cloud-services/redis');

Usage

Standard Redis Operations

You can perform standard Redis operations like setting, getting, deleting keys, etc. Here are some examples:

Set a Key

await redis.set('myKey', 'myValue');

Get a Key

const value = await redis.get('myKey');
console.log(value); // Outputs: myValue

Delete a Key

await redis.del('myKey');

Advanced Operations

Using Redis Cluster

If your Redis setup includes a cluster, the module automatically configures the client for cluster operations based on your configuration.

Subscribing to a Channel

redis.subscribe('myChannel', (message) => {
  console.log(`Received: ${message}`);
});

Publishing to a Channel

await redis.publish('myChannel', 'Hello, World!');

Managing Redis Clients

The module internally manages multiple Redis clients for different purposes (like separate clients for pub/sub). However, this is abstracted away from the standard use of the module.

Error Handling

The module is designed to gracefully handle Redis errors, including connection issues and cluster errors.