Open Age Service Framework

This repository contains a collection of essential services and components for building robust Node.js applications. Each component is designed to handle specific aspects of service architecture, from API handling to offline processing.

Installation

npm install @open-age/service --save

Components

1. API (@open-age/services).api

A framework component that implements essential boilerplate code for REST APIs.

Key Features

• User Authentication with session validation
• Endpoint Authorization
• Response Standardization
• Response Caching
• Response Remapping
• Bulk Request Processing

const oa = require('@open-age/services');
await oa.api.init({
    web: {},
    api: {
        info: {},
        host: 'https://api..../my-service/v1', // The base URL of the API
        prefix: 'api', // The prefix for all API routes. The route will become 'https://api..../my-service/v1/api/...'
        view: './public/specs.html', // The path where the swagger client is located. This will be served from the host url
        middlewares: {}
    }
}, logger)

Web

• port: The port on which the web server will listen
• limit: The maximum request body size, example '200mb'
• view: The view rendering engine to use for rendering views, example 'ejs'

Info

• title: The title of the API, example 'my-service'
• description: A brief description of the API. It will be displayed with the documentation.
• version: The version of the API, example 'v1', 'beta' etc

Middlewares

Auth

{
    "provider": {
        "type": "jwt", // The type of authentication provider (JWT in this case)
        "config": {
            "secret": "...", // The secret key used to sign the JWT
            "expiresIn": 1440 // The expiration time for the JWT
        }
    },
    "validate": [ // Additional validation rules (e.g., IP address validation)
        "ip"
    ]
}

Context

This configuration sets up the context for the application. It specifies attributes and a mapper function to serialize and deserialize the context.

{
    "attributes": {
        "user": {
            "get": async (receivedClaim, context) => {
                // Fetch user based on received claim
            },
            "after": async (populatedClaim, context) => {
                // Perform actions after populating the claim
            }
        }
    },
    "mapper": (context) => {
        return {
            user: context.user
        };
    }
}

Cache

This configuration sets up the cache server using Redis. It specifies the host, port, and options such as password and memory policies. This affects how the application caches responses and manages memory.

{
    "disabled": false,
    "root": "my-service", // The root namespace for the cache
    "provider": {
        "type": "redis/cache", // The type of cache provider (Redis in this case)
        "config": {
            "host": "127.0.0.1", // The host for the cache provider
            "port": 6379, // The port for the cache provider
            "options": {
                "maxmemory-policy": "allkeys-lru", // The memory policy for the cache
                "maxmemory": "1gb", // The maximum memory for the cache
                "enable_offline_queue": false // Whether to enable the offline queue
            }
        }
    }
}

2. Events

A convention-based pub/sub implementation over Redis queue for offline event processing.

Key Features

• Decoupled subscriber architecture
• Runtime subscriber discovery
• Multiple subscribers per action
• Convention-based file locations
• Context preservation
• Configurable serialization

Initializing the Event System

To initialize the event system, you need to configure the queue and subscribers. Here is an example:

const oa = require('@open-age/service');
const logger = oa.logger('app');

await oa.events.init({
    disabled: false, // Enables or disables the queue
    namespace: 'my-service', // It is used as the prefix is appended to all Redis keys used by the queue
    pauseOnError: false, // Determines if the system should pause on error
    concurrency: 4, // The number of concurrent requests to process
    timeout: 30 * 60 * 1000, // Sets the timeout for queue processing
    provider: {},
    queues: {},
    subscribers: [],
    context: {}
}, logger);

• disabled: processing will be done inproc

Provider

• type: Specifies the queue provider type
• config: Provides the queue server configuration

Redis Sample

{
    "provider": {
        "type": "redis/queue",
        "config": {
            "host": "127.0.0.1",
            "port": 6379
        }
    }
}

References:

• https://github.com/redis/ioredis/blob/ca5e940/lib/redis/RedisOptions.ts#L197

Kafka Sample

{
    "provider": {
        "type": "kafka/queue",
        "config": "TODO"
    }
}

Queues

Sample

{
    "default": { 
        "name": "my-app-offline", // Defines the default queue name
        "options": {
            "removeOnComplete": true,
            "removeOnFail": false
        }
    }
}

References:

• https://docs.bullmq.io/guide/queues/auto-removal-of-jobs

Subscribers

subscribers is a collection of subscribers that will be called on each event

{
    code: 'custom-subscriber',
    subscribe: async (event, context) => {
        // do someting
    }
}

Context

Events would serialize and deserialize the context in the payload. It will use the property context to deserialize the context. Here is uses the object

{
    attributes: { 
        // the attributes to be added to the context. It takes objects that gets the claim 
    },
    mapper: (context)=> {
        return {} // lean object representing the context
    }
}

attributes object is a set of objects that return the value to be added to the context.

{
    key: 'role', // the key of the claim
    get: async (recievedClaim, context)=> Object,
    after: async (popoulatedClaim, context)=> void
}

if the key is not specified, the attribute name will be used as key

mapper the function (context)=> Object that returns a lean object representing the context

Queueing Events

To queue an event or process it immediately if queueing is disabled, use the queue method.

Example:

const event = {
    entityType: "user", // The type of entity (e.g., "user", "order").
    action: "create", // The action performed (e.g., "create", "update").
    data: { name: "John Doe", email: "[email protected]" } // The entity the was acted on.
};
const context = { logger: myLogger, processSync: false };
await oa.events.queue(event, context);

It returns a promise that resolves when the event is queued or processed.

Creating a Separate Process to Initialize or Listen for Queues

You can create a separate process to initialize or listen for queues. Here is an example:

const oa = require('@open-age/service');
const logger = oa.logger('queue-listener');

const queueServerConfig = {
    host: '127.0.0.1',
    port: 6379,
    ns: 'my-app',
    options: {}
};

const config = {
    queue: {
        disabled: false,
        timeout: 30 * 60 * 1000,
        queues: {
            default: 'my-app-offline'
        },
        provider: {
            type: 'redis/queue',
            config: queueServerConfig
        }
    }
};

oa.events.init(config, logger).then(() => {
    return oa.events.listen(null, logger);
}).catch((err) => {
    logger.error(err);
    process.exit(1);
});

Subscribing and Handling Events

You can subscribe and handle events in different ways. Here are some examples:

1. Basic Subscription:

   // subscribers/user/created.js
   exports.subscribe = async (event, context) => {
       const logger = context.logger.start('processing user created event');
       // Handle the event
       logger.end();
   };

2. Using Process Method:

   // subscribers/user/created.js
   exports.process = async (data, context) => {
       const logger = context.logger.start('processing user created event');
       // Handle the event
       logger.end();
   };

3. Multiple Subscribers for an Action:

   // subscribers/user/created/sendEmail.js
   exports.subscribe = async (event, context) => {
       const logger = context.logger.start('sending email for user created event');
       // Send email
       logger.end();
   };
   
   // subscribers/user/created/logActivity.js
   exports.subscribe = async (event, context) => {
       const logger = context.logger.start('logging activity for user created event');
       // Log activity
       logger.end();
   };

4. Global Subscribers: Global subscribers can be passed to the events initializer and will be called on each event.

   const globalSubscriber = {
       subscribe: async (event, context) => {
           const logger = context.logger.start('global subscriber handling event');
           // Handle the event
           logger.end();
       }
   };
   
   await oa.events.init({
       // ...existing configuration...
       subscribers: [globalSubscriber]
   }, logger);

3. Client (@open-age/services).client

Client library for user directory interactions.

Configuration

This configuration sets up the directory client with the URL and role key. It affects how the application interacts with the user directory service.

{
    "providers": {
        "directory": {
            "url": "http://api.openage.in/directory/v1/api",
            "role": {
                "key": "<persistent_token>"
            }
        }
    }
}

Usage Example

const client = require('@open-age/services').client;

// Get a role by key
const role = await client.directory.roles.get(key, context);

4. Database (@open-age/services).db

This configuration initializes the database connection and sets up the models with the specified options.

const oa = require('@open-age/services');

global.db = await oa.db.init({
    database: 'my-database',
    provider: {},
    models: {}
}, logger);

Provider

{
    "type": "mongodb",
    "config": {
        "port": 27017,
        "host": "127.0.0.1",
        "options": {
            "useNewUrlParser": true,
            "useUnifiedTopology": true,
            "user": "...",
            "pass": "...",
            "authSource": "admin"
        }
    }
}

Models

{
    "nameFormat": "camelCase",
    "options": {
        "timestamps": true,
        "usePushEach": true
    }
}

5. Telemetry

Telemetry provides insights into the application's performance and usage. It can be configured to collect and report metrics.

Logger

{
    "logger": {
        "level": "info",
        "providers": [
            {
                "type": "console",
                "config": {
                    "handleExceptions": true,
                    "format": {
                        "timestamp": "HH:mm:ss",
                        "json": false,
                        "colorize": {
                            "all": true
                        }
                    }
                }
            }
        ]
    }
}

Usage Example

const oa = require('@open-age/services');
let logger = oa.telemetry.logger('processing');

6. Base

Api

const api = require('@open-age/service').base.api('categories', 'category');

7. Utility

Object

Date

Fields

Template

8. Constants

Folders

It has function to set the folders individually or in bulk. The set folders can be accessed through get. The library get following folders

const oa = require('@open-age/services');

oa.constants.folders.set('api', path.join(appRoot.path, 'api') )
oa.constants.folders.get('api')

// bulk
oa.constants.folders.set([
    { name: 'api', folder: path.join(appRoot.path, 'api') },
])

The library get following folders

• api
• subscribers
• middlewares
• services
• mappers
• models
• uploads
• temp
• specs.paths
• specs.definitions
• public

Errors

The errors module provides a standardized way to handle errors across the application. It allows defining custom error types and messages.

const oa = require('@open-age/services');
const errors = oa.constants.errors;

Here is the list of out of box errors:

• UNKNOWN
• ACCESS_DENIED
• INVALID_TOKEN
• CLAIMS_EXPIRED
• SESSION_EXPIRED
• INVALID_IP
• INVALID_DEVICE
• METHOD_NOT_SUPPORTED
• INVALID_STRING

Can be extended by adding errors to api configuration

{
    "api": {
        "errors": {
            "CUSTOM_ERROR": { 
                "code": "CUSTOM_ERROR", 
                "status": 403, 
                "message": "Custom Message" 
            }
        }
    }
}

Contributing

Please read our contributing guidelines before submitting pull requests.

License

This project is licensed under the MIT License - see the LICENSE file for details.