su-util

Overview

su-util is a utility module designed to standardize reusable code across different microservices. It provides functionalities such as logging, delay mechanisms, thread-local storage, package updates, and configuration management.

Installation

npm install su-util

Usage

Logging Setup

Ensure motifer is installed for logging purposes.

const { ExpressLoggerFactory } = require('motifer');

const Logger = new ExpressLoggerFactory('su-util', 'info');
const logger = Logger.getLogger(__filename);

Importing su-util

const { SearchUnifyUtil } = require('su-util');
const suUtil = new SearchUnifyUtil({ Logger });

Features

1. Delay Handling

The Delay module provides methods to introduce delays in execution.

const testDelay = async () => {
  logger.info('test starting');

  await suUtil.Delay().getDelayTill({
    event: 'test delayTill',
    epochTime: Math.floor((Date.now() + 1 * 1000) / 1000)
  });

  await suUtil.Delay().getDelay({
    timeInMs: 1000,
    event: 'test delay'
  });
};

testDelay();

2. Thread-Local Storage

ThreadLocalStorage provides a way to store and retrieve data within the scope of process.

const testLocalStorage = () => {
  const localStorage = suUtil.ThreadLocalStorage();
  logger.info('Initial storage:', localStorage.getAll());

  localStorage.set('t1', 'k1', 'v1');
  localStorage.set('t1', 'k2', 'v2');
  localStorage.set('t2', 'k3', 'v2');

  logger.info("localStorage.get('t1', 'k1')", localStorage.get('t1', 'k1'));
  logger.info('All storage:', localStorage.getAll());
};

testLocalStorage();

3. Tenant Aware Local Storage

TenantAwareThreadLocal this class internally uses ThreadLocalStorage. This util is specifically created for processes or scenarios where we only have data for one tenant only.

const testTenantLocalStorage = () => {
  const tenantStorage = suUtil.TenantAwareThreadLocal();
  tenantStorage.setTenantId('t1');
  tenantStorage.set('k1', 'v1');
  tenantStorage.set('k2', 'v2');
  logger.info('tenantStorage.getTenantId()', tenantStorage.getTenantId());
  logger.info("tenantStorage.get('k1')", tenantStorage.get('k1'));
  logger.info('tenantStorage.get()', tenantStorage.getAll());
};

testTenantLocalStorage();

4. Package Update Utility

This utility helps update properties in the package.json file dynamically.

This util will be used to update the start:prod command in microservices if we need to add options like --max-old-space-size while starting the server.

Steps to achieve this:

1. There should be a new property in the .env file that contains the options (envProperty).
   Example: STARTUP_OPTIONS=--max-old-space-size=4096

2. Add a placeholder string that will be replaced by the utility (toReplaceString).
   Example:

   "start": "node index.js"

   should be changed to:

   "start": "node placeholder index.js"

3. Create a JavaScript file with the following code snippet:

4. Add a new script in package.json that executes this newly created file.

const testUpdatePackage = async () => {
  logger.info('test starting');

  await suUtil.UpdatePackage({
    envProperty: 'STARTUP_OPTIONS',
    packageProperty: ['scripts', 'start:prod'], // Complete path of property to be edited
    packagePath: 'package.json',
    toReplaceString: 'placeholder'
  }).run();
};

testUpdatePackage();

5. Configuration Cloning Utility

This Util is specifically created because sequelize doesn't pick NODE_ENV. here we can create a new config file or override default.json file.

const testConfigModule = async () => {
  logger.info('test starting');

  await suUtil.ConfigModuleClone({
    envConfig: 'envconfig',
    newConfig: 'newconfig'
  }).run();
};

testConfigModule();

6. FS Util module

This Util is created for fs related functions.

const testFsUtil = async () => {
  const fsUtil = suUtil.FsUtil();
  fsUtil.createDirectories(['a/b/c', 'e/f/g']);

  const size = fsUtil.getTextSize('random text string', suUtil.Constants().UNITS.kb);
  logger.info('string size', size);
  const folderSize = await fsUtil.getFileOrDirSize('src/constants/', suUtil.Constants().UNITS.kb);
  logger.info('folderSize', folderSize);
};

testFsUtil();

7. Email Util

This Util is created for sending email.

const emailUtil = suUtil.emailUtil({
  region: 'region',
  accessKeyId: 'accessKeyId',
  secretAccessKey: 'secretAccessKey'
});

const sendMail = async () => {
  await emailUtil.sendEmail({
    from: '[email protected]',
    to: ['[email protected]'],
    cc: ['[email protected]'],
    bcc: ['[email protected]'],
    subject: 'test subject',
    text: 'this is a test email', // text or html is mandatory
    html: '<h1>Test </h1><br><br><span>test email/span>', // if both are passed html takes precedence
    attachments: [
      {
        filename: 'new-file.txt', //filename to be displayed in the email
        content: fs.readFileSync('new-file.txt') //filepath
      },
      {
        filename: '2nd-file.txt',
        content: fs.readFileSync('2nd-file.txt')
      }
    ]
  });
};

sendMail();

8. Kafka Util

This util is used for kafka initialization, consumer creation, topic subscription. Existing kafka lib can be modified to use su-util's kafka lib

// Sets up Kafka client, connects the producer, creates topics
// and subscribes consumers if provided.
const initializekafka = async ({ clientId, brokers, retryConfig }) => {
  try {
    await suUtil.KafkaUtil().initializeKafka({
      clientId,
      brokers,
      retryConfig,
      topicsToCreate, // Topics to create
      subscribeConsumersFn: consumers.subscribeConsumers // Function to subscribe all Kafka consumers
    });
  } catch (e) {
    logger.error('error in kafka init', e);
    throw e;
  }
};

// Publishes messages to the specified Kafka topic via suUtil KafkaUtil.
const publishMessage = async ({ topic, messages }) => suUtil.KafkaUtil().publishMessage({
  topic,
  messages
});

// Creates and returns a Kafka consumer for the given topic and consumer group using suUtil KafkaUtil.
const getConsumer = async ({ groupId, topic, otherproperties }) => suUtil.KafkaUtil().getConsumer({
  groupId,
  topic,
  otherproperties
});

// Creates a Kafka topic using suUtil KafkaUtil
const createTopic = async (topics) => {
  try {
    await suUtil.KafkaUtil().createTopics(topics);
  } catch (e) {
    logger.error(e);
  }
};

8. Setup su-util locally

Refer to local-setup-readme.md to know how to setup su-util on local environment and use it in other microservices.

Contributing

Please create a merge request if you would like to contribute or report issues via GitLab Issues.

License

MIT