🪵 LogVault

A generator of Winston logger instance with pre-defined and configurable transports and formats.

Extra functionality:

• capture console logging methods of your app process
• collect processes names, timestamps, environment name automatically
• send alarms on specific events to Telegram chat or group

Installation

npm install log-vault

or

yarn add log-vault

Usage

Log levels

By the default, max level of logs is set up to "info". All of the levels below will be supressed. You may define maxLevel in declaration of LogLevel class instance or set it up differently for each of the transports. Default levels:

• error
• warn
• info
• http
• verbose
• debug
• silly

Extra data

Default LogVault formatters extract all of the additional data passed with log to "extra" object, which is placed differently for each of the transports.

Metadata

Flat key-value pairs may be used as labels for each of the log entry. There are three keys, passing by the default:

• environment - NODE_ENV env variable of the process
• project - project root directory name (basepath of PWD env var)
• process - npm_package_name env variable (usually equal to "name" value in your project's package.json file).

You may extend the labels information using LogOptions constructor:

import { LogVault, LogOptions } from 'log-vault';

const { logger } = new LogVault()

logger.info(
  "A log record",
  new LogOptions({ meta: { myCustomKey: "value" } })
);

environment, project and process labels will be added automatically, no need to define it manually.

Masking sensitive data

Avoid exposing sensitive data by object keys. The default fields are being masked: "password", "token", "otp" and "secret". You may provide the configuration to the LogVault constructor:

import { LogVault, LogOptions } from 'log-vault';

const { logger } = new LogVault({
  maskOptions: {
    maskLabel: "***",
    fields: [ "secret-field-key" ]
  }
});

Truncating nested data

Limit the nested level of the information to keep you log messages light and simple. There is a configuration that is applied by the default, but you may override this:

import { LogVault, LogOptions } from 'log-vault';

const { logger } = new LogVault({
  truncateOptions: {
    depth: 2,
    stringLength: 1024,
    arrayLength: 5,
    replaceWith: "...(Truncated)"
  }
});

Console transport

import { LogVault } from 'log-vault';

const { logger } = new LogVault().withConsole();

logger.log({ level: "info", message: "Hey, I'm a log entry!" });
logger.warn("It's a warning");
logger.error("Whoops!");

Output:

Uncaught exceptions and promise rejections are captured to the defined transports by the default:

import { LogVault } from 'log-vault';

new LogVault().withConsole();

throw new Error("An error appear");

Output:

Native Console capturing

You may use an option to rewrite the native JS console methods: log, info, warn and error:

import { LogVault } from 'log-vault';

new LogVault().withConsole().captureConsole();

console.log('Something...');

Output:

Files transport

Usage:

import { LogVault } from 'log-vault';

const { logger } = new LogVault().withFiles();

By the default, LogVault will record log files into the "log" folder in the project root directory in json format with timestamp. You may configure placement of the log directory and other log options (see Configuration section).

Mongo transport

import { LogVault } from 'log-vault';

const { logger } = new LogVault().withMongo({
  db: "mongodb+srv://usr:[email protected]/?retryWrites=true&w=majority"
});

Loki transport

Use Loki to export your logs into Grafana.

import { LogVault } from 'log-vault';

const { logger } = new LogVault().withLoki();

Please, take into account that due to Loki nature, it is not recommended to have dynamic labels. By the default, only the process name and Node environment data is passed directly to the Loki labels. All of the remaining metadata will be placed into the "meta" object of the log message.

Example of log entry displayed with the default Grafana "Explore" view:

I advice you to be more creative with the Grafana dashboard setup to reach a display that would be more convenient for your purposes:

Notifications

In some cases, we should be notified with the events occuring in the system. You may set up LogVault Notificator and notification transport to be receive the specific log messages with the defined alarm channels. A Redis v^7 instance running on the machine or distant host is required for this feature.

Sending all logs from the process to the notificator:

const { logger } = new LogVault().withNotifications();

Redis host is considered to run on localhost port 6379 by the default.

Setting up a Notificator instance. You ought to run it in different nodejs process:

const notificator = new Notificator().add(
  new TelegramNotificationChannel({
    token: process.env.TG_BOT_TOKEN,
    chatId: YOUR_CHAT_ID,
    matchPatterns: [
      {
        level: 'error'
      },
      {
        match: {
          message: /error/gi
        }
      }
    ]
  })
);

You may add patterns to match the required log records. If any of the patterns is matched, the log record will be sent to the defined channel.

Supported pattern options:

• level: any of the supported levels
• meta: object containing plain key-value pairs
• message: string or RegExp to match

Currently, only Telegram notification channel is supported. The messages will be sent to a single chat with 5 seconds time gap by the default.

Example Telegram alarm message with the default template:

Notes on usage with microservices or distributed app

• If you use files transport, provide the full path to your logs folder if you want to avoid storing the logs in each app directory
• Use a separate Node process or microservice to serve Notificator instance
• Configure Notificator BullMQ worker to use several threads to increase the productivity
• Use the same queue name for the Notificator across the involved processes

Configuration reference

LogVault class constructor options

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |projectName|Name of your project|string|project root directory name (basepath of PWD env var)| |truncateOptions|Truncate log message data|obj-walker TruncateOptions|{ depth: 5, stringLength: 2048, arrayLength: 12, replaceWith: "...[Truncated]"}| |maskOptions|Masking sensitive fields options|LogVaultMaskFieldsOptions|{ maskLabel: "...[Masked]", fields: ["password", "token", "otp", "secret"] }| |... (rest parameters)|Winston logger creation options|LoggerOptions|{ levels: defaultLevels, level: "http", exitOnError: false, defaultMeta }|

LogVault withConsole method options

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |colors|Winston colors set for console|Winston AbstractConfigSetColors| default colors| |inspectOptions|Options for Node's native util.inspect|InspectOptions|{ compact: false, maxArrayLength: 5, maxStringLength: 2048, depth: 5, colors: true}| |... (rest parameters)| Winston console transport options|ConsoleTransportOptions| LogVault console formats and defaults

LogVault withFiles method options

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |errorLevel|name of the level that is considered as error|string|"error"| |... (rest parameters)| Winston daily rotate file parameters | DailyRotateFile.DailyRotateFileTransportOptions|LogVault file formats and defaults|

LogVault withMongo method options

Only Winston MongoDBConnectionOptions are accepted there. LogVault applies a set of formats and defaults for the convenience.

LogVault withLoki method options

LokiTransportOptions are accepted there. By the default, the "host" parameter is pointing to the localhost 3100 port.

LogVault withNotifications method options

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |name|Project's name for BullMQ Queue|string|LogVault's instance projectName attribute value| |queueOptions|BullMQ Queue options|QueueOptions|{ connection: defaultRedisConnection }| |jobOptions|BullMQ Redis Job options|RedisJobOptions|LogVault notifications Job options|

LogVault captureConsole method options

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |matchLevels|A map of Node's Console methods and Winston instance levels|{ log: string; warn: string; info: string; error: string; }|{ log: "info", warn: "warn", info: "info", error: "error" }|

LogVault uncaptureConsole method

No parameters are required there. Use this method to give back the default methods to Node Console.

LogOptions constructor parameters

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |meta|An Object with flat key-value pairs|Meta|---|

Notificator constructor parameters

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |queueName|Name of LogVault notifications transport queue|string|project root directory name (basepath of PWD env var)| |workerOpts|BullMQ Worker options|WorkerOptions|{ connection: defaultRedisConnection, autorun: true }|

Notificator class methods

• "add" - receives a NotificationChannel class instance
• "stop" - stops Notificator BullMQ Worker
• "run" - resume worker

TelegramNotificationChannel constructor options

| field | desc | type | default value | |-------------------|------------------|------------------|---------------------------| |token|Telegram bot secret|string|none (mandatory)| |chatId|Telegram chat or group id|number|none (mandatory)| |template|Message template in mustache format|string|Basic template| |workerOptions|BullMQ Worker options|WorkerOptions| { limiter: { max: 1, duration: 5000 } } | |matchPatterns|A set of patterns to match in log message to trigger the message sending|MatchPattern|[ ]| |inspectOptions|Node's util.inspect options to render the message object|InspectOptions|{ compact: false, maxArrayLength: 5, maxStringLength: 2048, depth: 5, colors: true}|