Logging Microservice for Node.js / ES2017 V2

This is a logging microservice from Pip.Services library. It collects execution logs from distributed microservices, stores and provides a single entry point to read all of them.

The microservice currently supports the following deployment options:

• Deployment platforms: Standalone Process
• External APIs: HTTP/REST
• Persistence: Memory

This microservice has no dependencies on other microservices.

Quick Links

• Use
• Download
• Develop
• Configure
• Run
• Source Documentation
• Node.js Client Library

Contract

Logical contract of the microservice is presented below. For physical implementation (HTTP/REST), please, refer to documentation of the specific protocol.

enum LogLevel {
  None = 0,
  Fatal,
  Error,
  Warn,
  Info,
  Debug,
  Trace,
}

class LogMessageV1 {
  public time: Date;
  public source: string;
  public level: LogLevel;
  public traceId: string;
  public error: ErrorDescription;
  public message: string;
}

interface ILoggingV1 {
  readMessages(
    context: IContext: string,
    filter: FilterParams,
    paging: PagingParams
  ): Promise<DataPage<LogMessageV1>>;

  readErrors(
    context: IContext: string,
    filter: FilterParams,
    paging: PagingParams
  ): Promise<DataPage<LogMessageV1>>;

  writeMessage(
    context: IContext: string,
    message: LogMessageV1
  ): Promise<LogMessageV1>;

  writeMessages(context: IContext: string, messages: LogMessageV1[]): Promise<void>;

  clear(context: IContext: string): Promise<void>;
}

Download

Right now the only way to get the microservice is to check it out directly from github repository:

git clone [email protected]:entinco/eic-services-infrastructure2.git

Pip.Service team is working to implement packaging and make stable releases available as zip downloadable archives.

Develop

Environment Setup

This is a Node.js project and you have to install Node.js tools. You can download them from official Node.js website: https://nodejs.org/en/download/

After node is installed you can check it by running the following command:

node -version

Then you need to configure node tools:

# Install typescript compiler
npm install typescript -g

# Install mocha test runner
npm install mocha -g

To work with GitHub code repository you need to install Git from: https://git-scm.com/downloads

If you are planning to develop and test using persistent storages other than flat files you may need to install database servers:

• Download and install MongoDB database from https://www.mongodb.org/downloads

Installing

After your environment is ready you can check out microservice source code from the Github repository:

git clone [email protected]:entinco/eic-services-infrastructure.git

Then go to the project folder and install dependent modules:

# Install dependencies
npm install

If you worked with the microservice before you can check out latest changes and update the dependencies:

# Update source code updates from github
git pull

# Update dependencies
npm update

Building

This microservice is written in TypeScript language which is transcompiled into JavaScript. So, if you make changes to the source code you need to compile it before running or committing to github. The process will output compiled javascript files into /bin folder.

tsc

When you do continuous edit-build-test cycle, you can run typescript compiler with --watch option to detect and compile changes you make automatically:

tsc --watch

Testing

Before you execute tests you need to set configuration options in config.yaml file. As a starting point you can use example from config.example.yaml:

copy config.example.yaml config.yaml

After that check all configuration options. Specifically, pay attention to connection options for database and dependent microservices. For more information check Configuration Guide

Command to run unit tests:

npm test

You can also execute benchmarks as:

npm run benchmark

Configure

Configuration structure follows the standard configuration structure.

Persistence

This microservice currently supports only in-memory persistence. In the future other types of persistence will be added like file, AWS S3, etc.

For more information on this section read Pip.Controllers Configuration Guide

Memory

Memory persistence has the following configuration properties:

• options: object - Misc configuration options
  • max_page_size: number - Maximum number of items per page (default: 100)
  • max_error_size: number - Maximum number of stored errors (default: 1000)
  • max_total_size: number - Maximum number of all stored messages (default: 10000)

Example:

- descriptor: "example:persistence:memory:*:1.0"
  options:
    max_page_size: 100
    max_error_size: 100
    max_total_size: 1000

Service

The service component contains the core business logic of the microservice. Besides component descriptor it doesn't expect configuration options.

Example:

- descriptor: "example:service:default:default:1.0"

Controller

The controller components (also called endpoints) expose external microservice API for the consumers. Each microservice can expose multiple APIs (HTTP/REST) and multiple versions of the same API type. At least one service is required for the microservice to run successfully.

Http

HTTP service has the following configuration properties:

• connection: object - HTTP transport configuration options
  • type: string - HTTP protocol - 'http' or 'https' (default is 'http')
  • host: string - IP address/hostname binding (default is '0.0.0.0')
  • port: number - HTTP port number

Example:

- descriptor: "example:controller:commandable-http:*:1.0"
  connection:
    protocol: "http"
    host: "0.0.0.0"
    port: 3000

For more information on this section read Pip.Controllers Configuration Guide

Run

Add config.yaml file to the config folder and set configuration parameters.

Example of microservice configuration:

- descriptor: "pip-services:context-info:default:default:1.0"
  name: "service-logging2"
  description: "Logging microservice for pip-services V2"

- descriptor: "pip-services:logger:console:default:1.0"
  level: "trace"

- descriptor: "service-logging:persistence:memory:default:1.0"

- descriptor: "service-logging:service:default:default:1.0"

- descriptor: "service-logging:controller:commandable-http:default:1.0"
  connection:
    protocol: "http"
    host: "0.0.0.0"
    port: 8080

Start the microservice using the command:

node ./bin/main.js

Use

The easiest way to work with the microservice is to use client library. The complete list of available client libraries for different languages is listed in the Quick Links.

If you use Node.js then you should add dependency to the client library into package.json file of your project:

{
    dependencies: {
        ...
        "client-logging2-node": "^1.0.*"
        ...
    }
}

Inside your code get the reference to the client library:

var sdk = new require("client-logging2-node");

Define client configuration parameters that match configuration of the microservice external API:

// Client configuration
var config = {
  connection: {
    protocol: "http",
    host: "localhost",
    port: 8080,
  },
};

Instantiate the client and open connection to the microservice:

// Create the client instance
var client = sdk.LoggingHttpClientV1(config);

// Connect to the microservice
try {
    await client.open(null);
    // Work with the microservice
    ...
} catch(err) {
    console.error('Connection to the microservice failed');
    console.error(err);
}

Now the client is ready to perform operations:

// Log a message
let message = await client.writeMessage(null, {
  time: new Date(),
  level: 2,
  message: "Restarted server",
});

var now = new Date();

// Get the list system activities
let page = await client.readMessages(
  null,
  {
    from_time: new Date(now.getTime() - 24 * 3600 * 1000),
    to_time: now,
    search: "server",
  },
  {
    total: true,
    skip: 0,
    take: 10,
  }
);

Acknowledgements

This microservice was created by Sergey Seroukhov and is currently maintained by Danylo Kuznetsov.