IHUB Framework

IHUB Framework allows you to quickly build nodejs microservices with support for all the stack used in IHUB.

Table of contents

• Requirements
• Installing
  • Using on Docker
  • Install the latest version
  • Install a specific version
• Getting started
• Usage
  • Core
  • Logger
    • Console logger
    • Loggly logger
    • Elasticsearch/Kibana logger
  • [Request (Axios)](#request-axioshttpsgithubcomaxiosaxios)
  • [Database (Mongoose/MongoDB)](#database-mongoosemongodbhttpmongoosejscom)
    • Models
  • Server (ExpressJS)
    • Routes
  • Tasks (RabbitMQ)
    • Queues
  • Kafka Topics
    • Topics
  • Routines (Cron)
    • Jobs
  • Elastic APM
    • Custom Span
• Troubleshooting
  • Cannot connect to the Docker daemon at unix:///var/run/docker.sock

Requirements

You need to have NodeJS version 8 or later, and to use it you must be inside Infracommerce network or VPN.

Installing

Install the latest version

npm i -S ihub-framework-js.git

Install a specific version

npm i -S [email protected]

Getting started

At the top of your main file, normally that should be the index.js, require the ihub-framework module and initialize it

require('ihub-framework-js')();

ATTENTION: THIS SHOULD BE PLACED AS THE FIRST REQUIRED MODULE IN YOUR APPLICATION OR SOME SERVICES WONT WORK

Since the application starts based on your files inside your components folder, all the functions and modules required should be binded in some way in your tasks.js or routes.js. If for some reason you need to add some loading logic outside this scope you can achieved it like this in your main file:

const ihubf = require('ihub-framework-js');

(async () => {
    await ihubf();

    // Your logic here
    require('some-other-module');
})();

Usage

The IHUB Framework looks for a component folder at the root of your application, without it the application will end with a process zero status.

All services can be enabled by enviroment variables, just create a .env file at the root of your application and enables the services you want, the avaiables services and tools are listed bellow.

Core

We abstract and centralize all the services in the core of the IHUB Framework, that way when we have more control when we need to update or upgrade some service.

You can have access to all the core functions inside any file inside your component in the components folder. To use it simple require the core and start using the functions according to the features you enabled.

const core = require('ihub-framework-js/core');

Logger

The logger component is enabled by default and you can't turn it off.

const { log } = require('ihub-framework-js/core');

// Log an error
log.error('This is an error');

// Log an info
log.info('So much information');

// Log a debug
log.info('So much information');

Console logger

The console logger is only avaiable when the NODE_ENV environment variable is set to development in your .env file, if you want to listen to all the debug logs then you must set the LOG_DEBUG=debug to true.

Loggly logger

The Loggly logger is disable by default, to enabled simple add the required variables into your .env file.

LOGS_LOGGLY_ENABLED=false
LOGS_LOGGLY_INPUTTOKEN=123
LOGS_LOGGLY_SUBDOMAIN=321

Elasticsearch/Kibana logger

The Elastic search logger is disable by default, to enabled simple add the required variables into your .env file.

LOGS_ELASTICSEARCH_ENABLED=true
ELASTICSEARCH_HOST=http://elasticsearch #REQUIRED: Hostname or ip address
ELASTICSEARCH_LOG_PORT=9200 #REQUIRED: Service port

Request (Axios)

The request service is an alias of the axios module and its enabled by default and you can't turn it off

const { request } = require('ihub-framework-js/core');

// Get the google search page
request.get('https://google.com');

// Get the google search page
request.get('https://api.com');

Database (Mongoose/MongoDB)

To enable the database feature simple add the required variables into your ./env file.

FT_DATABASE_ENABLED=true
DATABASE_SYNTAX=mongodb #REQUIRED mongodb or mongodb+srv
DATABASE_HOST=mongo #REQUIRED ostname or ip address
DATABASE_PORT=27017 #REQUIRED port
DATABASE_DBNAME=ihub-framework #REQUIRED database name
DATABASE_USER= #OPTIONAL: username
DATABASE_PASS= #OPTIONAL: password

The database is an alias of the mongoose module and you can access it from the core, see the example bellow.

const { db } = require('ihub-framework-js/core');

// Loads the User model if registered in the application
const User = db.model('User');

// Simple query to get all the users form the User collection
User.find({}, (error, users) => {})

Models

When the database feature is enabled, the IHUB Framework will look for models inside the components. For example, if you have a component named users you can create a folder named models inside and create any models files as you want. The model basic structure can be found in the component/example/model/ExampleModel.js. This files must follow the mongoose scheme model.

const { db } = require('ihub-framework-js/core');
var Schema = db.Schema;

var userSchema = new Schema({
  name:  String,
  email: String,
  password:   String,
});

module.exports = db.model('User', schema);

Server (ExpressJS)

To enable the server feature simple add the required variables into your ./env file.

FT_SERVER_ENABLED=true
SERVER_HTTP_PORT=8080 #REQUIRED: Server port address
JWT_SECRET=123 #OPTIONS: Json Web Token Secret

The server services is an alias of the express module and you can access it from the core, see the example bellow.

const { server } = require('ihub-framework-js/core');

// Closes the web server
server.close();

Routes

When the server feature is enabled, the IHUB Framework will look for routes your components. For example, if you have a component named users you can create a route file named routes.js, the route file must have a required scheme as bellow.

// The exports must be an array of objects
module.exports = [
  {
    method: 'get', // The HTTP method of this route
    path: '/all', // The resource path of this route
    controller: (req, res) => {} //The controller for this route
  },
  {
    method: 'put', // The HTTP method of this route
    path: '/:userId', // Resource path with parameter
    private: true,
    controller: (req, res) => {} //The controller for this route
  }
]

All the routes will have the component folder name as the first resource path, as the case above the routes will be reachable like /users/all, /users/:userId and so on.

Private Routes

If you wish to use private routes, we have support for JSON Web Tokens auth, you first must set the environment variable JWT_SECRET to your secret key, this must be the same as the secret usage to generate the token, then specify in your route that it's a private route.

// The exports must be an array of objects
module.exports = [
  {
    method: 'get', // The HTTP method of this route
    path: '/profile', // The resource path of this route
    private: true, // Enables the JWT middleware for this route
    controller: (req, res) => {} //The controller for this route
  },
]

Tasks (RabbitMQ)

To enable the tasks feature, simply add the required variables into your ./env file.

FT_TASKS_ENABLED=true #REQUIRED: Enable tasks feature

RABBIT_HOST=rabbit #REQUIRED: Hostname or ip address
RABBIT_PORT= # Port number
RABBIT_VHOST_NAME= # Virtual host name
RABBIT_USER= # Username
RABBIT_PASS= # Password

The tasks services is an alias of the amqplib module and you can access it from the core, see the example bellow.

const { tasks } = require('ihub-framework-js/core');

// Send a task to some queue
const payload = 'Some payload';
tasks.send('QUEUE_EXAMPLE', payload);

Queues

When the tasks feature is enabled, the IHUB Framework will look for queues listenners in your components. For example, if you have a component named users you can create a tasks file named tasks.js, the file must have a required scheme as bellow.

module.exports = [
  {
    queue: 'QUEUE_EXAMPLE', //The queue that is going to be listening on
    action: async (payload, done) => { // The action it will execute when some tasks is setted at the queue
      // Do your action here
      console.log(payload);
      // Call done() after everthing is done, await for any database queries or async functions before doing it
      done(); // Sends the ACK to the RabbitMQ
    },
  },
];

Kafka Topics

To enable the kafka feature, simply add the required variables into your ./env file.

FT_KAFKA_ENABLED=true #REQUIRED: to kafka works this environment variable must be equals true.

KAFKA_USERNAME= #username - optional
KAFKA_PASSWORD= #password - optional
KAFKA_BROKERS= #brokers host(s) - REQUIRED.

The send function is an example of library kafkajs see an example below:

 const { kafka } = require('ihub-framework-js/core');

// Send a message to some topic
const record =  {
   topic: 'teste-topic-1', // required
    acks: -1, // optional, default is -1, but you can change as you prefer.
    messages: [
      {
        key: 'key1', // optional, if key is not blank, the messages with the same key always will be sended in the same partition.
        value: { // Required
          teste: 'obj'
        },
      },
    ]
};
await kafka.send(record);

To see more informations about it, access the docs link: kafkajs doc

Topics

When the kafka feature is enabled process.env.FT_KAFKA_ENABLED, the IHUB Framework will estabilish the connection with the brokers and will look for a file named kafka.js inside the components, if this file exists, the IHUB Framework will subscribe inside these topics, and if any of them don't exist, the topic will be created automatically by the IHUB Framework and the partitions will too.

module.exports = [
  {
    topic: "teste-topic-1", // The topic that will be listened.
    partitions: 5, // Number of partitions of this tipic.
    handler: (result) => { // The function that will receive the messaged of this topic.
      console.log(result);
    },
  },
];

Routines (Cron)

Routines services allows you to execute something on a schedule. This is typically done using the cron syntax. It allow you to execute a function whenever your scheduled job triggers.

To enable the routines feature, simply add the required variables into your ./env file.

FT_ROUTINES_ENABLED=true #REQUIRED: Enable routines feature

The routines services is an alias of the cron module and you can access it from the core, see the example bellow.

Jobs

When the routines feature is enabled, the IHUB Framework will look for jobs in your components. For example, if you have a component named users you can create a routines file named routines.js, the file must have a required scheme as bellow.

module.exports = [
  {
    id: 'example.cron', // The identifier of this job
    action: async (done) => {
      // Do your action here...

      done(); // Call done() after everthing is done, await for any database queries or async functions before doing it
    },
    cron: '* * * * *', // The time to fire off your job. This can be in the form of cron syntax or a JS Date object.
    runOnInit: true, // [OPTIONAL] - This will immediately fire your action function as soon as the requisite initialization has happened. This option is set to false by default for backwards compatibility.
  },
];

Elastic APM

To enable the APM feature simple add the required variables into your ./env file.

FT_APM_ENABLED=true
ELASTICSEARCH_APM_HOST=http://elasticsearch-apm #REQUIRED: Hostname or ip address
ELASTICSEARCH_APM_PORT=8200 #REQUIRED: Port address
# ELASTIC_APM_LOG_LEVEL=true #OPTIONAL:
# To enable the APM debugger you can add the variable above, simple by adding it, the debbuger will start, use with caution

This module is very important when going to production, the server, tasks and database services are automatically monitored when transaction happens. The currently automatic transactions monitored are:

• Server Routes: All the routes are monitored including any database query inside your controllers
• Tasks Queue: All queue events are monitored including any database query inside your function

Custom Span

Is recommended to use custom spans for monitoring heavy process tasks, so we can debug latter for performance issues.

const { apm } = require('ihub-framework-js/core');

// Some component route file
module.exports = [
  {
    method: 'post',
    path: '/heavy',
    // This function is monitored by APM
    controller: (req, res) => {
      const span = apm.startSpan('doing some heavy math');
      for (let i = 0; i < 100000; i++) {
        // Some heavy calculation
        i + 0;
      }
      // Ends the span
      if (span) span.end();
    },
  },
]

Troubleshooting

Cannot connect to the Docker daemon at unix:///var/run/docker.sock

If you are trying to code behing the VPN server using docker toolbox or on mac you will need to make a hack to fixit

https://www.shanestillwell.com/2016/04/11/a-hack-to-get-docker-working-while-on-vpn/