Summary

This 'dbmanager' module makes it easy to connect to mongo from a custom-built app within Itential Automation Platform (IAP).

This is an extremely thin wrapper over mongo's native node driver: https://docs.mongodb.com/ecosystem/drivers/node/. The only reason this exists is so that:

1. You have a standardized way to store/manage mongo config within Itential Automation Platform (IAP)
2. You don't have to know/research any best practices for connecting to mongo
3. You can copy/paste code examples from example-cog.js for most common use cases

When/Why would I use this?

• downsides of a mongo adapter
  • data type serialization (no native Date objects for example)
  • very unscalable for large IAP ecosystems
  • limited API
• upsides of a mongo adapter
  • available in workflow builder

Installation/Usage

Inside your custom app's code, run npm install --save @itentialopensource/dbmanager

Also, the app or service using dbmanager must install the desired version of mongodb. Running npm install --save mongodb will install latest version of mongodb driver.

Then make sure to follow the "Configuring dbmanager" section below.

Then see the example cog file for examples on how to use dbmanager from your app's javascript code.

You can view the API docs

Configuring dbmanager

There are two things the mongo node driver (and therefore dbmanager) needs to connect to mongo:

• a connection string
• connection options (an object)

If you configure your connection settings in the right place (see below sections), then dbmanager will automatically detect those settings without you having to specify where they are.

Example Configuration Using SSL and Replica Sets

Here's an example dbmanager configuration if your mongo server is using ssl and replica sets. You'll notice that the password is encrypted using IAP's encrypt.js script. To use that script, log into a machine that has IAP installed, then cd into /opt/itential/current/node_modules/@itential/pronghorn-core/utils, and then run node encrypt.js your-password-here. The script will print an encrypted value of that password which you can copy/paste.

Where exactly you put this config depends on the version of IAP you're running (see sections below).

"dbManagerConfig" {
  "default": {
    "connectionString": 'mongodb://node1:27017,node2:27017,node2:27017?replicaSet=rs0',
    "connectionOptions": {
      "auth": {
        "user": "customerMongoUser",
        "password": "$ENC93eb9439537ae34196db49409dd0261a8b87218fafd0419"
      }
      "ssl": true,
      "sslValidate": false,
      "sslCA": './keys/mongodb-server.pem',
      "checkServerIdentity": true
    }
  },
}

You do not need to specify the database name being used. When you connect to this from javascript code, you specify there which database name you want to connect to. This way you can use multiple databases from within the same connection.

Notice that this format differs from what you would put into IAP's mongoProps. The reason it differs is because we want to be a pass-through to mongo's native node API, and MongoProps data format is a customized and different than the format mongodb's node driver expects. Because dbmanager is a pass-through to mongodb, this allows you, as the user of dbmanager, to have the full power of mongo's node driver at your disposal.

See http://mongodb.github.io/node-mongodb-native/3.2/reference/connecting/ for more details about how 'connectionString' and 'connectionOptions' is formatted.

Where to Put the Config (For IAP Versions 2019.2 or Later)

If you have IAP version 2019.2 or later, then you can edit properties specific to each app you write.

• Open up Itential
• Log in as an admin
• Go to Settings > Services
• Select your app from the services list that appears on the left
• add a new "dbManagerConfig" section to the properties object

Therefore, when you're finished adding "dbManagerConfig" to your app's service properties, the entire service config you see on screen will look something like below. Notice that "dbManagerConfig" is inside the "properties" section.

{
    "loggerProps": {
        "description": "Logging",
        "log_max_files": 100,
        "log_max_file_size": 1048576,
        "log_level": "warn",
        "log_directory": "/var/opt/logs",
        "log_filename": "MyApp.log",
        "console_level": "trace"
    },
    "isEncrypted": true,
    "model": "@custom/my-app",
    "name": "MyApp",
    "type": "Application",
    "properties": {
      "dbManagerConfig": {
        "default": {
          "connectionString": "mongodb://127.0.0.1:27017",
          "connectionOptions": {}
        },
        "connection2": {
          "connectionString": "mongodb://some.other.host",
          "connectionOptions": {}
        }
      }
    },
    "rabbitmq": {
        "protocol": "amqp",
        "port": 5672,
        "username": "username",
        "password": "password",
        "locale": "en_US",
        "frameMax": 0,
        "heartbeat": 0,
        "vhost": "/",
        "certPath": "",
        "keyPath": "",
        "passphrase": "passphrase",
        "caPath": "",
        "hosts": [
            "localhost"
        ]
    }
}

Where to Put the Config (For IAP Versions Prior to 2019.2)

For IAP versions prior to 2019.2, IAP reads all configuration from a file called properties.json. The dbmanager tool will look inside properties.json for a top-level property called "dbManagerConfig".

Therefore, you must edit properties.json file to include a "dbManagerConfig" section. You must edit the properties.json file for every IAP server (if you are using Itential HA, this will be multiple machines). The default location is /opt/itential/current/properties.json .

When you are finished adding dbManagerConfig to properties.json, the properties.json file will look something like this:

{
  "dbManagerConfig": {
    "default": {
      "connectionString": 'mongodb://node1:27017,node2:27017,node2:27017?replicaSet=rs0'
      "connectionOptions": {
          "auth": {
              "user": "customerMongoUser"
              "password": "$ENC:abcdef12345"
          }
          "ssl": true,
          "sslValidate": false,
          "sslCA": './keys/mongodb-server.pem'
          "checkServerIdentity": true,
          "dbName": 'customer_database',
      }
    },
    "connection2": {
      "connectionString": "mongodb://127.0.0.1:27017",
      "connectionOptions": {},
    }
  },
  "applicationProps": {
    "directory": "./node_modules/"
  },
  "uiProps": {
    "description": "UI",
    "layout": "node_modules/@itential/pronghorn-core/ui/views/layout.jade",
    "home": "node_modules/@itential/pronghorn-core/ui/views/home.jade",
    "login": "node_modules/@itential/pronghorn-core/ui/views/login.jade",
    "profile": "node_modules/@itential/pronghorn-core/ui/views/profile.jade",
    "user_config": "node_modules/@itential/pronghorn-core/ui/views/user_config.jade",
    "group_config": "node_modules/@itential/pronghorn-core/ui/views/group_config.jade",
    "new_user": "node_modules/@itential/pronghorn-core/ui/views/dialogs/new_user.jade",
    "edit_user": "node_modules/@itential/pronghorn-core/ui/views/dialogs/edit_user.jade",
    "new_group": "node_modules/@itential/pronghorn-core/ui/views/dialogs/new_group.jade",
    "fav_icon": "node_modules/@itential/pronghorn-core/ui/img/favicon.ico"
  },

  ...

Special Note For Those Migrating From 2019.1 to 2019.2 or Later

2019.2 fundamentally improved how all configuration works in IAP.

Therefore, if you are migrating from 2019.1 (or earlier) to 2019.2 or later, IAP will come with a migration script. This script will take any properties inside properties.json and add them to a "profile" (which resides directly in mongo; not a file anymore). If you have a running instance of IAP using 2019.2 or later, you can log in as an admin and navigate to Settings > Profiles to see profile config.

The dbmanager tool is smart enough to "just work" in this situation. If you already configured dbmanager for 2019.1, and you run the migration script, dbamanager will know to look inside the active "profile" and find all your pre-existing dbManagerConfig configuration without you having to do anything (other than run the migration script).

However, the intent of 2019.2 is to have every app ("service") contain its own configuration. So you will probably want to update each of your apps' configuration with dbManagerConfig at some point.

Any dbManagerConfig you set for that specific service (that is, set within a service's config) will override anything migrated into the "profile" config.

Advanced Usage

There are a few more advanced ways to use this module, but they require a bit more understanding to ensure you don't negate any value from caching connections. It's expensive to keep opening/closing connections, so dbmanager caches connections. Therefore, you should never use the following function unless you understand how it affects caching.

You can attach to mongo events via

const client = getConnection();
client.on(...)

If you want to manually close a client, use the closeConnection function.

Contributing

Your code contribution is welcomed! To contribute:

• You must run npm test locally (with docker installed to run integration tests) and everything must pass before being accepted into master. The pipeline will fail if tests do not pass.

• See development notes

License

Itential, LLC proprietary