config-manager

Simple config manager to retrieve configurations stored in DynamoDB.

Provides a consistent method for storing and retrieving credentials as well as delegating access control to IAM, as opposed to individual services managing how they handle configurations and sensitive information. In doing so, system administrators can control all configurations through AWS' IAM and DynamoDB exclusively (Including fine grained control) and need not worry about the details of each project; They need simply to provide an IAM user/role to developers.

More specifically

1. System admins create and populate a DynamoDB table that contains configurations (This table should be locked down and accessible by no one except sys admin)
2. System admins then use IAM users, roles and groups that enable access to these DynamoDB tables. Fine grained control can be specified to provide access at a record level.
3. System admins provide developers with these IAM users/roles as well as the keys of what is stored in DynamoDB. Developers also install lukes-config-manager into their project.
4. Developers now have access to configurations via IAM user/role and a means access them easily via lukes-config-manager. Use
   • IAM users: when developing locally. User stored in ~/.aws/credentials
   • IAM roles: when projects have been deployed

Therefore

• By developers storing IAM user credentials locally (~/.aws/credentials) and using roles once deployed, the project itself stores and knows nothing sensitive, but has access to the configurations it requires.
• Developers do not need to worry about how they should setup and manage configurations for new projects as there is an established pattern.
• System admins can manage configs in a central location and need not involve developers or deployments when changes are necessary.
• System admins always have control over what configurations services have access to. If at any time permissions need to be changed for a user/role, system admins can do so quickly in IAM.

Architecture

How to use

var ConfigManager = require('lukes-config-manager/dist');

const options = {
    // Mandatory
    dynamoTable: 'development_config', // Uses table 'development_config'

    // Optional
    iamUser: 'myIamUser', // User in ~/.aws/credentials. Dont set if 'IAM user' already set by process ||  'IAM role' is being used
    region: 'ap-southeast-2' // Defaults to 'ap-southeast-2''
}
var configManager = new ConfigManager(options)

const { config } = configManager.getConfig('mySQL'); // Fetch 'config' where resource = 'mySQL'. If second param not provided, 'config' is used by default. 
const { sshUsername } = configManager.getConfig('mySQL', ['sshUsername']); // Fetch 'sshUsername' where resource = 'mySQL'

Expected DynamoDB structure

| development_config         (Table)
| -> mySQL                    (Map)
|    -> config                  (Map: General config)
|       -> address                (String)
|       -> port                   (String)
|       -> username               (String)
|       -> password               (String)
|    -> sshUsername             (String: Other configs)
|    ...other configs

Motivation

Problem

Initially I had an EC2 running MySQL and a single service connecting to it. No problem.

I then introducted a second service which also needed access to MySQL. I now need to

1. Define another method to store and handle the same MySQL config
2. Handle the maintenance overhead of the DB details being stored in two places

Thoughts

This got me thinking: Could we decouple configuration from the service that uses it so we can reuse the same configuration across multiple services? I figured yes: We could store the details of MySQL in DynamoDB and have the services that need access to MySQL, fetch the details from there.

DynamoDB would then need to be locked down for security so config is only provided to authenticated services. This is achieved through IAM users and roles. Users for local development / Roles for deployed services in AWS.

Solution

Store all configuration details in DynamoDB and expose them via IAM users and roles, which are then assigned to deployed / local services for access. Each service / project will import 'lukes-config-manager' which encapsulates how the DynamoDB store is interacted with.

Considerations

• Configurations will be cached once retrieved. If configurations change in Dynamo, the old configurations will still be cached/being used on the consuming service. Additional work is required to clear cache periodically or for an alternative solution.
• Latency on initial call to retrieve config, before being stored in cache

Deploying a new version

• Bump version in package.json
• npm run-script build
• npm publish