Curity - Docker Environment Provisioning

This repository contains configuration files that enable the provisioning of a Curity environment based on either:

• Docker Compose
• Dockerfile

Docker Compose

Docker Compose is primarily intended for use in local developer environments.

This information is based on the official Curity HOWTO Guide - Clustering with Docker Compose but has been tailored to support Direct Wines installations.

There are two top-level Docker Compose files which support the following use cases:

• The docker-compose-standalone.yml is the best option for developers working locally.
• The docker-compose-clustered.yml supports running Curity in Cluster Mode. This emulates how Curity will run in an AWS environment with separate admin and runtime nodes, making it very useful for testing.

Dockerfile

A vanilla Dockerfile is primarily intended for deployments to AWS ECS/Fargate.

NOTE: This is because Docker Compose isn't supported for deploying to AWS ECS/Fargate clusters.

There are a pair of Dockerfiles named Docker.admin and Docker.runtime to support running Curity in an AWS ECS/Fargate environment. They can also be used in a local developer environment for testing.

Direct Wines Per-Environment Configuration

As well as the variation of running as a clustered or standalone environment, Direct Wines support different pre-production environment as well as a production environment. These are named as local, dev, sit, uat and prod where

• local, is a local development environment.
• dev, is a shared development environment, which is predominantly used as a CI environment or for AWS development support.
• sit is the System Integration environment.
• uat is the User Acceptance environment.
• prod is the Production environment.

The Direct Wines Curity Configuration is typically maintained in a common Curity Configuration file named curity-configuration.xml.

The ./config/curity-confg.xml file is setup using the Curity Parameterized XML feature. This simply means that the XML contains placeholder variables which are merged with environment variables.

The repository then provides a set of environment files named xxx.env for the different enviroments. The default environment file is local.env so developers typically won't need to configured this further.

Environment File Formats

• Docker Compose, uses an environment file format known as the DOTENV file format
• Docker Run and by extension AWS ECS/Fargate use a more basic form of environment file format. Notably, this more basic form doesn't support Multiline Strings or Quoted String values. There is no official name for the format, so it is referred to in these docs as the PLAINENV format.

So beware the following GOTCHA. These formats are incompatible! There is also no way to automatically convert, a manual conversion is required. Therefore you should be aware that:-

• local.env, is in the DOTENV format as it needs to be used with Docker Compose.
• dev.env,sit.env,uat.env and prod.env are in the PLAINENV format as it needs to be used with Docker Run and by extension AWS ECS/Fargate
• local.plain.env, is in the PLAINENV format to provide an example of the required manual conversions.

Operating the Clustered Environment

The Clustered Environment can be setup with either :-

1. Docker Compose OR
2. With a pair of Dockerfiles named Docker.admin and Docker.runtime.

The Docker Compose option is easier to work with in a local Linux or WSL developer environment. The Dockerfile option is required to use in an AWS ECS Fargate environment. However they can also be used and tested in a developer environment

Docker Compose Usage

# Assuming we start in the parent directory of the repository
export COMPOSE_PROJECT_NAME=curity
docker compose -f docker-compose-clustered.yml up -d

# Follow the logs in the admin service
docker compose logs -f admin

Dockerfile Usage

export COMPOSE_PROJECT_NAME=curity

# Docker Build steps. Build 1 image for the admin node and 1 for the runtime node
docker build . -t "dw/curity/runtime" -f Dockerfile.runtime
docker build . -t "dw/curity/admin" -f Dockerfile.admin

# Docker Run Steps

# TODO:- For local installation ONLY. This requires a custom network to take vadvantage of Dockers inbuilt DNS Environment See (https://docs.docker.com/config/containers/container-networking/#dns-services)

# This will ensure that the runtime node and the admin node can see wach other
docker run -d -p 8443:8443 --name curity-runtime-1 dw/curity/runtime
docker run -d -p 6749:6749 --name curity-admin --hostname admin dw/curity/admin

# For local installation ONLY:- To diagnose issues connect to the container. in this example we connect to the runtime node

docker exec -it curity-runtime-1 /bin/bash
docker logs curity-admin -f

Operating the Standalone Environment

The standalone environment is the default setup for developers working locally on a WSL or Linux environment, whether on a laptop or desktop computer.

Two versions of the standalone environment are available, each supported through different Docker profiles. Both versions are fully featured: one operates with ngrok, and the other operates without ngrok.

Operating the Standalone Environment with ngrok

Using ngrok is generally recommended for developers working on Curity, as it facilitates testing with oauth.tools and is helpful for demonstrations or troubleshooting with Curity support.

See Local Development Setup with ngrok for complete details

Operating the Standalone Environment without ngrok

The Standalone Environment setup using Docker Compose simplifies the process of starting the necessary containers under the same network. This setup includes:

1. The latest version of the Curity Identity Server
2. A local version of DynamoDB

Important Notes:

• The Curity DynamoDB schema must be provisioned into any local environment. Follow the instructions in the section database scripts if you haven't done this already.
• For users who do not require ngrok, the Curity Reference Application is recommended as an alternative tool for testing.

Docker Compose Usage (without ngrok)

# Assuming we start in the parent directory of the repository
export COMPOSE_PROJECT_NAME=curity
# Additional IDSVR Setup
export BASE_URL="<YOUR LOCAL HOST OR ALTERNATIVE HOST FOR TESTING>"

docker compose -f docker-compose-standalone.yml up -d

# Follow the logs in the curity admin service (change service name to dynamodb to view logs in that service)
docker compose logs -f admin

Customizing Logging for Development Debugging

1. You can globally enable DEBUG logging by setting LOGGING_LEVEL=DEBUG via an environment variable.
2. This approach can generate a lot of log noise, so a more refined method is available:
   • Uncomment the volume mapping for log4j2.xml in docker-compose-standalone.yml.
   • In the log4j2.xml file, keep the overall log level at INFO, then add: This way, you see more detailed logs only for specific classes (in this example, for API routes).

Operating the Standalone Environment with ngrok

Using ngrok is recommended for developers working on Curity, as it facilitates testing with oauth.tools and is helpful for demonstrations or troubleshooting with Curity support.

See Local Development Setup with ngrok for more details

Deploying to AWS ECS/Fargate in a Direct Wines Account.

The Curity release process to AWS environments at Direct Wines is managed by the Tech Services team. That process is described here: https://direct-wines.atlassian.net/wiki/spaces/DC/pages/2131425/Curity+Release+Process.

As this README is focused on deployments as part of development activities, that process won't be described further, with one exception shown below:

Dockerfile Usage to test environment files that will used in the AWS ECS environment

As described in the section on Environment File formats, there are 2 forms of environment file is use, and unfortunately they are incompatible!

The AWS ECS service uses the environment file format supported by the docker run command.

This variant is particularly fragile to any structural errors in the .env file. It can typically mean a corrupt curity-conf.xml file and the server will then not start. Thsi is to be avoided at all costs as it inflicts a time-consuming diagnosis and redeployment exercise.

Fortunately the standalone environment can be used to test any configuration file changes to the .env files, but by using the docker run command rather than the docker compose command

The following development process is therefore recommended in this instance:-

1. Use docker compose to development and test new features and configuration file changes, using the local.env file
2. If a configuration file has changed as part of task, incorporate the changes from local.env into the dev.env, sit.env, uat.env, and prod,env files.
3. Run the following commands to verify no breaking chnages to the env file

# Build the image
docker build . -t "dw/curity" -f Dockerfile.admin
#  Test the changes with the dev.env file
docker run  --env-file dev.env -p 6749:6749 --name curity-admin --hostname admin dw/curity
#   Check the logs,  if the server starts normally,  BIG TICK,  if not the logs should show the error
#
#   Repeat for sit.env, uat.env and prod.env (BUT see the NOTE below on repeating the tests.)

#   NOTE: Follow these steps after starting the container with the 'docker run'
#         command:
#   - Use either Ctrl-C to stop the container OR start another terminal window
#     and use the 'docker stop' command.
#   - Once the container is stopped, use the 'docker rm' command or the 'docker
#     system prune' command to remove the container.
#   - Once the container is stopped and removed, you can repeat the test.

Developer Preperation Tasks

Before committing and code commits

• Please run

# Install precommit hook to ensure environment files have spaces removed
npm run prepare

Database Scripts

Run Local DynamoDB Create Table Scripts

# Run the create table script for the respective brand
cd ./database-setup
sh local-create-tables.sh --brand uklait

We recommend using the dynamodb-admin admin tool in local environments to check that your tables are created accurately.

Run AWS DynamoDB Create Table Scripts

# Run the create table script for the respective brand in AWS enviroinment
cd ./database-setup

# SIT
sh dw-sit-create-tables.sh --brand uklait

# UAT
sh dw-uat-create-tables.sh --brand uklait

Conventional Commits

The Conventional Commits specification is a lightweight convention on top of commit messages. It provides an easy set of rules for creating an explicit commit history; which makes it easier to write automated tools on top of. This convention dovetails with SemVer, by describing the features, fixes, and breaking changes made in commit messages.

Types of commits

| Type | Release Type | Reason For Use | | --------------- | --------------------- | ----------------------------------------------------------------- | | BREAKING CHANGE | Major Release (1.0.0) | Make incompatible API changes | | feat | Minor Release (0.1.0) | Add functionality that's useable in a backwards compatible manner | | fix | Patch Release (0.0.1) | Make backwards compatible bug fixes |

Examples of release type that will be done based on commit types:

• Major Release - BREAKING CHANGE: The graphiteWidth option has been removed
• Minor Release - feat: add 'graphiteWidth' option
• Patch Release - fix: stop graphite breaking when too much pressure applied

Running a Release

When creating cutting a new version, use npm run release to create a new production build and changelog.