MCF Base System

This repository contains a base development environment with central services for the MyCareersFuture project.

Go through this before starting all other projects. See respective projects for the commands to start them.

Requirements

Make

Run which make && make --v to verify your version of Make.

$ which make && make --v

## - - output - -
# Built for x86_64-pc-linux-gnu
# Copyright (C) 1988-2014 Free Software Foundation, Inc.
# License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
# This is free software: you are free to change and redistribute it.
# There is NO WARRANTY, to the extent permitted by law.

Docker

Run which docker && docker version to verify your version of Docker.

$ which docker && docker version

## - - output - -
# Client:
#  Version:           18.06.1-ce
#  API version:       1.38
#  Go version:        go1.10.3
#  Git commit:        e68fc7a
#  Built:             Tue Aug 21 17:24:56 2018
#  OS/Arch:           linux/amd64
#  Experimental:      false
#
# Server:
#  Engine:
#   Version:          18.06.1-ce
#   API version:      1.38 (minimum version 1.12)
#   Go version:       go1.10.3
#   Git commit:       e68fc7a
#   Built:            Tue Aug 21 17:23:21 2018
#   OS/Arch:          linux/amd64
#   Experimental:     false

Docker Compose

Run docker-compose version to verify your version of Docker Compose.

$ which docker-compose && docker-compose version

## - - output - -
# docker-compose version 1.20.1, build 5d8c71b
# docker-py version: 3.2.1
# CPython version: 2.7.12
# OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016

Development Setup Overview

The following describes the system which will be run locally. Click on it to expand it.

![https://gitlab.gds-gov.tech/wsg/ops-base/raw/master/assets/DevelopmentSetupOverview.svg](https://gitlab.gds-gov.tech/wsg/ops-base/raw/master/assets/DevelopmentSetupOverview.svg

Usage - via Repository Cloning

This Repository

1. Clone this repository to your local drive
2. Spin up the base components by running:
   1. make dev if you're a developer
   2. make ops if you're doing devops/ops
3. If the above ran well, append a .d behind it to background the tasks
   1. make dev.d if you're a developer
   2. make ops.d if you're doing devops/ops
4. To change your local environment setup between running APIs locally/remotely
   1. make nginx to restart the nginx container with the new configuration
      1. e.g. USE_QA_JOBSEEKER=false USE_QA_PROFILE=false USE_QA_SVC_AUTH=false make nginx to run everything locally
5. To create a shell into any container, use make exec SVC="${SERVICE_ID}" where ${SERVICE_ID} is the ID of the listed service in the ./docker-compose.yml file
6. To view the logs for any container, use make logf SVC="${SERVICE_ID}" where ${SERVICE_ID} is the ID of the listed service in the ./docker-compose.yml file
7. For shutting down the base system, use make down
8. To clear all data stored locally, use make clean
9. Database root credentials are root:password

NGINX configuration

1. USE_QA_JOBSEEKER defaults to false

• false, means use your locally ran api-jobseeker code base
• true, connect to https://api-mcf-qa.gds-gov.tech

1. USE_QA_PROFILE defaults to false

• false, means use your locally ran api-profile code base
• true, connect to https://profile-mcf-qa.gds-gov.tech

1. USE_QA_SVC_AUTH defaults to true

• false, means use your locally ran svc-auth code base
• true, connect to https://account-mcf-nginx-development.gds-gov.tech

Using The Central Services

1. Reference the Docker network named mcf: do this by adding the following into your project's ./docker-compose.yml:

networks:
  mcf:
    name: mcf
    external: true

2. Connect your service in the ./docker-compose.yml of the project by adding the following property to the service properties:

networks:
- mcf

Examples

The directory ./example contains two sample projects that connect back to the "mcf" network defined in the main ./docker-compose.yml file. See the Docker Compose files for ./example/project-a and ./example/project-b. Creating a new schema in the database instance is also demonstrated via the db-ensure project.

1. Use make dev to setup the development environment
2. Use make ega to run example A
3. Use make egb to run example B

Usage - via NPM

Install the package via NPM:

npm install @mcf/base

Start the base services by running:

• mcf init for developers
• mcf init -x for devops

Append a -d to run it in the background.

Other Commands

mcf status shows the status of MCF related containers

mcf denit shuts down the set of central services

mcf test tests the structure of a repository

mcf db creates a shell into the MySQL database

What's Included

• Observability
  • AlertManager
  • Kibana
  • FluentD
  • Grafana
  • Prometheus
  • Zipkin
• Data Persistence
  • ElasticSearch
  • MySQL
  • Redis
• Custom Tools
  • DBMaker
  • Pulse
• Routing
  • Nginx

The following is a table to access all the local services once you have run make (dev|ops)(.d). dev is a complete subset of ops, so whatever is in dev is also ran in ops.

| Service | URL | User Type | | --- | --- | --- | | AlertManager | http://localhost:9093 | ops | | DB-Maker | http://localhost:2222 | dev | | ElasticSearch (REST API) | http://localhost:9200 | ops | | ElasticSearch (Node Coomunication) | http://localhost:9300 | ops | | ElasticSearch Exporter | http://localhost:9108 | ops | | FluentD | http://localhost:24224 | ops | | FluentD Exporter | http://localhost:24231 | ops | | Grafana | http://localhost:13000 | ops | | Kibana | http://localhost:5601 | ops | | MySQL | http://localhost:3306 | dev | | MySQL Exporter | http://localhost:9104 | ops | | Nginx | http://localhost:9999 | dev | | Prometheus | http://localhost:9090 | ops | | Pulse | http://localhost:54545 | dev | | Redis | http://localhost:6379 | dev | | Redis Exporer | http://localhost:9121 | ops | | Zipkin | http://localhost:9411 | ops |

Updating PlantUML

The above system diagram was generated using PlantUML.

To modify it:

1. Download it from https://sourceforge.net/projects/plantuml/files/plantuml.jar/download
2. Move it into /opt - mv ~/Downloads/plantuml.jar /opt/plantuml.jar
3. Create a script at /opt/plantuml - sudo touch /opt/plantuml)
4. Make it executable - sudo chmod +x /opt/plantuml
5. Create a symlink to your binaries directory - sudo ln -s /opt/plantuml /usr/bin/plantuml
6. Make the symlink executable - sudo chmod +x /usr/bin/plantuml
7. Paste the following into the /opt/plantuml file:

#!/bin/sh
java -jar /opt/plantuml.jar $@;

You should now be able to run plantuml ./assets/dev.puml to generate the image.

Cheers