Exframe testing module

Module that runs unit and contract tests, collects coverage information and stitches it together into a consolidated report. It uses the service's docker-compose.yml file to bring up the necessary environment, then it replaces the service itself with a version that records the code coverage before running the tests.

Use:

The Exframe testing module makes use of exframe-configuration and npm scripts to run tests and collect code coverage. Specifically, exframe-testing uses the following npm scripts:

unit-tests (runs the unit tests)

contract-tests (runs the contract tests)

combine-coverage (merges the coverage.json files output by the above scripts and produces reports of itemized and total code coverage)

In addition, the microservice's configuration must provide a script for running the service with code coverage enabled. This value should be stored in config.default.coverageScript. If not provided, the script defaults to:

./node_modules/.bin/nyc --handle-sigint --reporter lcov --report-dir ./documentation/contract-tests --temp-dir ./documentation/contract-tests node index.js'

The testing framework can be initiated from a service by:

node ./node_modules/exframe-testing/index.js

It is recommended to make the above the npm test script in the package.json of the service.

Executing the above script will run both the unit tests and the contract tests, then merge the coverage objects of both test suites into a single report. You can run either test suite individually with:

node ./node_modules/exframe-testing/index.js unit

or

node ./node_modules/exframe-testing/index.js contract

##Example

package.json:

{
  "name": "my-service",
  "version": "0.0.1",
  "description": "My Service",
  "main": "index.js",
  "repository": {
    "type": "git",
    "url": ""
  },
    "config": {
      "reporter": "spec"
  },
  "scripts": {
    "start": "node index.js",
    "test": "./node_modules/.bin/exframe-testing",
    "unit-tests": "nyc -r lcov --report-dir ./documentation/coverage -t ./documentation/coverage -x index.js -x '**/routing/**' _mocha -- -R $npm_package_config_reporter -c",
    "contract-tests": "./mocha --reporter $npm_package_config_reporter -- -c  ./test-contract",
    "combine-coverage": "nyc merge ./documentation/coverage ./.nyc_output/coverage-unit.json && nyc merge ./documentation/contract-tests ./.nyc_output/coverage-contract.json && nyc report --r lcov --report-dir ./documentation/combined"
  },
  "author": "Exzeo",
  "license": "ISC",
  "dependencies": {},
  "devDependencies": {
    "nyc": "^15.1.0",
    "chai": "^3.5.0",
    "mocha": "^3.2.0"
  }
}

./test-contract/tests/mytest.test.js:

/* eslint arrow-body-style: 0 */
/* eslint no-unused-expressions: 0 */
/* eslint no-multi-spaces: 0 */
/* eslint indent: 0 */


'use strict';

const expect = require('chai').expect;
const axios = require('axios');
const framework = require('exframe-testing');

describe('POST /something', () => {
  describe('Basic happy path', () => {
    const fields = ['description',             'data',   'expectedStatus'];
    const values = [
                   ['Does stuff with thing 1', 'thing1', 200]
                   ['Does stuff with thing2',  'thing2', 400]
                   ]
    const testCases = framework.build(fields, values);
    const test = testCase => {
      it(testCase.description, () => {
        const options = {
          method: 'POST',
          url: `${config.default.service.url}:${config.default.service.port}/something`,
          data: testCase.data
        };
        return axios(options)
          .catch(err => err.response)
          .then(response => {
            expect(response.status).to.equal(testCase.expectedStatus);
          });
      });
    };
    testCases.forEach(testCase => test(testCase));
  });
});

Utility Methods

waitFor(action: () -> Promise, condition: -> bool, period, maxTimeout) -> Promise

Perform an action once every {period} milliseconds for no more than {maxTimeout} milliseconds. Evaluate the result with the given condition callback until {condition} returns true or timesout. Returns a promise that resolves with the result of a successful action.

• action - The action to perform
• condition - The condition to evaluate against the action result
• period - The period in milliseconds to wait between action attempts
• maxTimeout - The max time in milliseconds that action will continue to be attempted. Timeouts will reject the promise with a 'Wait timeout' error.

const response = await waitFor(() => axios({...}), response => response.status === 200);

checkDependency(dependency) -> Promise

Waits a period of time (10 seconds by default) for the service dependency to be healthy, or reject if the timeout is reached.

• dependency (Object - Required): Service dependency that must be healthy before the promise resolves.
  • dependency.callback (Function - Required): Callback function used when making the health check. This is generally an asynchronous function that makes an http request and awaits the response. The health check will succeed if the callback function returns successfully. Note: If dependency.type does not equal "callback", this parameter is ignored.
  • dependency.config (Object - Optional): Configuration settings used in the dependency health check. Note: If dependency.type equals "callback", this parameter is required
    • dependency.config.serviceName (String - Optional): Identifies the dependency in log outputs. Note: If dependency.type equals "callback", this parameter is required
    • dependency.config.timeoutMs (Number - Optional): Number of milliseconds to wait for the dependency to become healthy before rejecting. Note: the default value is 10000
    • dependency.config.waitingInterval (Number - Optional): Number of milliseconds to wait before retrying the next health check after a failed check. Note: the default value is 250
  • dependency.exzeoService (String - Required): The name of the exzeo service with matching exframe-configuration data available to the exframe-testing module. Note: If dependency.type does not equal "exzeoService", this parameter is ignored. If exframe-testing does not find matching configuration data, the health check will immediately fail.
  • dependency.request (Object - Required): The data passed through to make an http request when performing a health check. Note: If dependency.type does not equal "request", this parameter is ignored.
    • dependency.request.body (Object - Optional): Request body that will be passed through to the http request when it performs a health check.
    • dependency.request.headers (Object - Optional): Request headers that will be passed through to the http request when it performs a health check.
    • dependency.request.method (String - Optional): Request method used for the http request. Valid values: "get", "post". Note: the default value is "get"
    • dependency.request.path (String - Optional): The path that will be appended after the dependency.request.url and dependency.request.port to form the fully qualified URL for the http request. Note: the default value is "/health/readiness"
    • dependency.request.port (Number - Optional): The port number that will be appended after the dependency.request.url to form the request URL for the http request. Note: the default value is 80
    • dependency.request.url (String - Required): The domain name of the service that will be used to form the request URL for the http request.
  • dependency.type (String - Required): Determines how the dependency's data will be processed to perform a dependency health check. Valid values: "callback", "exzeoService", "request"

  // dependency.type === 'callback'
  await checkDependency({
    callback: async () => {
      const myServiceHealthSdk = await new ServiceHealthSdk();
      await myServiceHealthSdk.checkHealth();
    },
    config: {
      serviceName: 'My-Service',
      timeoutMs: 250,
      waitingInterval: 10000
    },
    type: 'callback'
  });

  // dependency.type === 'exzeoService'
  await checkDependency({
    config: {
      serviceName: 'Harmony-Data',
      timeoutMs: 250,
      waitingInterval: 10000
    },
    exzeoService: 'harmonyData',
    type: 'exzeoService'
  });

  // dependency.type === 'request'
  await checkDependency({
    config: {
      serviceName: 'Harmony-Data',
      timeoutMs: 250,
      waitingInterval: 10000
    },
    request: {
      body: { 'harmony-access-key': 'body-access-key' },
      headers: { 'harmony-access-key': 'headers-access-key' },
      method: 'get',
      path: '/health/readiness',
      port: 80,
      url: 'http://harmony-data'
    },
    type: 'request'
  });

checkDependencies({ dependencies }) -> Promise

Waits a period of time (10 seconds by default) for all service dependencies to be healthy, or reject if the timeout is reached.

• dependencies (Array of Dependency Objects): List of service dependencies that must be healthy before the promise resolves.

  await checkDependencies({
    dependencies: [
      {
        callback: async () => {
          const myServiceHealthSdk = await new ServiceHealthSdk();
          await myServiceHealthSdk.checkHealth();
        },
        config: {
          serviceName: 'My-Service',
          timeoutMs: 250,
          waitingInterval: 10000
        },
        type: 'callback'
      },
      {
        config: {
          serviceName: 'Harmony-Data',
          timeoutMs: 250,
          waitingInterval: 10000
        },
        exzeoService: 'harmonyData',
        type: 'exzeoService'
      },
      {
        config: {
          serviceName: 'Harmony-Data',
          timeoutMs: 250,
          waitingInterval: 10000
        },
        request: {
          body: { 'harmony-access-key': 'body-access-key' },
          headers: { 'harmony-access-key': 'headers-access-key' },
          method: 'get',
          path: '/health/readiness',
          port: 80,
          url: 'http://harmony-data'
        },
        type: 'request'
      }
    ]
  });

FAQ

When I run the tests, there is an error telling me that the mocha module cannot be found, but mocha is in my package.json. Why can't the test framework find it?

mocha is likely a dev depedency and NODE_ENV=production is probably set in your Dockerfile. Legacy versions of exframe-logger required NODE_ENV=production to be set in order for the logger to send logs to Sematext Logsene. We have changed exframe-logger to send logs to Sematext Logsene only if a token is supplied. So, you may safely remove the following line from your Dockerfile:

ENV NODE_ENV=production

if you do this, you must upgrade to exframe-logger ^2.0.0. Otherwise, your microservice will no longer send logs to Sematext logsene. Please also verify that you are only providing exframe-logger with a valid Sematext token when creating your logger. Many microservices have used the following code:

const logger = require('exframe-logger').create(process.env.LOGSENE_TOKEN || 'token');

With exframe-logger ^2.0.0, there is no longer any need to provide a string as a token when process.env.LOGSENE_TOKEN is undefined. If you do, exframe-logger will continuously attempt to send logs to Sematext Logsene with the invalid token. So, please remove it:

const logger = require('exframe-logger').create(process.env.LOGSENE_TOKEN);

When I run the tests, my service comes up but the tests time out and say that the service is not running

You probably do not have a health check for your service. By default, the testing framework polls /health/readiness in order to determine that the service is healthy before it starts running the tests. If your microservice lacks a health check, you can easily add one using the exframe-health framework.

My service is compiled with babel. When I run npm test, the framework fails because it is attempting to run the service with the base code rather than the compiled code.

The default configuration for starting the service with code coverage is:

./node_modules/.bin/nyc --handle-sigint -reporter lcov --report-dir ./documentation/contract-tests --temp-dir ./documentation/contract-tests node index.js'

First, we strongly suggest you refactor your service so that it does not need to use babel. If this is impractical, there is a configuration option you can set to tell the testing framework to run the service with the compiled code. In the service's default configuration, set the coverageScript to point to the compiled code. For example in ./config/default.yml

default:
  service:
    port: "80"
    url: "http://file-index"
  coverageScript: ./node_modules/.bin/nyc --handle-sigint --reporter lcov --report-dir ./documentation/contract-tests --temp-dir ./documentation/contract-tests node ./dist/bootservice.js'

This points the framework to ./dist/bootservice.js and runs the compiled code to start the service.

I want to be able to seed my test database with data using the mongo-seed container

In order for testing to know that you are adding database seeding to your test runs, the framework will need to use a naming standard to determine if the seed container is running.

When setting up your docker-compose to add the mongo-seed container, use the container_name property set to 'mongo-seed'. The framework uses that name to lookup if the container is running and calls the health check on the container to determine when it is done so that contract tests can begin.

  mongo-seed:
    image: exzeo/integration-seed:latest
    container_name: mongo-seed
    depends_on:
      - mongo