Audit for Node.js/Express APIs

This package provides an Express middleware audit framework. It's a framework in the sense that it may be extended to provde additional audit log destinations. The package includes two destinations:

1. Console - audit entries are sent to stdout via console.log.

2. AWS SNS - audit entries are published to a topic managed by AWS Simple Notification Service (SNS).

Contents

• Summary
• Quick Start
• Options
• Usage
• Unit Testing
• Extensions

Summary

As an Express middleware package, it's intended to be incorporated through an Express route in a manner like the following sample.

1  const audit = require('@isab/audit).middleware(options);
   ...
2  router.post('/MyResource',
3    cors(corsOptions),
4    audit,
5    authenticator.verifyBearer,
6    authorizeScope,
7    requestHandler );

The point of including so many middleware components is simply to demonstrate the intended order. The numbered descriptions below relate to the line numbers above.

1. An audit middleware instance is configured through a set of properties. Without properties provided, a console audit is configured.
2. This is the usual method and resource name specified on an Express route.
3. If your route supports CORS, it should come before the audit middleware.
4. The audit middleware should come early in the chain to make it available to other middleware and the request handler. This step will add an audit property to the Request object.
5. Authentication middleware (just a sample).
6. Authorization middleware (just a sample).
7. The request handler.

Quick Start

A quick configuration can be obtained with the following Express application that simply returns OK. In this example we configure a response header MyApp that will return the request ID.

const express = require('express');
const audit   = require('@isab/audit').middleware({responseHeader: "MyApp"});
const app = express();
const port = 6014;

app.get('/test', audit, (req, res) => {
  req.audit.end();
  req.audit.status = 200;
  res.status(200).end('OK\n');
});

app.listen(port, () => console.log('Listening on', port));

The output from a curl invocation shows the MyApp response header holding the request ID.

$ curl http://localhost:6014/test -i
HTTP/1.1 200 OK
X-Powered-By: Express
MyApp: ee49ed90-4603-11ea-9621-77baedfd4eb4
Date: Sun, 02 Feb 2020 21:35:28 GMT
Connection: keep-alive
Content-Length: 3

OK

The output in the Express stdout will be sandwiched between two displays of the request ID .

$ node minimalApp
Configuring console handler.
Listening on 6014
------ Start request ID ee49ed90-4603-11ea-9621-77baedfd4eb4 ----------
 Start time: 2020-02-02T21:35:28.483Z
   End time: 2020-02-02T21:35:28.491Z
   Duration: 8 ms
HTTP Status: 200
  Component: /test
     Action: GET
 Auth Token: missing
  Client IP: ::1
  Client ID: undefined
  Issuer ID: undefined
     Fields: undefined
     Claims: {}
     Errors: []
------ End request ID ee49ed90-4603-11ea-9621-77baedfd4eb4 ------------

Most of the fields are empty or undefined. In practice, these are populated within the request handlers as described below. Only a few fields are populated automatically.

Options

The following properties apply to all destinations.

| Property | Value | Default | |------------------|-------------------------------------|-----------| | emitter | console or awssns | console | | responseHeader | Response header name for request ID | none | | quietInit | true = suppress init logging | false | | quietEmit | true = suppress emit logging | false |

If responseHeader is not included in the options object, no response header is added to the response by the audit middleware.

The configuration is echoed to stdout when the module is initialized. This may be suppressed with quietInit: true. Most emitters will log a summary of each emit. Set quietEmit: true to suppress this. The console audit ignores this last flag.

These additional properties apply to AWS Simple Notification Services. They are ignored when emitter is console.

| Property | Value | |---------------|------------------------------| | awsRegion | Region for AWS SNS | | awsTopicArn | Topic ARN for AWS SNS | | awsKeyId | API key for AWS user role | | awsSecret | API secret for AWS user role |

It is suggested that desired values be provided through environment variables. The following sample demonstrates the configuration of an SNS audit middleware from the environment.

const emitter        = process.env['AUDIT_EMITTER'] || 'console';
const responseHeader = process.env['AUDIT_RESPONSE_HEADER'] || 'x-request-idp';
console.log('Audit config:', emitter);
const auditOptions = { emitter, responseHeader };
if (emitter === 'awssns') {
  auditOptions.awsRegion   = process.env['AUDIT_AWS_REGION'];
  auditOptions.awsTopicArn = process.env['AUDIT_AWS_TOPIC'];
  auditOptions.awsKeyId    = process.env['AUDIT_AWS_KEY'];
  auditOptions.awsSecret   = process.env['AUDIT_AWS_SECRET'];
}
const auditMiddleware = Audit.middleware(auditOptions);

The middleware will echo the configuration to stdout. It will not divulge an awsSecret property, only whether it is set.

Usage

Configuring audit middleware is easy. Using it soundly in your request handler requires more work. Two principles to keep in mind while writing your request handler are

• An audit property on the Request object contains the audit instance.

• The request handler must call audit.end() before exiting the handler, otherwise the record is never emitted.

This latter can be difficult to accommodate in the presence of multiple (often asynchronous) code paths for a request handler. The following is an example of a callback passed to a database library. It has the typical signature of an error and a data object.

 1  const audit = req.audit;
    ...
 2  if (err) {
 3     response = {
 4        code: err.code,
 5        errno: err.errno,
 6        sqlMessage: err.sqlMessage ? err.sqlMessage.slice(0, 60) : 'None'
 7     }
 8     dbErrorLogger(err);
 9     res.status(500);
10     audit.status = 500;
11     audit.errors.push(err.code);
12     audit.errors.push(err.sqlMessage);
13  } else {
14     response.insertId = results.insertId;
15     res.status(200);
16     audit.status = 200;
17  }
18  res.json(response);
19  audit.end();

Lines 9 and 15 show the setting of the HTTP status code on the audit object. Lines 11 and 12 demonstrate that error messages are pushed onto the error array. Line 19 invokes end(). There will typically be a call to end() at the end of each asynchronous code path. This is not simply something that can be cleaned up at the end of the current event loop.

Unit Testing

When unit testing your request handlers, you'll need to supply an audit instance. Two approaches are

1. Use a tool like Jest to simulate an Audit instance through spies (mock functions) to ensure the audit function is invoked as expected.

2. Provide a ConsoleAudit instance to the test invocation. This won't avail itself to Expect methods like .toHaveBeenCalled. But you can still inspect the state of the Audit instance with Expect methods.

To get the second option working in your Jest tests, incorporate the following pattern into your tests.

const AuditConfig = require('@isab/audit');

let audit;

beforeEach(() => {
  audit = AuditConfig.instance();
});

test('Should find one error with this request', () => {
  // Create mocks for req and res.
  handler(req, res, audit);  // this tests expects one client error.
  expect(audit.errors).toHaveLength(1);
  expect(audit.status).toBe(400);
  expect(audit.done).toBe(true);
});

Extensions

One can easily extend the Audit class to customize your own output (in the case of the Console) or a different audit destination. Each extension module exports

• middleware - the (req, res, next) function that provides the middleware
• the class itself

Only the first is strictly necessary. Normal usage of the the audit doesn't involve direct access to the Audit subclass. But it can be useful for unit testing.

For extension samples, check the following classes in the GitHub repository:

• ConsoleAudit
• SnsAudit

For a sample of processing SNS events from SnsAudit, see the Lambda function in the aws folder.