NLP API Server

This is the backend REST API for NLP powered by NodeJS and MongoDB

The Package (@aivec/nlp-api-server)

This repo exposes TypeScript types/interfaces and helper modules as an npm package. All files in the ./package folder are included in the tarball when running npm publish. Note that the contents of this package are subject to change at any time.

Prerequisites

• node >= 14
• npm >= 8 (this is important for ensuring a unified structure in package-lock.json)
• docker
• docker-compose (usually packaged with Docker)

Getting Started

First, install packages and start the MongoDB and MailHog Docker containers:

$ npm install
$ npm run start:services

Developing

Environment variables (DB connection strings, etc.) are set using a .env file. This file is automatically generated/updated when running the node Docker container.

Curious about why .env isn't version controlled? Refer here.

Start the hot reload dev server with the following:

$ npm run dev

To transpile the TypeScript source and run Node directly (without hot reloading):

$ npm run start:node

Debugging

To make debugging automatic for vscode users, a .vscode/launch.json file is tracked in source control. Ctrl+Shift+D should show Attach to Node (nlp-api-server) as a selection in the debug environments dropdown. Note that the node server must be already running in order for the debugger to attach to it.

Testing

Run tests with the npm script:

$ npm test

Debugging Tests

First, run Mocha in inspect mode:

$ npm run test:debug

Mocha will wait for a debugger to attach before running tests. Follow the steps in Debugging to attach the debugger.

Documentation

Documentation for REST APIs are written in API blueprint format. APIB files live in the docs folder and mirror the folder/file structure of src/api. In addition to manually adding .apib files for individual APIs, a docs/all.apib file, containing all apib's can be automatically generated with npm run docs:generate.

Deploying

serverless framework is used to deploy an AWS serverless stack for our REST API (API Gateway -> Lambda + S3 -> DocumentDB). Because serverless applications differ so drastically from traditional server apps, the deployment process looks very different. A general summary of this process is outlined below.

NOTE: currently, none of the deployment commands are run from within docker

Accounts

The following Serverless and AWS accounts are used for all resources: | Provider | Account ID (email) | Login URL | | -------- | ------------------ | --------- | | AWS | [email protected] | https://console.aws.amazon.com/console/home | | Serverless Framework | [email protected] | https://app.serverless.com |

Prerequisites

In order to deploy, you must login to the serverless framework admin account.

$ npm run sls:login

Stages

There is a staging stage and a prod stage. These two stages represent completely separate environments. Serverless recommends using separate AWS accounts for different stages. However, currently, the same credentials and provider are used for both deployments/environments. Therefore, you can deploy to either one by simply specifying the stage in the sls deploy command like so:

$ npm run deploy:staging
$ npm run deploy:prod

Environment variables

Environment variables are handled on a per-stage basis by use of AWS SSM Parameters. The value of these parameters depends on the stage.

Where is everything?

Various analytics graphs can be monitored from the serverless framework dashboard.

CloudFormation is what serverless framework leverages in order to automate the deployment process. If you want to see detailed information for specific resources (S3, Lambda, etc.), use the CloudFormation dashboard.