Introduction

WARNING: this project is in its infancy, test hard before using in production

Restana-express is a partial Express compatibility layer for Restana, which implements most of the (req, res) additions over native node.js http(s) server made in Express. It may also work with other frameworks, but this is not guaranteed and may require slight modification.

To achieve this, restana-express imports some methods and properties from Express (if possible) or reimplements them.

Restana-express-compatibility does not aim to offer a 100% compatibility with Express and offers only most of the methods and properties outlined on the following pages:

Express Res

Express Req

Despite this, testing is mostly done with modified Express tests. However, please see the table in Compatibility section

Version & Changes

Please see CHANGELOG.md

License

MIT License

Usage

Installation

npm i --save restana-express-compatibility

Setting up the restana and middleware

	const restana = require('restana')
	let restanaExpressCompatibilityMod = require('restana-express-compatibility')
	let compatibilityLayerSettings = {
		res: {
			toUse: ['all'], // you can specify which res components you want. However, do take note that you may run into issues, if the component you use depends on another one, which you haven't specified. Check dependencies table in Observations and known problems
			toDisable: [], // Array of methods and properties to disable. Overrides 'toUse'
			render: {  // res.render was completely reimplemented and is now setup differently
				viewsDir: path.resolve(__dirname + "/views/"),
				renderExt: '.pug',
				renderEngine: 'pug',
				renderFunction: "__express"
			}
		},
		req: {
			toUse: ['all'], // you can specify which req components you want. However, do take note that you may run into issues, if the component you use depends on another one, which you haven't specified. Check dependencies table in Observations and known problems 
			toDisable: [], // Array of methods and properties to disable. Overrides 'toUse'
			proxyTrust: true, // express proxyTrust-related functions have been reimplemented, so it is setup here. Accepts Number, function, String or Array of IPs, Boolean (see Express proxy settings)
			queryParser: true, // true, "simple" uses restana's url query parser, "extended" or undefined enables Express's extended query parser (Check express docs for explanation)
			subdomainsOffset: 0, // defaults to 2
			// OR
			subdomainOffset: 0, // subdomainsOffset is still prioritized
			propertiesAsFunctions: true, // default - undefined. False to disable, true to enable
			etag: {
				type: 'tiny', // default - tiny, true enables Express's weak ETag, otherwise uses the same options as Express (see - https://expressjs.com/en/api.html#etag.options.table)
				seed: 01234567890, // Number, for use with Tiny ETag. Should be syncronised between all instances of your server on all machines, otherwise useless, since all ETags will be different. However, will generate a pseudo-random number by default instead.
				maxSize: 1000 // Used with Tiny ETag, size of internal cache (number of entries). 1000 is the default. You should probably set closer towards the number of resources available on your service. 
			}
		}
	}
	
	var app = restana();

	let restanaExpressCompatibility = new restanaExpressCompatibilityMod(compatibilityLayerSettings)

	app.use(restanaExpressCompatibility.middleware)

	app.get('/hi/', (req, res) => {
		res.send({
			msg: 'Hello World!',
			query: req.query,
			subdomains: req.subdomains,
			ip: req.ip,
		})
	})

	let server = app.start(3002, '0.0.0.0')
	  

Compatibility

Res

Properties

| Property or object | Implemented? | Notes | | --- | --- | --- | | locals | Yes | N/A | | headersSent | Native | Doesn't need reimplementation | | app | No | Probably not going to be implemented |

Methods

| Method or object | Implemented? | Notes | | --- | --- | --- | | append | Yes | N/A | | attachment | Yes | N/A | | cookie | Yes | cookieParser required for signed cookies, check req.cookie entry in req properties | | clearCookie | Yes | cookieParser required for signed cookies, check req.cookie entry in req properties | | download | Yes | N/A | | end | Native | Doesn't need reimplementation | | get | Yes | N/A | | json and jsonp | Partial | Reimplemented, but very crudely. It only JSON stringifies the object supplied. Needs to be rewritten | | links | Yes | N/A | | location | Yes | N/A | | redirect | Yes | N/A | | render | Rewritten from scratch | Rewritten from scratch, compatible with using only one render engine. Workarounds might be possible. Please read section on res.render | | send | Partial | Slightly modifies restana's res.send to make it respect change of http status code through res.status(code) or res.statusCode = code | | sendFile | Partial | Not sufficiently covered by tests, not all Express tests are ported. Likely works just fine | | sendStatus | Yes | N/A | | set / header | Yes | N/A | | status | Yes | N/A | | type / contentType | Yes | N/A | | vary | Yes | N/A | | format | Yes | Not sufficiently covered by tests |

res.render

restana-express does not import res.render, because it depends too much on express core. Therefore, this project partially reimplements the res.render itself.

Limitations

• Works only with one render engine, you can't specify many (workarounds might be possible by initializing this middleware several times differently in various places of your project)
• No express automagic :(

Settings

let compatibilityLayerSettings = {
	res: {
		toUse: ['all'], // should either be an Array with 'all' or at least include 'render'
		render: {  // res.render was completely reimplemented and is now setup differently
			viewsDir: path.resolve(__dirname + "/views/"), // ESSENTIAL parameter, without it res.render will send an error
			renderExt: '.pug', // essential, if you use something besides Pug. Dot at the beginning is needed. Defaults to '.pug'
			renderEngine: 'pug', // name of engine. restana-express will require it. Defaults to 'pug'
			renderFunction: "__express" // Internal function that works with express. Usually you should leave it unchanged. Defaults to '__express'
		}
	},
	...
}

Req

Properties

| Property or object | Implemented? | Notes | | --- | --- | --- | | app | No | Probably not going to be implemented | | baseUrl | No | Help is welcome | | headersSent | Native | Doesn't need reimplementation | | body | No | You may use body-parser middleware, since this is what express uses internally. You can app.use (or abuse :D) it either before or immediately after restana-express | | cookies | No, not in restana-express itself. Please, see notes | You should use cookie-parser middleware, as this property is not implemented in Express itself. | | fresh | Yes | N/A | | hostname | Yes | N/A | | ip | Yes | N/A | | ips | Yes | N/A | | method | Native | Does not need reimplementation | | originalUrl | Native | Does not need reimplementation | | path | Native | Does not need reimplementation | | protocol | Yes | N/A | | query | Yes | Set up the type of parser through compatibilityLayerSettings.req.queryParser, before initializing and app.use'ing the middleware. Accepts same values as express for this setting | | hostname | Yes | N/A | | route | No | Help is welcome | | secure | Yes | N/A | | signedCookies | No, not in restana-express itself. Please, see notes | Check entry in this table on req.cookies | | stale | Yes | N/A | | subdomains | Yes | subdomain offset is set either through compatibilityLayerSettings.req.subdomainsOffset OR compatibilityLayerSettings.req.subdomainOffset | | xhr | Yes | N/A |

Methods

| Method or object | Implemented? | Notes | | --- | --- | --- | | accepts | Yes | N/A | | acceptsCharsets | Yes | N/A | | acceptsEncodings | Yes | N/A | | acceptsLanguages | Yes | N/A | | get / header | Yes | N/A | | is | Yes | N/A | | param | Yes | Deprecated by express. Please note that you need body-parser middleware | | range | Yes | N/A |

Serve static

For serving static content, see this article: https://thejs701816742.wordpress.com/2019/07/12/restana-static-serving-the-frontend-with-node-js-beyond-nginx/

It is recommended that you limit these middleware to some routes, while restana-express-compatibility to other routes.

Observations and known problems

Tests

Tests currently cover only the default options, at which restana-express-compatibility is the most compatible with Express, but also the slowest. Help is welcome.

Restana-express-compatibility and cookie- and body-parser

In order for this middleware to work correctly for routes or middleware that were designed for Express, you need to speicfy and app.use it before them. Do take note, though, that you need to enable cookie-parser middleware before restana-express, if you plan to use req.secureCookies and send secure cookies:

var cookieParser = require('cookie-parser');

...
app.use(cookieParser(process.env.COOKIE_PARSER_SECRET || "wonderland"));
app.use(restanaExpressCompatibility.middleware)

The same applies to body-parser:

var cookieParser = require('cookie-parser');
const bodyParser = require('body-parser')


...
// parse application/json
app.use(bodyParser.json())
app.use(cookieParser(process.env.COOKIE_PARSER_SECRET || "wonderland"));
app.use(restanaExpressCompatibility.middleware)

Performance

With the default settings, restana-express-compatibility is highly compatible with express. This compatibility comes at a great cost - the performance is higher than with express, but not THAT much higher (only by up to 60%, depending on task). In other words, restana-express-compatibility can handle 16-17k requests per second instead of 10-11k with express with /json/ route (see benchmarks section)

Here are the tips to increase performance:

Tip 1 - Disable req properties, if you don't use them [EASY]

Some express req properties are very convenient at times. These include:

	['protocol', 'ip', 'ips', 'hostname', 'subdomains', 'fresh', "stale", 'xhr', 'secure'] // the list is exhaustive

However, before continuing with your application logic, each of them has to be processed, EVEN IF you don't end up using them. Internally, there are functions, like req.protocolPropFn which is run in the middleware, before carrying on to your routes. In this example, req.protocolPropFn populates req.protocol. In the end, it means that middleware is forced to traverse through these functions and then run each one that is enabled.

As such, if performance is vital, you can either disable some properties (very modest performance increase) OR disable them altogether (great speed increase). Alternatively, you can turn them into functions instead:

let compatibilityLayerSettings = {
	req: {
		propertiesAsFunctions: true,
	}
}

This way, whenever you need to know the protocol, instead of accessing req.protocol property, you'll need to call req.protocol() function. No arguments are required. This measure will boost restana-express-compatibility from handling 16-17k req/s to handling 21-23 req/s, which is a further 35% speed increase. You should be able to easily replace calls to req properties with calls to the new functions with your IDE or shell commands.

Tip 2 - Don't enable extended query parsing or weak or strong ETag [MEDIUM]

By default, restana-express-compatibility uses a different implementation of ETag, which should work just fine. Don't change the ETag setting, unless you know what you are doing. The same applies to query parsing - by default, Express.JS extended query parsing is used for Express compatibility reasons. We advise you to disable it and use the simple parser, as it is built-in, unless you REALLY need the extended parser. But in any case, you should avoid passing data as a query due to its inherent limitations.

You can also consider benchmarking with ETag disabled on specific routes with setting res.locals.NO_ETAG = true in your route or middleware. Beware that it may affect caching, if it is used.

Tip 3 - disable what you don't need [HARD]

By disabling things you don't need, you make the server use less RAM, which is beneficial in the long term. Additionally, the middleware merges the req and res objects with objects containing functions and properties taken from Express each time a route after compatibility middleware is requested by a client. This means that for every unnecessary method or property activated, the server spends more time merging the object with Object.assign.

For instance, with no methods or properties activated, the merge takes only ~5000 nanoseconds. However, with all methods or properties, it takes ~60000 nanoseconds. That is more than a third of the total middleware working time (with req.properties converted into functions). This means that keeping bare minimum of methods and properties you need will make the middleware work faster. Therefore, your own code will execute faster.

For your convenience, there are dependency tables provided after the tips. Take note that number of dependencies may be reduced in the next minor release.

Tip 4 - gradually stop using this module [POSSIBLY HARD]

By disabling this module and finding alternatives for your needs (if possible), you are going to have a much faster application. It is highly likely you do not need all of the methods and properties offered by express and only need a subset. Find appropriate middleware for your needs or reimplement the required methods to suit your needs. Your app performance is gonna thank you. Moreover, you can limit the middleware only to specific routes, so that it is not run, where it is not needed.

Dependency tables

Please take note that these tables serve as an indication only. It is highly possible some mistakes could have been made, especially with Dependents column

Res properties

| Property or object | Can be disabled? | Dependencies | | --- | --- | --- | | locals | No | None |

Res methods

| Method or object | Can be disabled? | Dependencies | Dependents | | --- | --- | --- | --- | | send | No | None | Irrelevant | | append | Yes | res.get, res.set | res.cookie | | attachment | Yes | res.type, res.set | None | | cookie | Yes | res.append, cookie-parse middleware (for signed cookies, see Compatibility and Observations and known problems sections ) | res.clearCookie | | clearCookie | Yes | res.cookie and its dependencies | None | | download | Yes | res.sendFile | None | | end | No, native | None | Irrelevant | | get | Yes | None | res.append, REQ.fresh, REQ.hostname | | json and jsonp | Yes | None | res.render | | links | Yes | res.set | None | | location | Yes | REQ.get && res.set | res.redirect | | redirect | Yes | res.location, res.format, res.set | None | | render | Yes | res.status, res.json, res.set | None | | sendFile | Yes | None | None | | sendStatus | Yes | None | None | | set / header | Yes | None | res.append, res.attachment, res.links, res.location, res.redirect, res.render, res.type/res.contentType, res.format | | status | Yes | None | res.render | | type / contentType | Yes | res.set | res.attachment | | vary | Yes | None | res.format | | format | Yes | res.vary, res.set | res.redirect |

Req properties

| Property or object | Can be disabled? | Dependencies | Dependents | | --- | --- | --- | --- | | headersSent | No, Native | None | Irrelevant | | fresh | Yes | RES.get | req.stale, RES.send (soft-fail, ETag functionality gets disabled) | | ip | Yes | None | None | | ips | Yes | None | None | | method | No, Native | None | Irrelevant | | originalUrl | No, Native | None | Irrelevant | | params | No, Native | None | Irrelevant | | path | No, Native | None | Irrelevant | | protocol | Yes | req.get | req.secure | | query | No | None | Irrelevant | | hostname | Yes | req.get | req.subdomains | | secure | Yes | req.protocol | None | | stale | Yes | req.fresh | None | | subdomains | Yes | req.hostname | None | | xhr | Yes | req.get | None |

Req methods

| Method or object | Can be disabled? | Dependencies | Dependents | | --- | --- | --- | --- | | accepts | Yes | None | None | | acceptsCharsets | Yes | None | None | | acceptsEncodings | Yes | None | None | | acceptsLanguages | Yes | None | None | | get / header | Yes | None | req.range, RES.location, req.protocol, req.hostname, req.xhr | | is | Yes | None | None | | param | Yes | req.params, req.body, req.query | None | | range | Yes | req.get | None |

Benchmarks (not scientific)

Route setup

The following route is used:

app.get('/hi/', async (req, res) => {
	res.send({
	  msg: 'Hello World!',
	  query: req.query

	})
})

For restana without restana-express, the following middleware is added:

app.use((req,res,next) => {
	res.json = function(inp) {
		res.setHeader('Content-Type', 'application/json');
		if (inp == null) return res.send(null)
		return res.send(JSON.stringify(inp))
	}
	return next()
})

Tool used

node ./performance/APP.js
sleep 5
wrk -t8 -c64 -d5s http://localhost:PORT/hi/

, where PORT and APP are changed to:

• 3001 and restana for restana
• 3002 and restanaExpressHighPerf for restana + restana-express (performance conscious, query-parser set to simple, req.properties turned into methods)
• 3003 and restanaExpress for restana + restana-express (default)
• 3004 and express for express

Hardware used

MacBook Pro 2019, 2,4 GHz Intel Core i9, 64 GB 2667 MHz DDR4

Results

| Server | Result, req/s | Gains over express | |---|---|---| | Restana | 48866.22 | 315% | | Restana + restana-express (performance conscious) | 23014.65 | 96% | | restana + restana-express (default) | 19087.31 | 62% | | Express | 11752.61 | 0% |