node-service

How a Node HTTP service behind a load balancer or proxy should serve and stop:

• Keep-alive: idle connections stay open longer than the proxy keeps its own, so the proxy is always the side that closes them.
• Graceful shutdown: on SIGTERM, fail readiness first, keep serving for a while, and only then stop listening.

Why the keep-alive matters

A proxy in front of the server pools its connections and sends the next request down an idle one. Node closes an idle connection after 5 seconds by default, much sooner than proxies do (Envoy's default is an hour). When the server closes a connection at the moment the proxy reuses it, that request fails and the caller gets an error from the proxy, typically a 502 or 503, for a request the service never saw.

gracefulShutdown({ server }) sets server.keepAliveTimeout from KeepAlive__TimeoutSeconds, or to 61 minutes (KEEP_ALIVE_TIMEOUT_MS) when that is unset. The proxy then closes idle connections before the server would, and that race goes away. It costs nothing at shutdown: since Node 19, server.close() drops idle connections straight away. Set the variable from the deployment, a little above the idle timeout of the proxy in front.

Why the order matters

A service that stops listening the moment it takes SIGTERM is still in the load balancer's rotation. A health check runs on an interval and its verdict still has to propagate, so for a few seconds afterwards the load balancer keeps sending requests to a socket nobody is accepting — and the caller sees a connection error rather than a clean response.

Failing the readiness check immediately and then waiting is what closes that window: the load balancer takes the instance out while it is still able to serve.

Install

npm install @dineroregnskab/node-service

Use

const { gracefulShutdown } = require('@dineroregnskab/node-service');

const app = require('http').createServer(handler);

const shutdown = gracefulShutdown({ server: app });

function handler(req, res) {
  if (req.url === '/health/ready') {
    res.writeHead(shutdown.readyStatus());
    return res.end('Healthy!');
  }
  // ...
}

app.listen(3000);

readyStatus() returns 200 while serving and 425 once SIGTERM has arrived. A service with readiness conditions of its own passes them in, and any false wins:

res.writeHead(shutdown.readyStatus(!restarting));

isShuttingDown() is the same state as a predicate, for code that is not a readiness endpoint — suppressing a restart-on-fatal path during shutdown, for instance.

Teardown beyond the HTTP server goes in onClosing, which runs once the server has stopped accepting connections, while it is still closing:

gracefulShutdown({
  server: app,
  onClosing: () => new Promise((resolve) => io.close(resolve)),
});

That ordering is deliberate. server.close() stops new connections straight away, but its callback waits for the open ones — and a long-lived connection such as a WebSocket is exactly what the teardown is there to end. Running the teardown only after that callback would wait for something only the teardown can cause, until the timeout gave up and killed the process with the clients still attached.

Configuration

| Variable | Default | What it is | | --- | --- | --- | | Shutdown__DelaySeconds | 10 | How long to keep serving after SIGTERM while readiness fails, so the load balancer can take the instance out of rotation. | | Shutdown__TimeoutSeconds | 30 | Ceiling on closing the server afterwards. Past it the process exits regardless. | | KeepAlive__TimeoutSeconds | 3660 | How long an idle connection stays open. Keep it above the idle timeout of the proxy in front. |

Set them from the deployment rather than in code, so the delay can track the health check interval of whatever sits in front of the service, and the keep-alive its idle timeout. The defaults are what a service gets when nothing sets them: a developer running it by hand, or a test.

An unset or empty value falls back to the default. So does one that is not a non-negative number, which also logs a warning.

The timeout is not decoration

server.close() stops accepting new connections but waits for in-flight ones, and a long-lived connection — a websocket — can keep it waiting indefinitely. Without the ceiling the process would sit there until whatever supervises it loses patience and kills it, which is the behaviour this package exists to avoid.

API

| Export | What it does | | --- | --- | | gracefulShutdown(options?) | Raises the server's keep-alive timeout, installs the SIGTERM handler, returns { isShuttingDown, readyStatus }. | | readSeconds(name, fallback) | Reads a non-negative number of seconds from the environment. | | DEFAULT_DELAY_SECONDS / DEFAULT_TIMEOUT_SECONDS | 10 and 30. | | KEEP_ALIVE_TIMEOUT_MS | 3660000, 61 minutes: the keep-alive when KeepAlive__TimeoutSeconds is unset. | | READY / NOT_READY | 200 and 425. |

options: server (keep-alive raised, closed after the delay), onClosing (teardown while it closes, inside the timeout), plus exit and process for tests.

Test

npm test