@nxtedition/logger

A high-performance, structured JSON logger built on pino with a dedicated worker thread for I/O.

Why a logger worker?

In Node.js, multiple threads writing to stdout concurrently can interleave partial writes, producing corrupted or merged log lines. This is especially problematic when several worker threads each emit structured JSON — a single torn line breaks every downstream log parser.

This package solves the problem by routing all log output through a single dedicated worker thread:

1. No tearing / interleaving — Only one thread ever calls write(2) on fd 1, so every JSON line is atomically written.
2. Better throughput — Writers serialize log messages into ring buffers backed by SharedArrayBuffer (SAB). The logger worker drains these buffers and writes to stdout, decoupling application threads from a slow consumer while the ring has free space. If the consumer stalls long enough to fill the ring (e.g. a wedged stdout pipe), the write path applies back-pressure: writeSync blocks the calling thread until space frees, and ultimately throws after ~60 s. The logger worker itself never crashes on stdout back-pressure — it retries EAGAIN/EBUSY indefinitely.
3. Minimal latency on the hot path — when the ring has space, writeSync into it is a memory copy + atomic store: no syscall, no serialization contention. The actual write(2) happens asynchronously on the worker thread.

Architecture

  ┌──────────────┐   ┌─────────────┐   ┌─────────────┐
  │  Main thread │   │  Worker A   │   │  Worker B   │
  │  (logger)    │   │  (logger)   │   │  (logger)   │
  └──────┬───────┘   └──────┬──────┘   └──────┬──────┘
         │                  │                 │
    writeSync()        writeSync()        writeSync()
         │                  │                 │
    ┌────▼─────┐       ┌────▼─────┐      ┌────▼─────┐
    │ Ring buf │       │ Ring buf │      │ Ring buf │
    │  (SAB)   │       │  (SAB)   │      │  (SAB)   │
    └────┬─────┘       └────┬─────┘      └────┬─────┘
         │                  │                 │
         └──────────┬───────┘─────────────────┘
                    │
             ┌──────▼──────┐
             │Logger worker│
             │  (single)   │
             │             │
             │ readSome()  │
             │ fs.writeSync│
             │   fd 1      │
             └─────────────┘

SAB = SharedArrayBuffer

Each call to createLogger() allocates a dedicated SharedHandle ring buffer (2 MiB requested, rounded up to a 4 MiB physical region). The writer (application side) serializes pino JSON into the buffer. The logger worker polls all registered readers, batches contiguous lines into a 256 KiB staging buffer, and flushes each batch to stdout with a single write(2) — roughly 6× less syscall time per line than writing line-by-line, which is what lets one logger worker absorb the aggregate output of many application threads.

Loggers are long-lived. The native backing of each ring buffer is never freed (a deliberate trade-off to avoid cross-thread use-after-free). Create one logger per thread and derive context with logger.child({ ... }); do not call createLogger() per request/connection — each call leaks ~4 MiB.

Registration and unregistration use BroadcastChannel with a SharedArrayBuffer-based ack to ensure the worker has set up the reader before the caller proceeds.

Graceful shutdown

The drain protocol runs from a process.on('exit') handler, which Node fires on normal termination — process.exit(), an empty event loop, and uncaught exceptions. On 'exit':

1. The main thread flushes every writer (flushSync()), publishing pending data to its ring buffer.
2. It signals the logger worker to drain via a shared Int32Array flag.
3. It blocks (up to 2 s) until the worker confirms the drain is complete.
4. The logger worker also registers its own process.on('exit') handler as a safety net that drains all readers a final time.

A default-disposition signal (SIGTERM/SIGINT/SIGHUP) would otherwise kill the process without firing 'exit', losing buffered logs. To prevent this, the logger installs handlers for those signals that convert them into a normal process.exit() (so the drain above runs) — but only when the application has not registered its own handler for that signal. The logger's handler is prepended, so it runs first and correctly detects an app handler regardless of registration order — including process.once() handlers, whose wrappers self-remove before running. If your app handles the signal itself (e.g. for graceful HTTP shutdown), it owns the shutdown sequence; ensure it calls process.exit() so the drain runs.

Two teardown details:

• Worker threads publish eagerly. When the main process exits, other threads are torn down without running their own 'exit' handlers, so worker-thread writers publish their data to the ring after every write — buffered worker-thread logs survive a main-thread process.exit() mid-tick.
• Logging from 'exit' handlers works. A log written from an app 'exit' handler after the stream has already been torn down is written directly to stdout instead of being dropped.

Usage

import { createLogger } from '@nxtedition/logger'

const logger = createLogger({ level: 'info' })
logger.info({ key: 'value' }, 'hello world')

createLogger accepts all pino options. Custom serializers are merged with the built-in set (err, req, res, etc.).

Worker threads

Call createLogger() on the main thread first — this starts the logger worker. Worker threads can then call createLogger() freely; each gets its own ring buffer registered with the shared logger worker.

// main.ts
import { createLogger } from '@nxtedition/logger'
const logger = createLogger({ level: 'info' }) // starts worker

// worker.ts
import { createLogger } from '@nxtedition/logger'
const logger = createLogger({ level: 'debug' }) // registers with existing worker

Development

yarn build           # compile TypeScript → lib/
yarn test            # run tests
yarn test:coverage   # run tests with c8 coverage report

License

See LICENSE.