@westayltd/logging

Structured logging and distributed tracing for Westay NestJS services.

One request crossing five services produces log lines in all five, and every one of them carries the same trace_id. Search that id in Graylog and you get the whole request back — in order, with per-hop timings.

TRACE 155576fd90e19c400baebb1cf75d54d8

  supplier-connect-service   GET  /supplier-connect-service/api/v1/health   200     2ms
  hotel-service              GET  /hotel-service/api/v1/health              503   307ms

No service passes a header to make that happen.

Install

npm i @westayltd/logging

Peer dependencies (@nestjs/common, @nestjs/core, rxjs, reflect-metadata) come from your service, not from here — two copies of Nest in one process break dependency injection.

Wiring

Six steps. They are the same in every service; only the name and the allowlist change.

1. src/tracing.ts — new file

import * as dotenv from 'dotenv';

dotenv.config({ path: ['.env.local', '.env'] });

// eslint-disable-next-line import/first
import { initTracing } from '@westayltd/logging/tracing';

initTracing({ serviceName: 'hotel-service' });

dotenv runs here on purpose — see Troubleshooting.

2. src/main.ts — first line

import './tracing'; // MUST be first
import { NestFactory } from '@nestjs/core';

Instrumentation patches http, pg and ioredis when they are required. If this import runs late it produces nothing — no error, no spans, no trace ids. The package detects this and refuses to boot rather than lying to you.

3. src/app.module.ts

import { LoggingModule } from '@westayltd/logging';

@Module({
  imports: [
    LoggingModule.forRoot({
      service: 'hotel-service',        // must match initTracing's serviceName
      capture: {
        request:  ['checkIn', 'checkOut', 'rooms', 'city', 'currency'],
        response: ['totalResults', 'page', 'pageSize'],
      },
    }),
    // ...existing modules unchanged
  ],
})
export class AppModule {}

capture is the production allowlist. Dev and staging log the full payload, so locally these names change nothing.

4. src/main.ts — install as the Nest logger

const app = await NestFactory.create(AppModule, { bufferLogs: true });
app.useLogger(app.get(WestayLogger));

This is what makes every existing new Logger(...) call site emit the standard envelope without being edited. Keep bufferLogs: true so bootstrap logs get the envelope too.

5. Environment

See Configuration.

6. Nothing else

Controllers, services, DTOs, validation and your HTTP clients stay exactly as they are. traceparent is injected into outbound calls automatically. If a step requires editing business logic, stop and raise it — that means the package is wrong, not your service.

Adding fields

WestayLogFields is the only way to add structure. TypeScript prevents inventing a top-level key; data is the sole open door.

constructor(private readonly logger: WestayLogger) {}

this.logger.info('supplier responded', {
  event: 'supplier_call',
  supplier_id: 'EXPEDIA',
  hotel_id: hotel.id,
  duration_ms: elapsed,
  data: { ratePlans: rates.length },   // free-form, per service
});

To attach ids deep in a call stack without threading parameters:

import { RequestContext } from '@westayltd/logging';

RequestContext.set({ booking_id: booking.id, customer_id: booking.customerId });

Every line emitted later in that request carries them.

data is serialised to one JSON string, never a nested object. A nested object would become westay_data_<key> — a new Elasticsearch field per key, per service, forever. Past the 1,000-field default new fields stop being indexed silently. The trade-off: westay_data is searchable but not aggregatable. If you need to avg() or group by something, open a PR promoting it to a real field.

Redaction

Two independent layers, both always on, in every environment:

• By name — password, token, secret, cardNumber, authToken, … Keys are normalised to snake_case first, because the estate is overwhelmingly camelCase and an anchored snake_case regex silently misses cardNumber.
• By shape — a card number in a field called reference is still masked. Card detection requires a recognised issuer prefix and a Luhn check, so booking-service's 15-digit supplier confirmation numbers survive — those are exactly what you need to debug a booking.

{"city":"Dubai","cardNumber":"********","authToken":"********"}

Local development

Graylog, OpenSearch and MongoDB, from this repo:

docker compose up -d
./scripts/provision-graylog-inputs.sh     # OTLP input + field-naming pipeline

Graylog at http://localhost:9000 (admin / admin — local container only). Point your service at it:

GRAYLOG_HOST=localhost
GRAYLOG_PORT=4318
GRAYLOG_PROTOCOL=http
DEPLOYMENT_ENVIRONMENT=dev
LOG_TRANSPORT=both      # stdout as well, so you can see it without the GUI

Verify the whole path end to end:

npm run test:trace      # emits inside a real span, reads it back out of storage

Configuration

| Variable | Default | Notes | |---|---|---| | DEPLOYMENT_ENVIRONMENT | — | required, strict enum dev\|staging\|production | | LOG_LEVEL | info | debug\|info\|warn\|error | | LOG_TRANSPORT | otlp | otlp\|stdout\|both | | LOG_CAPTURE_MODE | auto | auto = allowlist in production, open in dev/staging | | LOG_HTTP_STATUS_CLASSES | all | e.g. 4xx,5xx or 5xx,429 | | GRAYLOG_HOST | — | required unless LOG_TRANSPORT=stdout | | GRAYLOG_PORT | 4318 | Graylog's OpenTelemetry HTTP input | | GRAYLOG_PROTOCOL | https | http\|https | | GRAYLOG_PATH | /v1/logs | | | GRAYLOG_TOKEN | — | sent as Authorization: Bearer | | LOG_BATCH_MAX_MESSAGES | 50 | | | LOG_BATCH_FLUSH_MS | 1000 | also bounds the crash loss window | | LOG_BUFFER_MAX | 10000 | bounded queue; full means drop, never grow | | OTEL_ENABLED | true | |

Anything invalid throws at boot. Misconfiguration should be loud at startup, not silent at 3am.

Troubleshooting

Graylog shows nothing

Its time range defaults to Last 5 minutes. Widen it before assuming anything is broken — this is the single most common cause.

Logging throws at boot with GRAYLOG_HOST is required

LoggingModule.forRoot() reads process.env while app.module.ts is being imported, which happens before ConfigModule loads your .env. That is deliberate: config problems should fail at boot. Load dotenv in src/tracing.ts (step 1) — it already runs first. Containers supply the environment directly, so this only affects local development.

trace_id is missing from every line

The transport sends OTLP as protobuf, not JSON, and must stay that way. OTLP/JSON encodes trace ids as hex; Graylog reads them as base64 and discards hex silently — HTTP 200, message indexed, ids gone. npm run test:trace is the regression test for exactly this.

No trace_id under Jest

Jest replaces Node's module loader, so OpenTelemetry's instrumentation never patches http. This is a Jest limitation, not a bug. Integration tests that need real traces run under plain Node — see test/integration/.

Non-goals

• Not an APM. Spans are consumed in-process for ids and timings. Trace waterfalls would need an OTLP exporter to Tempo or Jaeger.
• Not a durability mechanism. Logs are best-effort: batched, and dropped rather than buffered without bound during an outage. Anything that must survive belongs in Postgres — audit_logs, outbox_events.
• Not a metrics library. Graylog aggregates the fields we emit; it is not a counter/histogram API.
• Not configurable per call site. The envelope is fixed on purpose. One shape across 13 services is what makes a single query work everywhere.