@diveinto/obs

Structured logging, tracing and metrics for Next.js and Node services. One record shape, written to stdout and to two rotating files, with W3C trace context threaded through it, and optional OpenTelemetry export that stays off until you configure an endpoint.

npm install @diveinto/obs

Why it exists

It grew out of consolidating four hand-maintained copies of the same library across four services. Copies drift: by the time they were merged, one had real OpenTelemetry spans around requests and the others did not, one had route templating that kept metric cardinality bounded, one had fixed a duration that went unrecorded when something threw a non-Error, and one had made log-file resolution a pure, testable function. This package is the union of the four, and each of those behaviours is pinned by a test.

What you get

• One record shape, frozen, identical in the text and JSON renderers: ts level trace_id span_id parent_span_id logger event msg plus context. A sibling Python implementation writes the same shape, so one grep can follow a trace across services written in either language.
• Three sinks: stdout (always), and two rotating files that are format-locked, so <stem>.log is always text and <stem>.jsonl is always JSON regardless of LOG_FORMAT. Daily rotation, 14 kept, both configurable. One writer per path per process, shared across module contexts on globalThis along with the sink registry and the level floor (v1.0.4): a framework that runs your instrumentation hook in a different module context from your route handlers will otherwise build a second writer on the same file, and the two then rename it out from under each other at the rotation boundary. Rotation also refuses to overwrite an existing rotated file, so a day of records cannot be lost to a rename. A single file also has a ceiling (256 MB by default, <PREFIX>_LOG_JSON_MAX_MB / _TEXT_MAX_MB, 0 to remove): daily rotation bounds how many files there are, not how large one can get, and a service that drops to DEBUG or loops on an error can write more in an afternoon than the fortnight around it. Worst case per service is now a number an operator can reason about rather than the free space on the disk.
• Real spans, always (v1.1.0). startOtel registers the tracer whether or not anything is exported, so span ids come from it either way: a request's SERVER span (named by its route template, continuing an inbound W3C traceparent, and using the framework's own SERVER span when it has opened one), withSpan and withTrace for units of work, and continueFrom for work another process or a later moment does on a request's behalf. Every record carries the ids of the span it was written in, and traceHeaders() sends them on, so one trace_id spans every hop.
• The span tree in the files. Each span at a process boundary (server, client, producer, consumer) and each span this service opens is written to the files when it ends, as one record: event: span.end with span.name, span.kind, span.status, dur_ms and its links. A tool that reads only the files can draw the tree an OTLP backend shows. A client call its own caller cancelled (an AbortError from an AbortSignal, such as a long poll stopped once it is no longer wanted) is no failure: it is written and exported as ok with call.cancelled; a timeout stays an error. A failed client span is a WARNING record, since it is another service's answer, which its caller retries or records in its own words. Any other failed span is an ERROR record. Both are exported as errors.
• Optional OTLP export to any OpenTelemetry collector, when OTEL_EXPORTER_OTLP_ENDPOINT is set; the first-party logs never depend on it. What leaves the process passes an allowlist (no credentials, network addresses, user agents, query strings, full URLs or personal identifiers; more with exportDenyKeys; a key in any spelling, studentSub as student_sub). Every exported text (messages, error messages, string attributes, a span's status and exception) passes a built-in scrub of email addresses, ids a sign-in provider issued (teachable:4242424; more with identityPrefixes), JSON Web Tokens and credentials in a query, then the service's own exportScrub; the files keep everything. Export is seen, not assumed: an obs.export record every minute counts what was sent and what failed, and a failure is noted in the files at most once a minute, never over the exporter that is failing.
• Serverless-aware: on Vercel or Lambda the file sinks switch themselves off rather than failing, because there is no persistent writable disk.

Usage

Once, from the app's instrumentation.ts, on the Node runtime:

export async function register() {
  if (process.env.NEXT_RUNTIME !== "nodejs") return;

  // Optional: start OTLP export first, so the earliest logs adopt OTel ids.
  const { startOtel } = await import("@diveinto/obs/start");
  await startOtel("my-service");

  const { createObs } = await import("@diveinto/obs");
  createObs({ service: "my-service", envPrefix: "MYSVC" });
}

Everywhere else, plain named imports:

import { getLogger, withApiTrace, withAction, traceHeaders } from "@diveinto/obs";

const log = getLogger("billing.invoice");

export const GET = (req: Request) =>
  withApiTrace(req, async () => {
    log.info("invoice.fetch", "fetching invoice", { invoice_id: id });
    const res = await fetch(url, { headers: traceHeaders() }); // same trace, next hop
    return Response.json(await res.json());
  });

withApiTrace opens or continues the trace, logs request.received and request.completed, records a duration histogram, and sets X-Trace-Id on the response. withAction does the same for a server action in a fresh root trace.

Work done later, or by another process, on a request's behalf: store the request's context with it, and continue from it.

import { continueFrom, currentTraceparent } from "@diveinto/obs";

// in the request: keep its context with the job
await db.insert(jobs).values({ id, traceparent: currentTraceparent() });

// in the worker: the job's span is a child of the request's
await continueFrom(job.traceparent, "process jobs", () => run(job), { kind: "consumer" });

A stored or forwarded context is untrusted input: one that is missing or malformed starts a new trace and never fails the work. links ties a span to the other requests it serves (a batch) without making it their child.

Configuration

| field | required | what it does | |---|---|---| | service | yes | The OpenTelemetry tracer and meter name, the service.name fallback, and the default var dir and file stem | | envPrefix | yes | The prefix on this service's own env vars. MYSVC reads MYSVC_LOG_TEXT_FILE, MYSVC_VAR_DIR and the rest, so a service adopting the library keeps the env file it already has | | defaultVarDir | no | State dir when <PREFIX>_VAR_DIR is unset. Default /var/lib/<service> | | fileStem | no | Stem for the two log files. Default: the last dash-separated word of service | | routeTemplate | no | Collapse dynamic path segments before a path names a server span or becomes a metric label: /orders/abc123 to /orders/{id}. Without it every id is its own time series. routeTemplateFor(["/orders/[id]", ...]) builds one from routes named as an app router names them. A framework's own request span named by its method alone, and its fetch spans named by a whole URL, are written and exported under the route (or collapseIds of the path), never with an id or a query | | traceSeed | no | Where a request's trace continues from when there is no W3C traceparent, for a front end that passes a correlation id of its own | | quietPathPrefixes | no | Request paths whose completion logs at debug rather than info, whose spans' records are debug, and whose spans are not exported unless they failed. Health checks and internal polling otherwise bury what matters | | exportDenyKeys | no | More attribute and field keys that never leave the process over OTLP, beside the built-in ones. The files keep them | | exportScrub | no | Rewrites exported text after the built-in identity scrub, for what only this service knows can name a person. The files keep text as written | | identityPrefixes | no | Prefixes of ids this service issues itself, as <prefix>:<id>, which the built-in scrub then replaces as it does a sign-in provider's |

Environment

Generic, never prefixed: LOG_LEVEL (trace/debug/info/warn/error/ fatal), LOG_FORMAT (pretty or json, stdout only), and the standard OTEL_* set.

Per service, prefixed: <PREFIX>_VAR_DIR, <PREFIX>_LOG_TEXT_FILE, _TEXT_ROTATION, _TEXT_RETENTION, _TEXT_MAX_MB, and the same four for _JSON_. A *_FILE value is a path, or 1 for the default path, or 0/off; *_MAX_MB is the per-file ceiling, 256 by default and 0 for none.

Cardinality

Metric attributes become a separate time series per distinct value, so they must be low cardinality: an outcome, a status class, a route template. Never a user id, order id or request id. Those belong on spans and logs, where the point is finding the one bad request, and are toxic on metrics. routeTemplate exists for exactly this reason.

The library registers no metric reader, so metrics are no-ops unless you wire one deliberately. That is a scar: auto-exporting metrics alongside traces once produced tens of millions of samples a month that nobody read.

Requirements

Node 22 or newer, ESM. Node only: the trace context is AsyncLocalStorage and the file sinks are node:fs, so in Next import it from the Node branch of instrumentation.ts and from route handlers, not from Edge middleware.

@opentelemetry/api and @opentelemetry/api-logs are required peers. The SDK packages (@vercel/otel, @opentelemetry/sdk-trace-base, @opentelemetry/sdk-logs, @opentelemetry/exporter-logs-otlp-proto, @opentelemetry/resources) are optional peers, imported dynamically by @diveinto/obs/start: install them to have real spans (@vercel/otel, and @opentelemetry/sdk-trace-base for export) and OTLP logs. Without them the library falls back to ids of its own on AsyncLocalStorage, as before v1.1.0.

A long-running process wants @opentelemetry/sdk-trace-node too (and @opentelemetry/core, which comes with it). Where VERCEL is not set, startOtel registers the SDK's own Node provider with it: W3C trace context and baggage, every span sampled. @vercel/otel's registration, kept where Vercel runs the process for its flush at the end of each invocation, takes the first span of a trace id it sees as that trace's root and ends every span of the trace still open when the root ends, so a process that runs two pieces of work continuing one stored trace at once would have the first to finish end the other's span under it. Without the Node provider installed, startOtel uses @vercel/otel's registration as before.

License

MIT