@bevingh/telemetry

0.2.1: createLogMiddleware now redacts credentials and no longer ships bodies by default — see "Redaction" below. 0.2.2: path is now the full request path (0.2.0/0.2.1 logged the Router-relative path, e.g. /login for /api/v1/auth/login).

PR-19 + PR-20 + Phase 25. Shared reporting client for health, free-form metrics, and (Phase 25) logs. PR-20 ported batching/retry from central-logging-service/client/log-shipper.js and added optional instanceId; Phase 25 added reportLog/createLogMiddleware, finishing that port — this package now covers everything log-shipper.js did, plus metrics/health, behind one unified key.

Purpose

Pure client for reporting app health, free-form metrics, and logs to a central collector (central-logging-service's /api/v1/metrics* and /api/v1/logs routes). No transport beyond fetch; no DB client; no required metric shape.

| Field | Value | |---|---| | surfaceShape | pure_core_plus_express_adapter | | dependsOnPackages | (none) | | status | built (not published this session) |

Public API

import { createTelemetryClient } from '@bevingh/telemetry';

const telemetry = createTelemetryClient({
  appId: 'academicx',
  collectorUrl: process.env.TELEMETRY_COLLECTOR_URL!,
  apiKey: process.env.TELEMETRY_API_KEY!,
  // optional:
  // instanceId: 'manual-id',
  // batchSize: 50,           // default — from log-shipper.js
  // flushIntervalMs: 5000,   // default — from log-shipper.js
  // maxBufferSize: 10000,    // default — from log-shipper.js overflow cap
  onReportError: (err, context) => logger.warn(`telemetry ${context}`, err),
});

await telemetry.reportMetrics({ students: 412 });
await telemetry.reportHealth();
await telemetry.reportLog({ path: '/students', statusCode: 200, duration: 42 });
// process exit: beforeExit / SIGINT / SIGTERM flush automatically
// or: await telemetry.stop();

Or auto-log every request via the Express adapter instead of calling reportLog by hand:

import { createLogMiddleware } from '@bevingh/telemetry/adapters/express';

app.use(createLogMiddleware({ client: telemetry }));

| Export | Role | |---|---| | createTelemetryClient | Buffered client + reportHealth / reportMetrics / reportLog / flush / stop | | createHealthHandler | Express /health adapter (pull; separate from push client) | | redactLogEntry (+ redactHeaders/redactValue/redactUrl/redactString) | 0.2.1 — the redaction createLogMiddleware applies; use it yourself before a direct reportLog call that carries request data | | createLogMiddleware | Express auto request/response logging via reportLog — the @bevingh/telemetry replacement for log-shipper.js's .middleware() |

Batching & retry (PR-20 — from log-shipper.js)

| Setting | Default | Source in log-shipper.js | |---|---:|---| | batchSize | 50 | options.batchSize \|\| 50 | | flushIntervalMs | 5000 | options.flushInterval \|\| 5000 | | maxBufferSize | 10000 | requeue path: if (buffer.length > 10000) buffer = buffer.slice(-10000) |

Behavior (same shape as LogShipper):

1. reportHealth / reportMetrics enqueue (never throw).
2. Flush when buffer.length >= batchSize, or on the interval timer (.unref()'d).
3. On ship failure: requeue drained items to the front of the buffer; call onReportError with context 'flush'.
4. If buffer exceeds maxBufferSize, drop oldest (slice(-maxBufferSize)).
5. Shutdown: beforeExit, SIGINT, SIGTERM call flush() (library does not process.exit — unlike log-shipper).

dryRun and fetchImpl still work: dryRun flushes without network; inject fetchImpl in tests.

Logs (Phase 25)

reportLog(entry?) shares the same buffer as reportHealth/reportMetrics, but flushes differently: the collector's POST /api/v1/logs takes a batch ({ logs: [...] }), unlike the metrics/health routes (one report per POST). So when a flush happens, every buffered log entry coalesces into a single POST instead of one request per entry — the metrics/health items in the same flush still go one-request-each, unchanged.

All fields are optional and default sensibly: timestamp → now, level → 'info', service → config.appId, traceId → a generated UUID. Needs a key with the logs:write scope (see "Auth" below).

await telemetry.reportLog({
  level: 'error',
  method: 'GET',
  path: '/v1/students',
  statusCode: 500,
  duration: 812,
  error: { message: 'timeout', code: 'ETIMEDOUT' },
});

createLogMiddleware({ client }) (from @bevingh/telemetry/adapters/express) wraps this for auto request/response logging — same shape as log-shipper.js's .middleware(), but pushing through the shared buffer instead of shipping one request at a time.

Redaction (0.2.1 — security fix, read before mounting the middleware)

0.2.0's createLogMiddleware shipped raw request headers, request bodies and response bodies, so every Authorization bearer token, Cookie, API key, login/reset password and issued access/refresh token went to the collector, which stores entries as-is (it does no redaction of its own). 0.2.1 fixes this in the package, so every app on ^0.2.0 gets it with a plain npm update:

| What | 0.2.1 behavior | |---|---| | Headers | authorization, proxy-authorization, cookie, set-cookie, and any header name matching /token\|secret\|signature\|api[-_]?key\|password\|session\|credential/i → "[REDACTED]". Other header values are JWT-scrubbed. | | Request/response bodies | Not shipped by default. includeBodies: true opts in — then deep-redacted by key name (password, token, secret, otp, pin, apiKey, cvv, cardNumber, signature, session, cookie, credential, privateKey…, any depth; a JSON-string body from res.json is parsed first). | | Query object + metadata.url | Same key rule; e.g. /reset?token=… → /reset?token=[REDACTED]. | | Anywhere else | JWT-shaped strings (eyJ….….…) → "[REDACTED]". | | Failure | If redaction (or your redact hook) throws, the entry is dropped, never shipped raw. |

app.use(createLogMiddleware({
  client: telemetry,
  includeBodies: false,              // default; true only if you know what your routes return
  redactHeaders: ['x-tenant-key'],   // extra names/RegExps, on top of the defaults
  redactKeys: [/^nationalId$/],      // extra body/query/metadata keys
  redact: (entry) => entry,          // final app-specific pass
}));

Direct reportLog calls are not auto-redacted (you control that data) — wrap them with redactLogEntry(entry) if they carry request data. Upgrading from 0.2.0: entries already in the collector from 0.2.0 still hold whatever was shipped; purge/rotate on the collector side as needed.

instanceId (PR-20)

• Optional on config.
• If omitted: once per process, ${process.env.K_REVISION ?? 'unknown'}-${randomHex}.
• Included on both health and metrics payloads.
• Apps that do not care can ignore it.

Never throws

Reporting failures never reject to the host request path. Only onReportError is notified.

Express /health (unchanged)

import { createHealthHandler } from '@bevingh/telemetry/adapters/express';
app.get('/health', createHealthHandler({ appId: 'academicx' }));

Uptime Kuma pull — separate from buffered push to the collector.

Collector / out of scope

• Collector routes & auth-scoping: central-logging-service (not this package). /api/v1/metrics, /api/v1/metrics/health, and /api/v1/logs all exist and share one unified, scoped auth (Phase 25) — see "Auth" below.
• AcademicX wiring: separate session.
• Not published to GitHub Packages/npm as of Phase 25.

Auth (Phase 25 — unified; read before wiring any app)

apiKey in TelemetryClientConfig is not the legacy flat key list central-logging-service used to check by itself. As of Phase 25, one per-app key (sk_live_/sk_test_, bcrypt-hashed server-side) authorizes whichever of logs/metrics/health it was issued scopes for — matched via @bevingh/auth's matchApiKey. Metrics routes additionally 403 if the report body's appId doesn't match the key's own subjectId, so a leaked AcademicX key still can't post as a different app.

Before wiring any app, issue it a key with the scopes it needs (logs:write for reportLog/createLogMiddleware, metrics:write for reportMetrics/reportHealth) via the collector's admin UI (/admin/keys.html), its setup wizard (npm run setup), or the CLI:

# on the collector, MongoDB reachable:
npm run generate-app-key -- academicx --scopes=logs:write,metrics:write --live
# prints a one-time sk_live_... key — put it in that app's TelemetryClientConfig.apiKey
# (a bcrypt hash is stored; the raw key is never persisted or shown again)

instanceId is always sent by this client on health/metrics reports (config value or the process-derived default) — the collector's Metric model requires it. Log entries do not carry instanceId (the collector's log schema has no such field).

Tests

npm run test -w @bevingh/telemetry

Original 6 (with batchSize: 1 so a single report still ships immediately in those cases) + batch threshold, interval flush, overflow, retry requeue, stop() flush, instanceId set/default, plus Phase 25's log-reporting and Express log-middleware suites (batching into one POST, default-filling, mixed-context flush, retry, dry run).