@octaviaflow/logger

Production-ready logger for Octaviaflow microservices. Structured logging with multi-transport support, size- and date-based file rotation, and optional remote shipping to New Relic or a custom HTTP endpoint.

Features

• Multi-service presets: backend, engine, and a browser-safe ui preset.
• Structured logs with rich typed metadata (requests, users, workflows, errors, etc.).
• Multiple output formats: json, pretty, compact, and ECS (Elastic Common Schema).
• Size-, date-, and age-based file rotation with optional gzip compression and retention.
• Cloud archive transports: AWS S3, Google Drive, OneDrive, Dropbox — credentials auto-loaded from env vars, never stored in the config file.
• Declarative logger.config.ts with loader + validator (scaffold it with npx @octaviaflow/cli logger init).
• Async-safe context via AsyncLocalStorage on Node, with an in-memory fallback in browsers.
• Specialized helpers for audit, security, performance, and metric events.
• Built-in sanitization for sensitive fields.
• Fully typed (TypeScript).

Installation

npm install @octaviaflow/logger

Requires Node.js >= 22.

Quick Start

Backend Service

import { createBackendLogger } from "@octaviaflow/logger/presets";

const logger = createBackendLogger({
  level: "info",
  version: "1.0.0",
});

logger.info("Server started", { port: 3000 });

logger.withRequest(
  {
    requestId: "req_123",
    method: "GET",
    path: "/api/workflows",
    ip: "127.0.0.1",
    protocol: "http",
    hostname: "localhost",
    url: "http://localhost:3000/api/workflows",
  },
  async () => {
    // request handler — every log emitted in this scope carries the request context.
    logger.info("Handling request");
    // ...
  },
);

Engine Service

import { createEngineLogger } from "@octaviaflow/logger/presets";

const logger = createEngineLogger({
  level: "info",
  version: "1.0.0",
  workerId: "worker_1",
});

await logger.withWorkflow(
  { workflowId: "wf_123", workflowName: "Data Sync", executionId: "exec_456" },
  async () => {
    logger.info("Processing workflow step");
    // ...
  },
);

UI Service (Browser)

The ui preset is safe for browser bundles. The package's browser field stubs Node-only modules (node:fs, node:os, node:async_hooks, …) so bundlers don't pull them in, and the preset avoids process.on handlers.

import { createUILogger } from "@octaviaflow/logger/presets";

const logger = createUILogger({
  level: "warn",
  version: "1.0.0",
  enableRemote: true,
  remoteUrl: "https://api.example.com/logs",
});

logger.info("User action", {
  action: "button_click",
  component: "WorkflowBuilder",
});

Note: in the browser, async context is backed by a single-slot in-memory store, not AsyncLocalStorage. This is not async-safe across concurrent chains — scope context tightly with logger.run(...).

API Reference

Basic Logging

logger.fatal("Fatal error");
logger.error("Error occurred");
logger.warn("Warning message");
logger.info("Info message");
logger.debug("Debug information");
logger.trace("Trace details");

Specialized Logging

logger.audit("user.login", {
  actor: { id: "user_123", type: "user" },
  resource: { type: "session", id: "session_456" },
  result: "success",
});

logger.security("auth_failure", {
  event: "auth_failure",
  severity: "high",
  action: "blocked",
});

logger.performance("api.request", 145, {
  metrics: { dbQueries: 3, cacheHits: 5 },
});

logger.metric("api.requests.total", 100);

Context Management (async-safe)

The recommended way to attach context to a scope is run / withRequest / withWorkflow. Context installed this way is held in AsyncLocalStorage and cannot leak into sibling async chains.

logger.run({ user: { userId: "user_123", organizationId: "org_456" } }, async () => {
  logger.info("Inside scope"); // carries user context
  await doWork();
});

logger.withRequest(req, async () => {
  logger.info("Inside request"); // carries request context
});

logger.withWorkflow(wf, async () => {
  logger.info("Inside workflow"); // carries workflow context
});

Child loggers share transports with the parent and layer on additional context:

const apiLogger = logger.child({ component: "api", endpoint: "/api/workflows" });
apiLogger.info("Handled"); // carries component/endpoint

setContext / clearContext / startRequest / startWorkflow still work for backwards compatibility, but they are deprecated and will emit a one-time console warning. They use AsyncLocalStorage.enterWith, which is not async-safe — context set in one async chain can leak into sibling chains. Prefer the run-based helpers above.

Utilities

logger.profile("operation_1");
// ... do work ...
logger.profile("operation_1"); // logs duration

const getElapsed = logger.startTimer();
// ... do work ...
const durationMs = getElapsed(); // returns milliseconds

await logger.flush();
await logger.close();

Configuration

import { Logger } from "@octaviaflow/logger";

const logger = new Logger({
  service: "backend",
  environment: "production",
  version: "1.0.0",
  level: "info",
  format: "json",
  transports: [
    { type: "console", enabled: true, level: "info", format: "pretty" },
    {
      type: "file",
      enabled: true,
      level: "info",
      file: {
        path: "./logs/app.log",
        maxSize: "10m",
        maxFiles: 10,
        compress: true,
        datePattern: "YYYY-MM-DD",
      },
    },
    {
      type: "http",
      enabled: true,
      level: "error",
      http: {
        url: "https://api.example.com/logs",
        batchSize: 10,
        flushInterval: 5000,
        maxBufferSize: 1000,
        retry: { attempts: 3, delay: 1000 },
      },
    },
  ],
  sanitize: { enabled: true, fields: ["password", "token", "apiKey"] },
  handleExceptions: true,
  handleRejections: true,
});

Log Formats

JSON (default for production)

{
  "timestamp": "2025-11-21T10:30:45.123Z",
  "level": "info",
  "service": "backend",
  "message": "API request completed",
  "requestId": "req_abc123",
  "statusCode": 200,
  "duration": 145
}

Pretty (default for development)

2025-11-21T10:30:45.123Z INFO  [backend] API request completed {"requestId":"req_abc123","statusCode":200}

Compact

10:30:45 I [backend] API request completed

ECS (Elastic Common Schema)

{
  "@timestamp": "2025-11-21T10:30:45.123Z",
  "log.level": "info",
  "message": "API request completed",
  "service": { "name": "backend", "version": "1.0.0" },
  "host": { "hostname": "api-server-1" }
}

File Rotation

Logs are rotated by:

• Size — when the current file reaches maxSize (default: 10 MB), the logger rolls over to a numeric suffix: app-2025-11-21.log → app-2025-11-21.1.log → app-2025-11-21.2.log, …
• Date — at the next calendar day, the suffix resets and the filename picks up the new date.
• Compression — old rollovers are gzipped if compress: true.
• Retention — only the newest maxFiles (default: 10) are kept; older files are deleted. If maxAgeDays is set, files whose mtime is older than the cutoff are also pruned (the currently-active log file is never deleted).

Note: file transport is Node-only and will throw if instantiated in a browser.

Config File (logger.config.ts)

For projects that prefer a declarative config over a runtime new Logger(...), use the config loader. Generate one interactively with the CLI (npx @octaviaflow/cli logger init) or write it yourself:

// logger.config.ts
import { defineLoggerConfig } from "@octaviaflow/logger/config";

export default defineLoggerConfig({
  service: "backend",
  mode: "production",
  version: "1.0.0",
  level: "info",
  format: "json",
  targets: {
    console: { enabled: true, format: "pretty" },
    file: {
      enabled: true,
      dir: "./logs",
      name: "app",
      compress: true,
      retention: { maxSizeMB: 10, maxFiles: 10, maxAgeDays: 30 },
    },
    s3: {
      enabled: true,
      bucket: "my-app-logs",
      region: "us-east-1",
      prefix: "backend",
      retention: { maxSizeMB: 10, maxFiles: 10, maxAgeDays: 30 },
      // envMap is optional — override env var names here if yours differ
      // envMap: { accessKeyId: "MY_AWS_KEY", secretAccessKey: "MY_AWS_SECRET" },
    },
  },
});

Then bootstrap a logger from it:

// src/logger.ts
import { loadLoggerConfig } from "@octaviaflow/logger/config";

export const logger = await loadLoggerConfig();

Or, if you already have the config object in-process:

import { createLoggerFromConfig } from "@octaviaflow/logger/config";
import config from "../logger.config.js";

export const logger = createLoggerFromConfig(config);

loadLoggerConfig(configPath?, cwd?) searches logger.config.{ts,mjs,js,json} in cwd (default process.cwd()) and returns a ready-to-use Logger. A .ts config requires a TS loader (e.g. tsx, ts-node) registered in the running process — or compile to .mjs/.js.

Use validateLoggerFileConfig(config) to lint the file shape in tests or build scripts without instantiating the logger. It collects every issue in a single pass rather than bailing on the first error.

Cloud Transports

Four cloud archive targets are available under @octaviaflow/logger/cloud (also reachable via the targets block in logger.config.ts). They all share the same contract: batch NDJSON, flush on size or time or close(), and enforce retention (maxSizeMB, maxFiles, maxAgeDays) against the remote listing.

Credentials never live in the config file. Each transport reads env vars at construction time. Override the env var names via the optional envMap, never the values.

| Target | Required env vars | Notes | | ---------- | -------------------------------------------------------------------- | --------------------------------------------------------------------- | | s3 | AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION | Optional AWS_SESSION_TOKEN. Pure WebCrypto SigV4 — no aws-sdk. | | gdrive | GDRIVE_ACCESS_TOKEN | Multipart upload into the configured folderId. | | onedrive | ONEDRIVE_ACCESS_TOKEN | Microsoft Graph simple upload; 4 MB cap per batch. | | dropbox | DROPBOX_ACCESS_TOKEN | Dropbox-API-Arg header; non-ASCII chars escaped. |

Every cloud transport writes objects keyed as <prefix>/YYYY/MM/DD/HH/<host>-<pid>-<timestamp>.log (or .log.gz with compress: true). The bounded ring buffer drops oldest on overflow and reports the drop count as a synthetic {"__meta":"dropped"} line in the next batch — matching the semantics of http / newrelic.

Pair with ods logger doctor to verify env vars and ods logger test to emit sample lines through every configured target.

Transport Reliability

• http and newrelic transports buffer up to maxBufferSize entries (default 1000). Excess entries are dropped (drop-oldest) and the count is reported in the next batch as logger.droppedCount.
• Retries use exponential backoff. 4xx responses are non-retryable except 408 and 429; 429 honors Retry-After.
• Flush timers are unref()-ed, so they never keep the Node process alive on their own.

Development

npm run build
npm run test
npm run typecheck
npm run lint
npm run bench

Performance

Targets (not guarantees — re-measure with npm run bench in your environment):

• Simple log (console transport): sub-millisecond per operation.
• Throughput: >10,000 ops/second for synchronous paths.

Actual numbers depend on Node version, CPU, disk (for file transport), and transport selection. The included benchmark uses Date.now() (millisecond precision), so for more accurate measurements switch to performance.now() and a larger iteration count.

License

Apache-2.0

Author

Octaviaflow Team