@nm-logger/logger

Minimal JSON logger for Express + Axios with optional S3 sync.

Features

• Logs handled Express requests as success
• Logs Express errors as error
• Logs Axios calls as external
• Optional database error logs as db_error
• Local daily files under configurable baseDir:
  • YYYY/MM/DD/daily_logs_success.json
  • YYYY/MM/DD/daily_logs_error.json
  • YYYY/MM/DD/daily_logs_external.json
  • YYYY/MM/DD/daily_logs_db_error.json
• S3: one JSON file per day per category (append by read + merge + put)
• Previous day's local folder is removed when the date changes
• employee_id taken from req.user.employee_id || req.user.emp_code || req.user.id (used internally only, not written to JSON)
• Each log entry includes:
  • url, method, type (internal/external/db_error)
  • status_code
  • user_agent
  • project_version (from Logger options)
  • error
  • optional response (for external errors)
  • count (when consecutive identical entries are compacted)
• External API logging:
  • Logs response body only for error-like calls (4xx/5xx or when a custom externalErrorDetector marks it as error)
  • Long responses are truncated with maxResponseLength

Install

npm install @nm-logger/logger

Basic usage (Express + Axios)

const express = require("express");
const axios = require("axios");
const Logger = require("@nm-logger/logger");

const app = express();

const nm_logger = new Logger(
  {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: "us-east-1",
    bucket: "project",
  },
  {
    baseDir: "/var/log/nm-logger",
    maskFields: ["aadhaar", "pan"],

    // Optional metadata for logs
    projectVersion: "Project-1.0.0",

    // Background S3 uploader (set to false if you prefer cron; see below)
    enableDailyWatcher: true,

    // Interval for background S3 uploads when enableDailyWatcher = true
    uploadIntervalMs: 60 * 1000, // 1 minute

    // How many characters of external response to keep (after masking)
    maxResponseLength: 4000,

    // If true (default), include external response body only for error-like calls
    logExternalResponseOnError: true,

    // Control logging of successful external calls
    // - true  (default): log all external calls
    // - false: log only error-like calls (4xx/5xx or externalErrorDetector = true)
    logExternalSuccess: true,

    // Optional semantic error detector for 2xx/3xx external responses
    // Example below in "External error-only logging"
    externalErrorDetector: null,
  }
);

// Express middlewares
app.use(express.json());
app.use(nm_logger.requestLoggerMiddleware());
app.use("/api", require("./routes"));
app.use(nm_logger.expressMiddleware());

// Attach Axios interceptor
nm_logger.attachAxiosLogger(axios);

app.listen(3000, () => {
  console.log("Server listening on 3000");
});

Folder permissions

Make sure the base directory exists and is writable by your Node process:

mkdir -p /var/log/nm-logger
chmod -R 777 /var/log/nm-logger

External error-only logging (recommended for high load)

If your external API returns HTTP 200 but the body contains error info, you can:

• Skip logging successful calls completely
• Still log error-like responses with full (masked + truncated) body

Example:

const nm_logger = new Logger(
  {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: "us-east-1",
    bucket: "project",
  },
  {
    baseDir: "/var/log/nm-logger",
    maskFields: ["aadhaar", "pan"],

    enableDailyWatcher: false, // use cron for S3 (see next section)
    logExternalSuccess: false, // do NOT log success responses

    externalErrorDetector: ({ response }) => {
      try {
        // Adjust this according to your external API response format
        if (response && response.success === false) return true;
        if (response && typeof response.status === "string" &&
            response.status.toLowerCase() === "error") return true;
        if (response && (response.error || response.error_code)) return true;
      } catch (e) {
        return false;
      }
      return false;
    },
  }
);

With this config:

• Only external calls that look like errors are logged (4xx/5xx or externalErrorDetector = true).
• For these, the masked + truncated response body is stored in JSON.

S3 upload modes

There are two ways to sync daily logs to S3.

Mode 1: Background timer (default)

If enableDailyWatcher: true (or omitted), the logger will:

• Start an internal setInterval in the same Node process
• Every uploadIntervalMs:
  • Read local daily log JSONs
  • Merge them with S3 files
  • Upload merged logs back to S3
  • Clean up previous day's local folder

This is simple but adds some extra work inside your API process.

Mode 2: Cron-based upload (recommended for production)

To keep your API process as light as possible, you can:

1. Disable the internal watcher in your API
2. Use a separate cron script to upload logs to S3

1. API server: disable watcher

const nm_logger = new Logger(
  {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: "us-east-1",
    bucket: "project",
  },
  {
    baseDir: "/var/log/nm-logger",
    maskFields: ["aadhaar", "pan"],
    enableDailyWatcher: false,  // 🔴 no setInterval in API process
  }
);

app.use(nm_logger.requestLoggerMiddleware());
app.use("/api", routes);
app.use(nm_logger.expressMiddleware());
nm_logger.attachAxiosLogger(axios);

This way, the API only writes local JSON files.

Mode 2b: node-cron inside Node (no system crontab)

If you are already using node-cron in your Node project, you can avoid Linux crontab entirely and schedule uploads from your app code.

1. Disable the internal watcher in your API (same as before):

const nm_logger = new Logger(
  {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: "us-east-1",
    bucket: "project",
  },
  {
    baseDir: "/var/log/nm-logger",
    maskFields: ["aadhaar", "pan"],
    enableDailyWatcher: false,  // no setInterval
  }
);

2. Use node-cron in your Node app to trigger uploads:

const cron = require("node-cron");
const Logger = require("@nm-logger/logger");

const nm_logger = new Logger(
  {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: "us-east-1",
    bucket: "project",
  },
  {
    baseDir: "/var/log/nm-logger",
    maskFields: ["aadhaar", "pan"],
    enableDailyWatcher: false,
  }
);

// Every 5 minutes
cron.schedule("*/5 * * * *", async () => {
  try {
    await nm_logger.uploadDailyLogsOnce();
    console.log("[nm-logger] ✅ Daily logs uploaded to S3 (node-cron)");
  } catch (err) {
    console.error("[nm-logger] ❌ Upload failed (node-cron):", err);
  }
});

This way you:

• Do not depend on OS-level crontab.
• Still avoid setInterval-based uploads inside the main request flow.
• Keep all scheduling logic inside Node where you are already comfortable.

3. System Crontab entry (optional, Linux)

If you prefer OS-level cron instead of node-cron, you can still use a separate script as before.

Create a separate script (for example in your backend root):

// upload-logs.js
require("dotenv").config();
const Logger = require("@nm-logger/logger");

async function main() {
  const logger = new Logger(
    {
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
      region: "us-east-1",
      bucket: "project",
    },
    {
      baseDir: "/var/log/nm-logger",
      maskFields: ["aadhaar", "pan"],
      enableDailyWatcher: false, // no internal timers here either
    }
  );

  try {
    await logger.uploadDailyLogsOnce();
    console.log("[nm-logger] ✅ Daily logs uploaded to S3");
  } catch (err) {
    console.error("[nm-logger] ❌ Failed to upload daily logs:", err);
    process.exitCode = 1;
  }
}

main();

On your server (Linux):

crontab -e

Add a line like:

*/5 * * * * /usr/bin/node /path/to/upload-logs.js >> /var/log/nm-logger-cron.log 2>&1

• This will not be created automatically by the package.
• You must add it manually using crontab -e.
• Make sure the node path (/usr/bin/node) and script path (/path/to/upload-logs.js) are correct for your server.

Notes

• The logger does not create or manage cron jobs automatically; you must configure node-cron or crontab yourself if you choose cron mode.
• For lower CPU and disk usage on busy servers:
  • Prefer logExternalSuccess: false + externalErrorDetector to log only problematic external calls.
  • Use cron-based upload (Mode 2 / Mode 2b) so S3 sync runs in a separate short-lived context.