API Logger Middleware with MongoDB And Kafka.

A lightweight logging library for Express applications that supports both MongoDB and Kafka for log storage and streaming. Automatically logs API requests and provides functionality for manual logging.

Features

• Logs incoming API requests and outgoing responses
• Captures request/response bodies and headers
• Saves logs to MongoDB
• Custom logger function for manual log entries
• Optional Kafka integration for log streaming
• Automatic Kafka topic creation and consumption

Installation

npm install @shivamjha1288/logs-manager

Usage

1. Initialize the Logging System

import { logsInit } from "./your-log-file.js";

await logsInit({
  appName: "your-app-name",
  mongoUrl: "mongodb://localhost:27017/logsdb",
  enableKafka: true,
  kafkaConfig: {
    clientId: "your-app",
    brokers: ["localhost:9092"],
    groupId: "logs-group",
    topicConfig: {
      topicName: "logs-topic",
      numPartitions: 1,
      replicationFactor: 1,
    },
  },
});

2. Use as Middleware in Express

import express from "express";
import { apiRequestLogger } from "./your-log-file.js";

const app = express();
app.use(express.json());
app.use(apiRequestLogger());

3. Manually Log Data

import { logData } from "./your-log-file.js";

await logData("error", "Something went wrong", {
  error: { message: "Example error" },
});

4. Full Example

import express from "express";
import { logsInit, apiRequestLogger, logData } from "@shivamjha1288/logs-manager";

const app = express();
const port = 3000;
const mongoUrl = "mongodb://admin:admin@localhost:27017/logs?authSource=admin";
const appName = "test-logs";
const kafkaConfig={
  clientId:"logsapp",
  brokers:["localhost:9092"],
  topicConfig:{
    topicName:"logs-topic",
    partition:1,
    replicationFactor:1
  },
  groupId:"logs-group"
}

logsInit({
  appName,
  mongoUrl,
  enableKafka: true,
  kafkaConfig
});

app.use(express.json());
app.use(apiRequestLogger(mongoUrl));

app.get("/", (req, res) => {
  res.send("Hello from test app!");
});
app.get("/error", (req, res) => {
  res.status(500).json({ staus: 500, error: "Internal server error" });
});

app.post("/log", async (req, res) => {
  await logData(mongoUrl, "info", "Manually logged from /log route", {
    body: req.body,
  });
  res.send("Custom log saved.");
});

app.listen(port, () => {
  console.log(`http://localhost:${port}`);
});

Log Schema Fields

• appName: Application identifier (string, required)
• level: Log level (info, error, etc.)
• message: Log message
• timestamp: Date and time
• method: HTTP method (GET, POST, etc.)
• url: Request URL
• statusCode: HTTP status code
• responseTime: Time taken for response (ms)
• requestBody: Body of the request
• requestHeaders: Request headers
• responseBody: Body of the response
• responseHeaders: Response headers
• error: Error object if any

Dependencies

• mongoose
• kafkajs

License

ISC