@cannyminds/common

A comprehensive utilities package for CannyMinds organization, featuring advanced logging capabilities with Pino, Express middleware, and Loki integration.

Installation

npm install @cannyminds/common

Logging Module

The logging module provides a complete logging solution built on top of Pino with Express middleware and Loki HTTP client for querying logs.

Quick Start

import { createLogger, requestLoggerMiddleware, errorLoggerMiddleware } from '@cannyminds/common/log';
import express from 'express';

// Create logger instance
const logger = createLogger({
  serviceName: 'my-api',
  level: 'info',
  lokiEnabled: true,
  lokiHost: 'http://localhost:3100'
});

// Setup Express app with logging middleware
const app = express();
app.use(requestLoggerMiddleware(logger));
app.use(errorLoggerMiddleware(logger));

// Use in routes
app.get('/users', (req, res) => {
  req.logger.info('Fetching users');
  req.logger.debug('database', { query: 'SELECT * FROM users' });
  res.json({ users: [] });
});

app.listen(3000);

Logger Configuration

The createLogger function accepts a configuration object with the following options:

interface LoggerConfig {
  serviceName: string;           // Required: Name of your service
  level?: LogLevel;             // Optional: Log level (default: 'info')
  lokiHost?: string;            // Optional: Loki server URL
  lokiEnabled?: boolean;        // Optional: Enable Loki transport (default: false)
  lokiBasicAuth?: {             // Optional: Basic auth for Loki
    username: string;
    password: string;
  };
  additionalLabels?: Record<string, string>; // Optional: Extra labels
  redactPaths?: string[];       // Optional: Paths to redact in logs
}

Environment-Based Configuration

The logger automatically detects NODE_ENV for output format only:

• Development: Uses custom colorized formatter with clean, human-readable output
• Production: Uses JSON output with optional Loki transport

Note: Log level is not automatically set based on environment. You have full control:

// Same level works in both environments
const logger = createLogger({
  serviceName: 'my-service',
  level: 'debug'  // Will work in dev and prod
});

// You can set environment-specific levels manually if desired
const logger = createLogger({
  serviceName: 'my-service',
  level: process.env.NODE_ENV === 'production' ? 'warn' : 'debug'
});

Available Log Levels (in order of severity):

• fatal - Application crashes/exits
• error - Error conditions that need attention
• warn - Warning conditions (default for production in examples)
• info - General information (default for all environments)
• debug - Debug information for development
• trace - Detailed trace information

Example Configurations

Basic Logger:

const logger = createLogger({
  serviceName: 'user-service'
});

Production Logger with Loki:

const logger = createLogger({
  serviceName: 'user-service',
  level: 'warn',  // Optional: reduce log noise in production
  lokiEnabled: true,
  lokiHost: 'https://loki.company.com',
  lokiBasicAuth: {
    username: process.env.LOKI_USERNAME!,
    password: process.env.LOKI_PASSWORD!
  },
  additionalLabels: {
    environment: process.env.NODE_ENV!,
    version: process.env.APP_VERSION!
  },
  redactPaths: ['user.password', 'auth.token']
});

Express Middleware

Request Logger Middleware

Automatically logs incoming requests and outgoing responses:

import { requestLoggerMiddleware } from '@cannyminds/common/log';

app.use(requestLoggerMiddleware(logger));

Features:

• Generates unique requestId for each request
• Attaches requestId to request object and response headers
• Creates child logger with requestId context
• Logs request details: method, path, query, userAgent, IP
• Logs response details: statusCode, duration, contentLength

Error Logger Middleware

Catches and logs errors with full context:

import { errorLoggerMiddleware } from '@cannyminds/common/log';

app.use(errorLoggerMiddleware(logger));

Features:

• Logs error details: name, message, stack trace
• Includes request context: method, path, headers, body
• Preserves requestId for tracing
• Passes error to next error handler

Using Request Logger in Routes

app.get('/api/users/:id', (req, res) => {
  const { id } = req.params;
  
  req.logger.info(`Fetching user ${id}`);
  
  try {
    const user = await getUserById(id);
    req.logger.debug('user-data', { userId: id, email: user.email });
    res.json(user);
  } catch (error) {
    req.logger.error(error, { userId: id });
    res.status(500).json({ error: 'Failed to fetch user' });
  }
});

Loki Client

Query logs from Grafana Loki using the HTTP API:

import { LokiClient } from '@cannyminds/common/log';

const lokiClient = new LokiClient({
  host: 'http://localhost:3100',
  basicAuth: {
    username: 'admin',
    password: 'admin'
  },
  timeout: 30000,
  defaultLimit: 1000
});

Configuration Options

interface LokiClientConfig {
  host: string;                 // Required: Loki server URL
  basicAuth?: {                 // Optional: Basic authentication
    username: string;
    password: string;
  };
  timeout?: number;             // Optional: Request timeout (default: 30000ms)
  defaultLimit?: number;        // Optional: Default query limit (default: 1000)
}

Querying Logs

Point-in-time Query:

const result = await lokiClient.query({
  query: '{service="user-service"} |= "error"',
  limit: 100,
  time: new Date()
});

Range Query:

const logs = await lokiClient.queryRange({
  query: '{service="user-service", level="error"}',
  start: new Date(Date.now() - 3600000), // 1 hour ago
  end: new Date(),
  limit: 500,
  direction: 'backward'
});

Get Available Labels:

const labels = await lokiClient.getLabels();
const serviceNames = await lokiClient.getLabelValues('service');

Tail Logs in Real-time:

for await (const logEntry of lokiClient.tail('{service="user-service"}')) {
  console.log(`[${logEntry.timestamp}] ${logEntry.logLine}`);
}

Common LogQL Queries

// All errors in the last hour
const errors = await lokiClient.queryRange({
  query: '{service="user-service"} |= "ERROR"',
  start: new Date(Date.now() - 3600000),
  end: new Date()
});

// Specific user activity
const userLogs = await lokiClient.queryRange({
  query: '{service="user-service"} |~ "userId.*123"',
  start: new Date(Date.now() - 86400000), // 24 hours
  end: new Date()
});

// HTTP 500 errors
const serverErrors = await lokiClient.queryRange({
  query: '{service="user-service"} | json | statusCode="500"',
  start: new Date(Date.now() - 3600000),
  end: new Date()
});

// Performance issues (slow requests)
const slowRequests = await lokiClient.queryRange({
  query: '{service="user-service"} | json | duration > 1000',
  start: new Date(Date.now() - 3600000),
  end: new Date()
});

Advanced Usage

Child Loggers

Create contextual loggers for specific operations:

app.post('/api/users', (req, res) => {
  const operationId = uuidv4();
  const operationLogger = req.logger.child({ 
    operationId, 
    operation: 'createUser' 
  });
  
  operationLogger.info('Starting user creation');
  
  // Use operationLogger throughout the request
  operationLogger.debug('validation', { email: req.body.email });
  operationLogger.info('User created successfully');
});

Development Logger Output

In development, the logger provides clean, colorized output with different behaviors by log level:

// Single-line output for info and warn (no object display)
logger.info('User logged in');              // Clean single line
logger.warn('High memory usage detected');  // Clean single line

// Debug shows objects for inspection
logger.debug({ userId: '123', email: '[email protected]' }, 'User data');
// Output: User data
//         {
//           "userId": "123", 
//           "email": "[email protected]"
//         }

// Error shows stack traces automatically
try {
  throw new Error('Something went wrong');
} catch (err) {
  logger.error({ err }, 'Database operation failed');
  // Output: Database operation failed
  //         Error: Something went wrong
  //             at Object.<anonymous> (/path/to/file:10:15)
  //             ...stack trace in red...
}

// Request logging with visual indicators
// → POST /api/users    (incoming request)
// ← 201 45ms          (outgoing response with color-coded duration)

Output Features:

• Timestamps in gray
• Log levels color-coded (INFO=green, WARN=yellow, ERROR=red, DEBUG=cyan)
• Service name in bold brackets
• Request IDs in cyan (last 8 characters)
• HTTP methods color-coded (GET=green, POST=yellow, DELETE=red, etc.)
• Status codes color-coded (2xx=green, 3xx=cyan, 4xx=yellow, 5xx=red)
• Durations color-coded (fast=green, medium=yellow, slow=red)
• Stack traces in red for errors
• Objects pretty-printed for debug level only

Custom Log Levels and Structured Logging

// Different logging methods
logger.fatal('System is shutting down');
logger.error({ err: new Error('DB failed') }, 'Database connection failed');
logger.warn('High memory usage detected');
logger.info('User logged in');
logger.debug({ key: 'user:123', ttl: 300 }, 'Cache hit');
logger.trace('Function entry');

Environment Variables

Configure the logger using environment variables:

# .env file
NODE_ENV=production
LOG_LEVEL=info
LOKI_HOST=https://loki.company.com
LOKI_USERNAME=service_account
LOKI_PASSWORD=secret_token
SERVICE_NAME=user-api

const logger = createLogger({
  serviceName: process.env.SERVICE_NAME || 'unknown-service',
  level: (process.env.LOG_LEVEL as LogLevel) || 'info',
  lokiEnabled: !!process.env.LOKI_HOST,
  lokiHost: process.env.LOKI_HOST,
  lokiBasicAuth: process.env.LOKI_USERNAME ? {
    username: process.env.LOKI_USERNAME,
    password: process.env.LOKI_PASSWORD!
  } : undefined
});

Development

Building the Package

npm run build

Testing

npm test

Publishing

npm publish

License

MIT

Contributing

Please read our contributing guidelines before submitting pull requests.

Support

For issues and questions, please use the GitHub issue tracker.