@egintegrations/observability

Observability utilities for logging (Winston), metrics (Prometheus), and telemetry (OpenTelemetry). Designed for use across API, BOT, ETL, and service applications.

Features

• Logging: Winston-based logging with configurable service name, level, and format
• Metrics: Prometheus metrics with configurable namespace and framework-agnostic design
• Telemetry: OpenTelemetry integration with auto-instrumentation support
• TypeScript: Full TypeScript support with type definitions
• Dual Module: ESM and CommonJS support
• Framework Agnostic: No Express or framework-specific dependencies in core functionality

Installation

npm install @egintegrations/observability

Quick Start

import {
  createLogger,
  initializeMetrics,
  initializeTelemetry,
} from '@egintegrations/observability';

// Set up logging
const logger = createLogger({ serviceName: 'my-service' });
logger.info('Service starting');

// Set up metrics
const metrics = initializeMetrics({ namespace: 'myapp' });
metrics.httpRequestsTotal.inc({ method: 'GET', path: '/health', status: '200' });

// Set up telemetry
const telemetry = initializeTelemetry({
  serviceName: 'my-service',
  serviceVersion: '1.0.0',
});

Logging

Basic Usage

import { createLogger } from '@egintegrations/observability';

const logger = createLogger({ serviceName: 'my-service' });

logger.info('Application started');
logger.warn('Low memory warning');
logger.error('Database connection failed');
logger.debug('Processing item', { itemId: 123 });

Configuration Options

import { createLogger } from '@egintegrations/observability';

const logger = createLogger({
  serviceName: 'my-service',
  level: 'debug', // Log level: error, warn, info, debug
  format: 'json', // 'json' or 'text'
  additionalMeta: {
    environment: 'production',
    version: '1.0.0',
  },
});

Configuration:

• serviceName (required): Name of your service
• level (optional): Log level (default: info or LOG_LEVEL env var)
• format (optional): Output format - 'json' or 'text' (default: 'text' in development, 'json' in production)
• additionalMeta (optional): Additional metadata to include in all logs

Child Loggers with Correlation IDs

import { createLogger, createChildLogger } from '@egintegrations/observability';

const logger = createLogger({ serviceName: 'api-service' });

// Create a child logger for a specific request
const requestLogger = createChildLogger(logger, 'req-12345');
requestLogger.info('Processing request'); // Includes correlationId: 'req-12345'

Quick Setup

import { createDefaultLogger } from '@egintegrations/observability';

// Simple setup with defaults
const logger = createDefaultLogger('my-service');
logger.info('Quick and easy!');

Metrics

Basic Usage

import { initializeMetrics } from '@egintegrations/observability';

const metrics = initializeMetrics({ namespace: 'myapp' });

// Track HTTP requests
metrics.httpRequestsTotal.inc({ method: 'GET', path: '/api/users', status: '200' });
metrics.httpRequestDuration.observe(
  { method: 'GET', path: '/api/users', status: '200' },
  0.123 // duration in seconds
);

// Track tool executions
metrics.toolExecutionsTotal.inc({ tool: 'database-query', status: 'success' });
metrics.toolExecutionDuration.observe(
  { tool: 'database-query', status: 'success' },
  0.5
);

// Track errors
metrics.errorTotal.inc({ type: 'ValidationError' });

// Track idempotency
metrics.idempotencyHits.inc({ tool: 'payment-processor' });
metrics.idempotencyMisses.inc({ tool: 'payment-processor' });

// Update health status
metrics.healthStatus.set(1); // 1 = healthy, 0 = unhealthy

Configuration Options

import { initializeMetrics } from '@egintegrations/observability';

const metrics = initializeMetrics({
  namespace: 'myapp', // Prefix for all metrics (default: 'app')
  enableDefaultMetrics: true, // Enable Node.js default metrics (default: true)
});

Available Metrics

All metrics are prefixed with the configured namespace (e.g., myapp_):

| Metric | Type | Labels | Description | |--------|------|--------|-------------| | {namespace}_http_requests_total | Counter | method, path, status | Total HTTP requests | | {namespace}_http_request_duration_seconds | Histogram | method, path, status | HTTP request duration | | {namespace}_tool_executions_total | Counter | tool, status | Total tool executions | | {namespace}_tool_execution_duration_seconds | Histogram | tool, status | Tool execution duration | | {namespace}_errors_total | Counter | type | Total errors | | {namespace}_idempotency_hits_total | Counter | tool | Idempotency cache hits | | {namespace}_idempotency_misses_total | Counter | tool | Idempotency cache misses | | {namespace}_health_status | Gauge | - | Health status (1=healthy, 0=unhealthy) |

Exposing Metrics Endpoint

Express Example

import express from 'express';
import { initializeMetrics, getMetricsOutput, getMetricsContentType } from '@egintegrations/observability';

const app = express();
const metrics = initializeMetrics({ namespace: 'myapi' });

app.get('/metrics', async (req, res) => {
  res.set('Content-Type', getMetricsContentType(metrics.registry));
  res.send(await getMetricsOutput(metrics.registry));
});

app.listen(3000);

HTTP Server Example

import http from 'http';
import { initializeMetrics, getMetricsOutput, getMetricsContentType } from '@egintegrations/observability';

const metrics = initializeMetrics({ namespace: 'myapp' });

const server = http.createServer(async (req, res) => {
  if (req.url === '/metrics') {
    res.setHeader('Content-Type', getMetricsContentType(metrics.registry));
    res.end(await getMetricsOutput(metrics.registry));
  } else {
    res.statusCode = 404;
    res.end('Not found');
  }
});

server.listen(9090);

Middleware Example

import { initializeMetrics } from '@egintegrations/observability';

const metrics = initializeMetrics({ namespace: 'api' });

// Express middleware to track request metrics
app.use((req, res, next) => {
  const start = Date.now();

  res.on('finish', () => {
    const duration = (Date.now() - start) / 1000;
    metrics.httpRequestsTotal.inc({
      method: req.method,
      path: req.route?.path || req.path,
      status: res.statusCode.toString(),
    });
    metrics.httpRequestDuration.observe(
      {
        method: req.method,
        path: req.route?.path || req.path,
        status: res.statusCode.toString(),
      },
      duration
    );
  });

  next();
});

Telemetry

Basic Usage

import { initializeTelemetry } from '@egintegrations/observability';

const telemetry = initializeTelemetry({
  serviceName: 'my-service',
  serviceVersion: '1.0.0',
});

// Telemetry is now active and will automatically instrument:
// - HTTP requests/responses
// - Database queries
// - External API calls
// - And more...

Configuration Options

import { initializeTelemetry } from '@egintegrations/observability';

const telemetry = initializeTelemetry({
  serviceName: 'my-service',
  serviceVersion: '1.0.0',
  otlpEndpoint: 'http://jaeger:4318/v1/traces', // Optional (default: env or localhost)
  enableAutoInstrumentation: true, // Enable auto-instrumentation (default: true)
  disableFileSystemInstrumentation: true, // Disable FS instrumentation (default: false)
});

Configuration:

• serviceName (required): Name of your service
• serviceVersion (required): Version of your service
• otlpEndpoint (optional): OTLP endpoint URL (default: OTEL_EXPORTER_OTLP_ENDPOINT env var or http://localhost:4318/v1/traces)
• enableAutoInstrumentation (optional): Enable automatic instrumentation (default: true)
• disableFileSystemInstrumentation (optional): Disable file system instrumentation (default: false)

Getting the SDK

import { getTelemetrySDK } from '@egintegrations/observability';

const sdk = getTelemetrySDK();
// Returns the active NodeSDK instance or null if not initialized

Graceful Shutdown

import { shutdownTelemetry } from '@egintegrations/observability';

// Shutdown telemetry gracefully
await shutdownTelemetry();

The telemetry module automatically handles SIGTERM and SIGINT signals for graceful shutdown.

Integration with Jaeger

// Set environment variable
process.env.OTEL_EXPORTER_OTLP_ENDPOINT = 'http://jaeger:4318/v1/traces';

// Or configure directly
initializeTelemetry({
  serviceName: 'my-service',
  serviceVersion: '1.0.0',
  otlpEndpoint: 'http://jaeger:4318/v1/traces',
});

Integration with Honeycomb

initializeTelemetry({
  serviceName: 'my-service',
  serviceVersion: '1.0.0',
  otlpEndpoint: 'https://api.honeycomb.io:443',
});

Set the API key via environment variable:

export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=YOUR_API_KEY"

Complete Example

import express from 'express';
import {
  createLogger,
  createChildLogger,
  initializeMetrics,
  initializeTelemetry,
  getMetricsOutput,
  getMetricsContentType,
} from '@egintegrations/observability';

// Initialize observability
const logger = createLogger({
  serviceName: 'my-api',
  level: 'info',
  additionalMeta: { environment: 'production' },
});

const metrics = initializeMetrics({
  namespace: 'myapi',
  enableDefaultMetrics: true,
});

const telemetry = initializeTelemetry({
  serviceName: 'my-api',
  serviceVersion: '1.0.0',
});

// Create Express app
const app = express();

// Metrics middleware
app.use((req, res, next) => {
  const start = Date.now();
  const reqLogger = createChildLogger(logger, req.id);

  reqLogger.info('Request received', {
    method: req.method,
    path: req.path,
  });

  res.on('finish', () => {
    const duration = (Date.now() - start) / 1000;

    metrics.httpRequestsTotal.inc({
      method: req.method,
      path: req.route?.path || req.path,
      status: res.statusCode.toString(),
    });

    metrics.httpRequestDuration.observe(
      {
        method: req.method,
        path: req.route?.path || req.path,
        status: res.statusCode.toString(),
      },
      duration
    );

    reqLogger.info('Request completed', {
      status: res.statusCode,
      duration,
    });
  });

  next();
});

// Metrics endpoint
app.get('/metrics', async (req, res) => {
  res.set('Content-Type', getMetricsContentType(metrics.registry));
  res.send(await getMetricsOutput(metrics.registry));
});

// Health endpoint
app.get('/health', (req, res) => {
  metrics.healthStatus.set(1);
  res.json({ status: 'healthy' });
});

// API endpoints
app.get('/api/users', (req, res) => {
  logger.info('Fetching users');
  res.json({ users: [] });
});

// Error handling
app.use((err, req, res, next) => {
  metrics.errorTotal.inc({ type: err.name });
  logger.error('Request error', {
    error: err.message,
    stack: err.stack,
  });
  res.status(500).json({ error: 'Internal server error' });
});

app.listen(3000, () => {
  logger.info('Server started', { port: 3000 });
});

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Run tests with coverage
npm test -- --coverage

# Type check
npm run typecheck

# Lint
npm run lint

# Watch mode
npm run dev

Contributing

Contributions are welcome! Please ensure:

1. All tests pass (npm test)
2. Code coverage remains ≥70%
3. TypeScript types are properly defined
4. Code follows the existing style (run npm run lint)

License

MIT © EGI Integrations

Extracted From

This package was extracted from the egi-botnet project's bot-sdk-node package and refactored to be configurable and framework-agnostic.

Related Packages

• @egintegrations/core-utils - Retry logic, error handling, health checks, idempotency

Support

For issues and questions, please open an issue on GitHub.