@ciq-dev/neoiq-foundation-node

Node.js observability foundation for CommerceIQ services. Integrates with Groundcover via OpenTelemetry.

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   Our Service   │────▶│  OTEL Collector │────▶│   Groundcover   │
│  (auth-service) │     │   (in-cluster)  │     │   (dashboard)   │
└─────────────────┘     └─────────────────┘     └─────────────────┘
        │                        │
        ├── Traces (OTLP) ───────┤
        └── Metrics (OTLP) ──────┘

        │
        └── Logs (stdout) ──────▶ Groundcover Agent scrapes container logs

Features

• Traces: Automatic span creation for HTTP requests (incoming & outgoing)
• Metrics: HTTP request counts, durations, errors + custom business metrics
• Logs: Structured JSON logs with automatic trace context (traceId, spanId, correlationId)
• Selective Enablement: Enable only the features you need
• Type-Safe Configuration: Zod validation with helpful error messages
• Fastify Plugin: One-line integration for request lifecycle
• HTTP Client: Axios wrapper with retry, circuit breaker, and trace propagation
• Object Store Adapter: Provider-agnostic wrapper for cloud object storage (e.g. S3) to avoid direct SDK coupling

Quick Start

1. Install

npm install @ciq-dev/neoiq-foundation-node

# Peer dependencies
npm install zod @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/sdk-metrics \
  @opentelemetry/exporter-trace-otlp-grpc @opentelemetry/exporter-metrics-otlp-grpc \
  @opentelemetry/auto-instrumentations-node @opentelemetry/resources @opentelemetry/semantic-conventions \
  pino pino-pretty axios axios-retry opossum fastify-plugin

# Optional (only if you want AWS S3 implementation of ObjectStore)
npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

2. Initialize (First Line of the App)

import { createFoundation } from '@ciq-dev/neoiq-foundation-node';

// Create foundation instance (must be called BEFORE other imports)
const foundation = createFoundation({
  serviceName: 'auth-service',
  serviceVersion: '1.0.0',
});

// Export for use in other files
export { foundation };

3. Add Fastify Plugin

import Fastify from 'fastify';
import { foundation } from './foundation';

const app = Fastify();
app.register(foundation.fastifyPlugin);

4. Set Environment Variable in Kubernetes

env:
  - name: OTEL_EXPORTER_OTLP_ENDPOINT
    value: 'http://otel-stack-deployment-collector.observability.svc.cluster.local:4317'

That's it! Traces and metrics will flow to Groundcover.

Selective Feature Enablement

Enable only the features you need for lightweight deployments:

import { createFoundation } from '@ciq-dev/neoiq-foundation-node';

// Full observability (default)
const foundation = createFoundation({
  serviceName: 'auth-service',
});

// Logging only (lightweight mode)
const foundation = createFoundation({
  serviceName: 'simple-worker',
  features: {
    tracing: false,
    metrics: false,
    logging: true,
  },
});

// Custom auto-instrumentation
const foundation = createFoundation({
  serviceName: 'api-gateway',
  features: {
    tracing: true,
    metrics: true,
    autoInstrumentation: {
      http: true,
      fastify: true,
      mongodb: false,  // Not using MongoDB
      redis: false,    // Not using Redis
      pg: true,        // Using PostgreSQL
    },
  },
});

API Reference

createFoundation(options)

Create a foundation instance with full observability capabilities.

import { createFoundation } from '@ciq-dev/neoiq-foundation-node';

const foundation = createFoundation({
  // Required
  serviceName: 'my-service',
  
  // Optional
  serviceVersion: '1.0.0',        // Default: '1.0.0' or $SERVICE_VERSION
  environment: 'production',       // Default: 'development' or $NODE_ENV
  
  // Feature toggles
  features: {
    tracing: true,                 // Enable distributed tracing
    metrics: true,                 // Enable metrics collection
    logging: true,                 // Enable structured logging
    autoInstrumentation: {         // Fine-grained control
      http: true,
      fastify: true,
      express: true,
      mongodb: true,
      pg: true,
      mysql: true,
      redis: true,
      ioredis: true,
      fs: false,                   // Disabled by default (noisy)
      dns: false,                  // Disabled by default
    },
  },
  
  // OTEL configuration
  otel: {
    endpoint: 'http://...:4317',   // Default: cluster OTEL Collector
    metricsIntervalMs: 5000,       // Default: 5000 (5 seconds)
    traceSampleRate: 1.0,          // Default: 1.0 (100%)
  },
  
  // Logging configuration
  logging: {
    level: 'info',                 // 'debug' | 'info' | 'warn' | 'error'
    prettyPrint: false,            // Auto-detect from environment
  },
});

Foundation Instance

The foundation instance provides access to all observability features:

// Logging
foundation.logger.info({ userId: '123' }, 'User logged in');
foundation.logger.error({ error: err.message }, 'Failed to process');

// Metrics
const meter = foundation.getMeter('my-service');
const counter = meter.createCounter('logins.total');
counter.add(1, { provider: 'workos' });

// Tracing
const tracer = foundation.getTracer();
tracer.startActiveSpan('validateToken', (span) => {
  // ... do work
  span.end();
});

// HTTP Client
const client = foundation.createHttpClient({
  baseURL: 'http://user-service:3000',
  serviceName: 'user-service',
});

// Fastify Plugin
app.register(foundation.fastifyPlugin);

// Shutdown
await foundation.shutdown();

// Check features
console.log(foundation.features);
// { tracing: true, metrics: true, logging: true }

foundation.logger

Structured logger with automatic trace context injection.

foundation.logger.info({ userId: '123' }, 'User logged in');
foundation.logger.error({ error: err.message }, 'Failed to process');
foundation.logger.warn({ attempt: 3 }, 'Retry attempt');
foundation.logger.debug({ payload: data }, 'Processing request');

// Child logger for component-specific logging
const authLogger = foundation.logger.child({ component: 'auth' });
authLogger.info({}, 'Auth module initialized');

Log Output:

{
  "level": "info",
  "time": 1703318400000,
  "service": "auth-service",
  "traceId": "abc123...",
  "spanId": "def456...",
  "correlationId": "req-789",
  "userId": "123",
  "msg": "User logged in"
}

foundation.getMeter(name, version?)

Get a meter for custom metrics.

const meter = foundation.getMeter('auth-service');

// Counter
const counter = meter.createCounter('logins.total');
counter.add(1, { provider: 'workos' });

// Histogram
const histogram = meter.createHistogram('token.validation.duration');
histogram.record(150, { status: 'success' });

// Observable Gauge
meter.createObservableGauge('active_sessions', {}, (result) => {
  result.observe(getActiveSessions());
});

foundation.fastifyPlugin

Fastify plugin for automatic request handling.

app.register(foundation.fastifyPlugin);

Automatically:

• Extracts/generates correlation ID (x-request-id header)
• Creates OpenTelemetry spans for each request
• Logs request received/completed with full context
• Records HTTP metrics (http.server.requests.total, http.server.request.duration)
• Skips health check endpoints (/health, /healthz, /ready, /live)

foundation.createHttpClient(options)

Create an Axios client with full observability.

const userClient = foundation.createHttpClient({
  baseURL: 'http://user-service:3000',
  serviceName: 'user-service',
  timeout: 10000,
  retry: {
    retries: 3,
    retryDelay: 1000,
    retryStatusCodes: [408, 429, 500, 502, 503, 504],
  },
  circuitBreaker: {
    enabled: true,
    resetTimeout: 30000,
    errorThresholdPercentage: 50,
  },
});

const response = await userClient.get('/api/users/123');

Automatically:

• Propagates trace context (traceparent header)
• Propagates correlation ID (x-request-id header)
• Logs outbound requests/responses
• Records HTTP client metrics
• Retries on 5xx errors with exponential backoff
• Circuit breaker protection

foundation.shutdown()

Gracefully shutdown OTEL providers. Call on application shutdown.

process.on('SIGTERM', async () => {
  await app.close();
  await foundation.shutdown(); // Flush telemetry
});

Object Store Adapter (S3 / Cloud Abstraction)

Use this when you want to decouple application code from aws-sdk / cloud vendor SDKs.

AWS S3 (optional peer deps)

import { AwsS3ObjectStore } from '@ciq-dev/neoiq-foundation-node';

const store = new AwsS3ObjectStore({
  clientOptions: { region: process.env.AWS_REGION || 'us-east-1' },
});

const ref = { bucket: 'my-bucket', key: 'path/to/file.json' };

// Upload
await store.putObject(ref, JSON.stringify({ ok: true }), { contentType: 'application/json' });

// Presigned PUT URL (browser upload)
const uploadUrl = await store.presignPutObject(ref, { expiresInSeconds: 60, contentType: 'application/json' });

In-memory (local/tests)

import { InMemoryObjectStore } from '@ciq-dev/neoiq-foundation-node';

const store = new InMemoryObjectStore();
await store.putObject({ bucket: 'b', key: 'k' }, 'hello');
const obj = await store.getObject({ bucket: 'b', key: 'k' });

Complete Example

// src/foundation.ts
import { createFoundation } from '@ciq-dev/neoiq-foundation-node';

export const foundation = createFoundation({
  serviceName: 'auth-service',
  environment: process.env.NODE_ENV as 'development' | 'production',
});

// src/index.ts
import Fastify from 'fastify';
import { foundation } from './foundation';

const app = Fastify();

// Register observability plugin
app.register(foundation.fastifyPlugin);

// Create HTTP clients
const userClient = foundation.createHttpClient({
  baseURL: process.env.USER_SERVICE_URL!,
  serviceName: 'user-service',
});

// Custom metrics
const meter = foundation.getMeter('auth-service');
const loginCounter = meter.createCounter('auth.logins.total');

// Routes
app.post('/api/v1/auth/login', async (req, reply) => {
  const { provider } = req.body as { provider: string };
  
  loginCounter.add(1, { provider });
  foundation.logger.info({ provider }, 'Login attempt');
  
  // Call user service (trace context automatically propagated)
  const { data: user } = await userClient.get('/api/users/me');
  
  return { success: true, user };
});

// Start with graceful shutdown
app.listen({ port: 3000 });

process.on('SIGTERM', async () => {
  await app.close();
  await foundation.shutdown();
});

Environment Variables

| Variable | Description | Default | |----------|-------------|---------| | OTEL_EXPORTER_OTLP_ENDPOINT | OTEL Collector URL | http://otel-stack-deployment-collector.observability.svc.cluster.local:4317 | | SERVICE_VERSION | Service version | 1.0.0 | | NODE_ENV | Environment | development | | LOG_LEVEL | Log level | info |

What Gets Sent to Groundcover

Traces

• Every HTTP request (incoming and outgoing)
• Correlation ID linking requests across services
• Span attributes: method, url, status_code, duration

Metrics

• http.server.requests.total - Incoming request count
• http.server.request.duration - Incoming request latency
• http.server.requests.errors - Incoming request errors
• http.client.requests.total - Outgoing request count
• http.client.request.duration - Outgoing request latency
• Custom business metrics you define

Logs (via stdout)

• Structured JSON logs written to stdout (Pino)
• Include traceId, spanId, correlationId for correlation
• Groundcover Agent scrapes container logs
• Automatically correlated with traces in Groundcover dashboard

Project Structure

src/
├── index.ts                    # Main exports
├── config.ts                   # Zod configuration schemas
├── foundation.ts               # createFoundation() entry point
├── features/
│   ├── context.ts              # AsyncLocalStorage context manager
│   ├── logging.ts              # Pino logger setup
│   ├── tracing.ts              # OTEL trace provider
│   └── metrics.ts              # OTEL meter provider
└── integrations/
    ├── fastify-plugin.ts       # Fastify observability plugin
    ├── http-client.ts          # Axios wrapper with observability
    └── object-store/           # Cloud object store abstraction (S3, etc.)

Development

# Install dependencies
npm install

# Build
npm run build

# Type check
npm run typecheck

# Lint
npm run lint

# Format
npm run prettier:write

License

MIT