@usewhawit/collector-node

Plug-and-play observability for Node.js applications. Automatically captures console output, exceptions, and sends them to the Whawit platform.

Features

• 🔌 Plug-and-Play: Just initialize once, no code changes needed
• 📝 Auto-Capture Console: Automatically captures console.log, console.warn, console.error
• 🚨 Auto-Capture Exceptions: Captures unhandled exceptions and promise rejections
• 📊 Auto-Capture Host Metrics: CPU, memory, event loop lag, and Node.js process stats
• OpenTelemetry Compatible: Implements standard OTEL exporters for traces, logs, and metrics
• High Performance: Redis-backed buffer to prevent data loss during network issues
• Automatic Batching: Intelligent batching with configurable flush intervals
• Retry Logic: Exponential backoff for failed exports
• Zero Data Loss: Graceful shutdown ensures all buffered data is flushed

Installation

npm install @usewhawit/collector-node
# or
yarn add @usewhawit/collector-node

Quick Start (Plug-and-Play)

Just add this at the very beginning of your application (before any other code):

import { initWhawIt } from '@usewhawit/collector-node';

// Initialize once - that's it!
initWhawIt({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-service',
  environment: 'production',
});

// All your existing code works unchanged!
// console.log, console.error, etc. are automatically captured
console.log('This will appear in Whawit!');
console.error('Errors too!');

// Unhandled exceptions are automatically captured
throw new Error('This exception will be logged to Whawit');

Works with any Node.js application:

Express/NestJS:

// In your main.ts or app.ts - add at the very top
import { initWhawIt } from '@usewhawit/collector-node';

initWhawIt({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-api',
});

// Rest of your app
import express from 'express';
const app = express();
// ...

Simple Node.js script:

import { initWhawIt } from '@usewhawit/collector-node';

initWhawIt({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-script',
});

// Your script code - all console.log goes to Whawit
console.log('Starting process...');
console.warn('This is a warning');
console.error('Something went wrong!');

Auto-Capture Configuration

Control what gets automatically captured:

initWhawIt({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-service',

  // Auto-capture configuration (all enabled by default)
  autoCapture: {
    console: true,        // Capture console.log, console.warn, console.error
    exceptions: true,     // Capture unhandled exceptions and rejections
    consoleMinLevel: 'info',  // Minimum level: 'debug' | 'info' | 'warn' | 'error'
  },
});

Disable Auto-Capture

If you want to use the SDK manually (like traditional OpenTelemetry):

initWhawIt({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-service',
  
  // Disable auto-capture
  autoCapture: {
    console: false,
    exceptions: false,
  },
});

// Now you need to use the SDK explicitly
const sdk = getWhawItSDK();
const logger = sdk.getLogger('my-app');
logger.emit({
  severityNumber: SeverityNumber.INFO,
  body: 'My log message',
});

Host Metrics (Automatic)

The SDK automatically collects system and Node.js metrics:

| Metric | Description | |--------|-------------| | system.cpu.utilization | System CPU usage (user, system, idle) | | system.cpu.count | Number of CPU cores | | system.memory.usage | System memory (used, free) | | system.memory.utilization | Memory usage percentage | | system.load.average | System load average (1m, 5m, 15m) | | process.cpu.utilization | Process CPU usage | | process.memory.usage | Heap, RSS, external memory | | process.uptime | Process uptime in seconds | | nodejs.eventloop.lag | Event loop lag in ms | | nodejs.active_handles | Active handles (detect leaks) | | nodejs.active_requests | Active requests |

Configure Host Metrics

initWhawIt({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-service',

  // Host metrics configuration (all enabled by default)
  hostMetrics: {
    enabled: true,      // Enable/disable all host metrics
    cpu: true,          // CPU metrics
    memory: true,       // Memory metrics
    process: true,      // Node.js process metrics
    eventLoop: true,    // Event loop lag
  },
});

Configuration

Full Configuration Options

import { initWhawIt } from '@usewhawit/collector-node';

initWhawIt({
  // Required
  apiKey: 'your-api-key',
  projectId: 'urn:project:your-project-id', // Project URN from Whawit dashboard
  serviceName: 'my-service',

  // Optional
  endpoint: 'https://api.whawit.ai',  // Whawit API endpoint
  environment: 'production',            // Environment name
  serviceVersion: '1.0.0',              // Service version
  debug: false,                          // Enable debug logging

  // Redis buffer configuration (optional, falls back to in-memory)
  redis: {
    host: 'localhost',
    port: 6379,
    password: 'your-password',
    db: 0,
    keyPrefix: 'whawit:buffer',
    maxBufferSize: 10000,
    connectTimeout: 5000,
  },

  // Batch processing configuration
  batch: {
    maxBatchSize: 100,        // Max items per batch
    flushIntervalMs: 5000,    // Max time between flushes
    maxRetries: 3,            // Retry attempts for failed exports
    retryBaseDelayMs: 1000,   // Base delay for exponential backoff
    retryMaxDelayMs: 30000,   // Max delay for exponential backoff
    exportTimeoutMs: 30000,   // Export request timeout
  },

  // Additional resource attributes
  resourceAttributes: {
    'deployment.environment': 'production',
    'service.namespace': 'my-namespace',
  },
});

Using Individual Exporters

If you prefer to configure OpenTelemetry manually, you can use the exporters directly:

Trace Exporter

import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { WhawItTraceExporter } from '@usewhawit/collector-node';

const provider = new NodeTracerProvider();

const exporter = new WhawItTraceExporter({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-service',
});

provider.addSpanProcessor(new BatchSpanProcessor(exporter));
provider.register();

Log Exporter

import { LoggerProvider, BatchLogRecordProcessor } from '@opentelemetry/sdk-logs';
import { WhawItLogExporter } from '@usewhawit/collector-node';

const loggerProvider = new LoggerProvider();

const exporter = new WhawItLogExporter({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-service',
});

loggerProvider.addLogRecordProcessor(new BatchLogRecordProcessor(exporter));

Metric Exporter

import { MeterProvider, PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
import { WhawItMetricExporter } from '@usewhawit/collector-node';

const exporter = new WhawItMetricExporter({
  apiKey: process.env.WHAWIT_API_KEY!,
  projectId: process.env.WHAWIT_PROJECT_ID!,
  serviceName: 'my-service',
});

const meterProvider = new MeterProvider({
  readers: [
    new PeriodicExportingMetricReader({
      exporter,
      exportIntervalMillis: 60000,
    }),
  ],
});

Graceful Shutdown

The collector handles graceful shutdown automatically when using initWhawIt(). For manual setup:

process.on('SIGTERM', async () => {
  await exporter.shutdown();
  process.exit(0);
});

Data Flow

Your App → OpenTelemetry SDK → Whawit Exporter → Redis Buffer → Whawit API → ClickHouse

1. Your application emits traces, logs, and metrics via OpenTelemetry
2. Whawit exporters batch the data and buffer it in Redis
3. Batches are sent to the Whawit API via OTLP/HTTP
4. The API processes and stores data in ClickHouse for analysis

Environment Variables

| Variable | Description | Default | |----------|-------------|---------| | WHAWIT_API_KEY | Your Whawit API key | - | | WHAWIT_PROJECT_ID | Your project URN | - | | WHAWIT_ENDPOINT | API endpoint | https://api.whawit.ai | | WHAWIT_SERVICE_NAME | Service name | - | | WHAWIT_ENVIRONMENT | Environment | production | | WHAWIT_DEBUG | Enable debug logging | false |

License

MIT