heapspy 📊

Comprehensive, production-ready application metrics and performance monitoring for Node.js applications. Get real-time insights into your application's health, performance, and resource utilization with zero-configuration setup.

✨ Features

| Feature | Description | Why It Matters | |---------|-------------|----------------| | 🧠 Memory Intelligence | V8 heap analytics, memory leak detection, trend analysis | Prevent crashes before they happen | | ⚡ Performance Profiling | GC timing, event loop monitoring, request waterfalls | Identify bottlenecks in production | | 🌐 Request Tracking | Full HTTP request tracing with percentiles | Understand user experience | | 🖥️ System Monitoring | CPU, disk, network, and OS-level metrics | Know your infrastructure limits | | 🚨 Smart Alerting | Configurable thresholds with multiple notification channels | Get notified before issues impact users | | 📈 Multiple Exporters | Console, file, Prometheus, APM integrations | Use with your existing monitoring stack | | 🔍 Production-Ready | Minimal overhead, safe defaults, graceful degradation | Monitor without affecting performance |

📦 Installation

# npm
npm install heapspy

# yarn
yarn add heapspy

# pnpm
pnpm add heapspy

🚀 Quick Start

Basic Setup (30 seconds)

// app.js or index.js
const heapspy = require('heapspy');

// Initialize with auto-start
const metrics = heapspy({
  enabled: true,
  serviceName: 'my-awesome-api',
  environment: process.env.NODE_ENV || 'development'
});

// That's it! Monitoring started automatically

Express.js Integration

const express = require('express');
const heapspy = require('heapspy');

const app = express();
const metrics = heapspy();

// Add heapspy middleware
app.use(metrics.middleware());

// Your routes
app.get('/api/users', async (req, res) => {
  // Automatic request tracking
  const users = await User.find();
  res.json(users);
});

// Health check endpoint
app.get('/health', (req, res) => {
  res.json(metrics.health());
});

// Metrics endpoint (Prometheus format)
app.get('/metrics', metrics.endpoint());

app.listen(3000, () => {
  console.log('Server with heapspy monitoring running!');
  console.log('Health: http://localhost:3000/health');
  console.log('Metrics: http://localhost:3000/metrics');
});

🔧 Configuration

heapspy supports flexible configuration through constructor options or environment variables:

Constructor Options

const metrics = heapspy({
  // Basic
  enabled: process.env.NODE_ENV !== 'test',
  serviceName: 'user-service',
  environment: process.env.NODE_ENV,
  
  // Monitoring intervals
  sampleInterval: 5000,           // 5 seconds
  gcSampleInterval: 10000,        // 10 seconds for GC
  
  // Memory thresholds (percentage)
  memoryWarningThreshold: 0.7,    // 70%
  memoryCriticalThreshold: 0.85,  // 85%
  
  // Alerting
  alerts: {
    enabled: true,
    webhook: process.env.SLACK_WEBHOOK_URL,
    email: '[email protected]',
    pagerDuty: process.env.PAGERDUTY_KEY
  },
  
  // Exporters
  exporters: ['console', 'prometheus', 'datadog'],
  
  // Custom metrics
  customLabels: {
    region: process.env.AWS_REGION,
    version: process.env.APP_VERSION
  },
  
  // Performance
  captureHttpRequests: true,
  captureDatabaseQueries: true,
  captureExternalCalls: true,
  
  // Storage
  historySize: 1000,  // Keep last 1000 samples
  retentionDays: 7    // Keep data for 7 days
});

Environment Variables

HEAPSPY_ENABLED=true
HEAPSPY_SERVICE_NAME=api-gateway
HEAPSPY_ENVIRONMENT=production
HEAPSPY_SAMPLE_INTERVAL=5000
HEAPSPY_MEMORY_WARNING_THRESHOLD=0.7
HEAPSPY_EXPORTERS=console,prometheus
HEAPSPY_ALERT_WEBHOOK=https://hooks.slack.com/...

📊 API Reference

Core Methods

// Initialize
const metrics = heapspy(config);

// Get current metrics snapshot
const snapshot = metrics.get();

// Get health status
const health = metrics.health();

// Get Prometheus metrics
const prometheus = metrics.prometheus();

// Track custom metric
metrics.counter('user.login', 1, { provider: 'google' });
metrics.gauge('queue.size', 42);
metrics.histogram('response.time', 245, { endpoint: '/api/users' });

// Measure function execution
const result = await metrics.timer('database.query', async () => {
  return await db.query('SELECT * FROM users');
});

// Create a trace
const trace = metrics.trace('checkout.process');
// ... your code ...
trace.end();

Middleware

// Express middleware
app.use(metrics.middleware());

// Koa middleware
app.use(metrics.koa());

// Fastify plugin
fastify.register(metrics.fastify());

// HTTP server wrapper
const server = metrics.wrap(createServer(app));

Database Monitoring

// MongoDB (mongoose)
metrics.monitor.mongoose(mongoose);

// PostgreSQL (pg)
metrics.monitor.pg(pool);

// Redis
metrics.monitor.redis(client);

// MySQL
metrics.monitor.mysql(connection);

External Services

// Axios
const client = metrics.monitor.axios(axios);

// Fetch API
const fetch = metrics.monitor.fetch(global.fetch);

// HTTP module
const http = metrics.monitor.http(require('http'));

📈 Exporters & Integrations

heapspy supports multiple exporters out of the box:

Console Exporter (Development)

// Prints formatted metrics to console
heapspy({ exporters: ['console'] });

Prometheus Exporter (Production)

// Exposes /metrics endpoint in Prometheus format
heapspy({ exporters: ['prometheus'] });

File Exporter (Logging)

// Writes metrics to files
heapspy({ 
  exporters: ['file'],
  filePath: './logs/metrics'
});

Cloud Exporters

// Datadog
heapspy({ 
  exporters: ['datadog'],
  datadog: { apiKey: 'xxx', appKey: 'xxx' }
});

// New Relic
heapspy({ exporters: ['newrelic'] });

// AWS CloudWatch
heapspy({ exporters: ['cloudwatch'] });

Custom Exporter

heapspy({
  exporters: [{
    name: 'custom',
    export: (metrics) => {
      // Send to your internal system
      myInternalSystem.send(metrics);
    }
  }]
});

🚨 Alerting

Configure alerts for critical conditions:

const metrics = heapspy({
  alerts: {
    rules: [
      {
        name: 'high-memory',
        condition: 'memory.heap.usage > 0.85',
        severity: 'critical',
        channels: ['slack', 'email']
      },
      {
        name: 'slow-response',
        condition: 'requests.p95 > 2000',
        severity: 'warning',
        channels: ['slack']
      },
      {
        name: 'high-error-rate',
        condition: 'requests.error_rate > 0.05',
        severity: 'critical',
        channels: ['slack', 'pagerduty']
      }
    ],
    
    // Notification channels
    slack: { webhook: process.env.SLACK_WEBHOOK },
    email: { 
      from: '[email protected]',
      to: ['[email protected]']
    },
    pagerDuty: { serviceKey: process.env.PD_KEY }
  }
});

🔍 Advanced Usage

Memory Leak Detection

// Enable automatic leak detection
const metrics = heapspy({
  memory: {
    leakDetection: {
      enabled: true,
      sampleCount: 20,
      increaseThreshold: '10%', // Alert if memory grows >10% over 20 samples
      checkInterval: 60000      // Check every minute
    }
  }
});

// Manually take heap snapshot
await metrics.memory.snapshot('before-operation');

// Your code...

await metrics.memory.snapshot('after-operation');

// Compare snapshots
const comparison = await metrics.memory.compare(
  'before-operation',
  'after-operation'
);
console.log('Objects retained:', comparison.retainedObjects);

Custom Performance Tracing

// Create a distributed trace
const trace = metrics.trace('user.checkout', {
  traceId: req.headers['x-trace-id'],
  parentSpanId: req.headers['x-span-id']
});

// Add spans
const authSpan = trace.span('authentication');
// ... auth logic ...
authSpan.end({ userId: user.id });

const paymentSpan = trace.span('payment.process');
// ... payment logic ...
paymentSpan.end({ amount: 99.99, currency: 'USD' });

trace.end({ success: true });

// Export traces to Jaeger/Zipkin
metrics.exporters.jaeger({
  endpoint: 'http://jaeger:14268/api/traces'
});

Custom Metrics Collection

// Create custom collectors
metrics.collector('business', async () => {
  return {
    active_users: await getActiveUserCount(),
    revenue_today: await getTodayRevenue(),
    pending_orders: await getPendingOrderCount()
  };
});

// Schedule collection
metrics.schedule('business', '*/5 * * * *'); // Every 5 minutes

📊 Dashboard & Visualization

heapspy includes optional dashboard support:

# Start heapspy dashboard
npx heapspy-dashboard

# Or embed in your app
const dashboard = metrics.dashboard();
app.use('/monitoring', dashboard);

Dashboard features:

• Real-time metrics visualization
• Historical trend analysis
• Alert management
• Service dependency graph
• Performance waterfall charts

🏗️ Architecture

graph TB
    A[Your App] --> B[heapspy Core]
    B --> C[Memory Monitor]
    B --> D[Performance Tracker]
    B --> E[Request Logger]
    B --> F[Resource Monitor]
    B --> G[Alert Manager]
    
    C --> H[Exporters]
    D --> H
    E --> H
    F --> H
    G --> H
    
    H --> I[Console]
    H --> J[Prometheus]
    H --> K[File]
    H --> L[Cloud Services]
    H --> M[Custom Exporters]

📈 Performance Impact

heapspy is designed for minimal overhead:

| Feature | Typical Overhead | Notes | |---------|-----------------|-------| | Basic Monitoring | < 1% CPU | Negligible impact | | Request Tracking | < 5ms per request | Async processing | | Memory Monitoring | < 50MB additional | Configurable sample rate | | Full Monitoring | 2-5% total | Production-safe |

🔐 Security

• Endpoint Protection: Metrics endpoints can be secured with middleware
• Data Sanitization: Sensitive headers and data automatically redacted
• Rate Limiting: Built-in protection against metrics scraping attacks
• CORS: Configurable CORS policies for dashboard access

🧪 Testing

// In your tests
const heapspy = require('heapspy');

describe('My Service', () => {
  let metrics;
  
  beforeEach(() => {
    metrics = heapspy({ enabled: false }); // Disabled for tests
  });
  
  it('should track performance', async () => {
    metrics.enableForTest();
    
    await myFunction();
    
    const stats = metrics.get();
    expect(stats.performance.myFunction.avg).toBeLessThan(100);
  });
});

📚 Recipes

Docker Integration

FROM node:16-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install heapspy
COPY . .
CMD ["node", "--max-old-space-size=4096", "server.js"]

# Docker Compose
version: '3.8'
services:
  app:
    build: .
    environment:
      - HEAPSPY_ENABLED=true
      - HEAPSPY_EXPORTERS=prometheus
    ports:
      - "3000:3000"
      - "9090:9090"  # Prometheus metrics
  
  prometheus:
    image: prom/prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9091:9090"

Kubernetes Configuration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
      - name: app
        env:
        - name: HEAPSPY_ENABLED
          value: "true"
        - name: HEAPSPY_PROMETHEUS_PORT
          value: "9090"
        ports:
        - containerPort: 3000
        - containerPort: 9090  # Metrics port
        readinessProbe:
          httpGet:
            path: /health
            port: 3000
        livenessProbe:
          httpGet:
            path: /health
            port: 3000

🤝 Contributing

We welcome contributions! Here's how to get started:

1. Fork the repository
2. Clone your fork: git clone https://github.com/your-username/heapspy.git
3. Install dependencies: npm install
4. Run tests: npm test
5. Create a branch: git checkout -b feature/amazing-feature
6. Make your changes
7. Run tests again: npm test
8. Commit: git commit -m 'Add amazing feature'
9. Push: git push origin feature/amazing-feature
10. Open a Pull Request

Development Setup

# Clone and install
git clone https://github.com/yourusername/heapspy.git
cd heapspy
npm install

# Run tests
npm test

# Run linting
npm run lint

# Build
npm run build

# Start development server with examples
npm run dev

📄 License

MIT © Shahzer Baig

🆘 Support

• 📚 Documentation
• 🐛 Issue Tracker
• 💬 Discussions
• 📧 Email

🙏 Acknowledgments

• Inspired by popular monitoring solutions
• Built with performance and developer experience in mind
• Thanks to all contributors and users

Additional Documentation Files You Should Create:

1. API.md (Detailed API Documentation)

# heapspy API Reference

## Table of Contents
- [Constructor](#constructor)
- [Methods](#methods)
- [Middleware](#middleware)
- [Monitors](#monitors)
- [Exporters](#exporters)
- [Types](#types)

## Constructor
`heapspy(options: Options): MetricsInstance`

## Methods
Detailed documentation for each method...

## Examples
Complete examples for each use case...

2. RECIPES.md (Common Use Cases)

# heapspy Recipes

## Basic Express App
Complete example with Express...

## Microservices Architecture
Monitoring multiple services...

## Kubernetes Deployment
Configuration for K8s...

## Database Monitoring
MongoDB, PostgreSQL, Redis...

## Alerting Setup
Slack, PagerDuty, Email alerts...

3. TROUBLESHOOTING.md (Common Issues)

# Troubleshooting Guide

## Memory Issues
- High memory usage...
- Memory leaks...

## Performance Impact
- High CPU usage...
- Slow requests...

## Exporters Not Working
- Prometheus endpoint...
- File exports...

4. CHANGELOG.md (Version History)

# Changelog

## [1.0.0] - 2024-01-01
### Added
- Initial release
- Core monitoring features

## [1.1.0] - 2024-01-15
### Added
- Dashboard support
- Additional exporters...

5. CONTRIBUTING.md (For Developers)

# Contributing Guide

## Development Setup
Detailed setup instructions...

## Code Style
Linting, formatting rules...

## Testing
Test structure, writing tests...

## Pull Request Process
Review process, requirements...

6. examples/ Directory

Create an examples directory with:

• basic-app.js - Simple setup
• express-app.js - Full Express app
• microservice.js - Microservice example
• docker-compose.yml - Docker setup
• kubernetes.yaml - K8s deployment

7. TypeScript Definitions

// index.d.ts
declare module 'heapspy' {
  export interface Options { /* ... */ }
  export interface MetricsInstance { /* ... */ }
  export default function heapspy(options?: Options): MetricsInstance;
}