flow-debugger

v1.9.9

Published

a month ago

Production-safe flow-level debugging SDK. Traces requests, measures timing, classifies errors, detects root causes, and provides endpoint analytics with a live dashboard.

🔍 flow-debugger v2.0

Enterprise-grade APM & debugging SDK for Node.js, Web, and React Native.

Traces requests step-by-step, measures timing, classifies errors, detects root causes, and provides endpoint analytics with a live dashboard.

🎉 NEW v2.0: Distributed Tracing, Metrics, Alerting, Anomaly Detection, Error Clustering, Trend Analysis, Log Correlation, Dependency Graphs!

🚀 Easy 3-Line Setup (Beginner Friendly)

Beginners can get a professional dashboard and request tracing with just 3 lines of code:

const debug = flowDebugger({ enableDashboard: true }); // 1. Init
app.use(debug.middleware);                             // 2. Middleware
mongoTracer(mongoose, { getTracer: debug.getTracer }); // 3. Auto-Trace

✨ Features

Core Features

Request Tracing — Every request gets a unique traceId with step-by-step timing
Auto-Instrument — MongoDB, MySQL, PostgreSQL, Redis — zero extra code needed
Root Cause Detection — Automatically identifies why a request failed or was slow
Error Classification — INFO / WARN / ERROR / CRITICAL auto-categorization
Timeout Detection — Catches hanging queries/operations with configurable timeouts
Slow Query Detection — Flags slow SQL/NoSQL queries with threshold alerts
Endpoint Analytics — Per-route stats: requests, errors, slow, avg/p95/max latency
Service Health Monitor — Real-time dependency health: Redis: healthy, Mongo: degraded
Live Dashboard — Beautiful analytics UI at /__debugger/dashboard
Console Timeline — Visual request timeline directly in your terminal
Production-Safe — Never blocks requests, never crashes your app, all try/catch wrapped
Sampling — Configurable sampling rate for high-traffic environments

🆕 Enhanced Observability (v2.0)

Distributed Tracing — W3C Trace Context for multi-service tracing
Metrics System — Counter, Gauge, Histogram, Summary with Prometheus export
Real-time Alerting — Webhooks, Slack, Discord integration
Anomaly Detection — Statistical anomaly detection (standard deviations)
Error Clustering — Groups similar errors using fingerprinting
Trend Analysis — Hourly/daily patterns for capacity planning
Log Correlation — Automatic traceId injection into console logs
Dependency Graph — Service dependency mapping and health tracking

📖 See Enhanced Observability Guide

📦 Installation

npm install flow-debugger

🚀 Quick Start (Express)

Basic Setup

import express from 'express';
import { flowDebugger, mongoTracer, mysqlTracer, pgTracer, redisTracer } from 'flow-debugger';

const app = express();

// 1. Initialize debugger
const debug = flowDebugger({
  slowThreshold: 300,      // ms — mark steps as slow
  slowQueryThreshold: 300, // ms — flag slow DB queries
  enableDashboard: true,   // serve /__debugger/dashboard
});

app.use(debug.middleware);

// 2. Auto-instrument databases (optional — zero code tracing!)
mongoTracer(mongoose, { getTracer: debug.getTracer });
mysqlTracer(mysqlPool, { getTracer: debug.getTracer });
pgTracer(pgPool, { getTracer: debug.getTracer });
redisTracer(redisClient, { getTracer: debug.getTracer });

// 3. Use manual steps in your routes
app.post('/api/login', async (req, res) => {
  const tracer = req.tracer;

  const user = await tracer.step('DB find user', async () => {
    return User.findOne({ email: req.body.email });
  }, { service: 'mongo' });

  await tracer.step('Redis cache session', async () => {
    return redis.set(`session:${user.id}`, JSON.stringify(user));
  }, { service: 'redis' });

  const token = await tracer.step('JWT generate', async () => {
    return jwt.sign({ id: user.id }, SECRET);
  });

  res.json({ token, user });
});

app.listen(3000);

🆕 Enhanced Observability Setup (v2.0)

import { flowDebugger, globalMetrics } from 'flow-debugger';

const debug = flowDebugger({
  // Core features
  slowThreshold: 300,
  enableDashboard: true,
  
  // ✅ Enhanced Observability
  enableDistributedTracing: true,    // W3C Trace Context
  enableLogCorrelation: true,        // Auto traceId in logs
  enableAnomalyDetection: true,      // Statistical anomalies
  anomalySensitivity: 2,
  enableTrendAnalysis: true,         // Pattern detection
  enableAlerting: true,              // Real-time alerts
  alertWebhooks: ['https://hooks.slack.com/...'],
  alertOnErrorSpike: true,
  errorSpikeThreshold: 5,
  enableErrorClustering: true,       // Group similar errors
});

app.use(debug.middleware);

// Custom metrics
app.post('/api/orders', async (req, res) => {
  globalMetrics.inc('orders_created');
  const order = await req.tracer.step('Create order', createOrder);
  globalMetrics.observe('order_value', order.total);
  res.json({ order });
});

// Prometheus metrics endpoint
app.get('/metrics', (req, res) => {
  res.type('text/plain');
  res.send(globalMetrics.exportPrometheus());
});


### Console Output:

┌─── flow-debugger ── req_m3k2_abc123_1 ── POST /api/login ─── │ [0ms] Request start │ [2ms] DB find user ✔ (14ms) [mongo] │ [16ms] Redis cache session ✔ (3ms) [redis] │ [20ms] JWT generate ✔ (1ms) │ [21ms] Response 200 │ │ Classification: INFO │ Total: 21ms └──────────────────────────────────────────────────────


---

## 🧩 Auto-Instrument (Zero Code Tracing)

```typescript
import { mongoTracer, mysqlTracer, pgTracer, redisTracer } from 'flow-debugger';

// Mongoose — patches Query.prototype.exec
mongoTracer(mongoose, { getTracer: debug.getTracer });

// mysql2 — patches pool.query() and pool.execute()
mysqlTracer(pool, { getTracer: debug.getTracer });

// pg — patches pool.query() and client.query()
pgTracer(pgPool, { getTracer: debug.getTracer });

// Redis — wraps get, set, del, hget, etc.
redisTracer(redisClient, { getTracer: debug.getTracer });

Auto output (no extra code needed):

MySQL SELECT users → 12ms ✔
Postgres SELECT orders → 18ms ✔
Redis SET session → 3ms ✔
Mongo users.findOne → 22ms ✔

⏱ Timeout Detection

await tracer.step('DB query', async () => {
  return db.query('SELECT * FROM orders');
}, { service: 'postgres', timeout: 2000 });

If the query takes >2s:

CRITICAL: DB query timed out after 2000ms
Root cause: DB timeout

🏷 Dependency Tagging

await tracer.step('Cache lookup', fn, { service: 'redis' });
await tracer.step('User query', fn, { service: 'postgres' });

Dashboard grouping:

Top failures:
  Redis     42%
  Postgres  33%
  External  12%

📊 Dashboard

Access the live analytics dashboard at:

http://localhost:3000/__debugger/dashboard

API Endpoints:

| Route | Description | |---|---| | GET /__debugger | Full analytics JSON | | GET /__debugger/dashboard | Live HTML dashboard | | GET /__debugger/health | Service health status | | GET /__debugger/endpoint?path=/api/login | Per-endpoint stats |

⚙️ Configuration

Core Configuration

flowDebugger({
  enabled: true,              // Enable/disable debugger
  slowThreshold: 300,         // ms — slow step threshold
  slowQueryThreshold: 300,    // ms — slow query threshold
  defaultTimeout: 30000,      // ms — default step timeout
  samplingRate: 1,            // 0-1 (1 = 100%)
  alwaysSampleErrors: true,   // Always trace errors even if sampled out
  maxTraces: 1000,            // Max traces in memory
  enableTimeline: true,       // Console timeline output
  enableDashboard: true,      // Serve /__debugger routes
  logger: console.log,        // Custom logger function
});

🆕 Enhanced Observability Configuration (v2.0)

flowDebugger({
  // Distributed Tracing
  enableDistributedTracing: true,    // W3C Trace Context propagation
  
  // Log Correlation
  enableLogCorrelation: true,        // Auto-inject traceId into console logs
  
  // Anomaly Detection
  enableAnomalyDetection: true,      // Statistical anomaly detection
  anomalySensitivity: 2,             // Standard deviations (default: 2)
  
  // Trend Analysis
  enableTrendAnalysis: true,         // Detect hourly/daily patterns
  trendResolutionMinutes: 60,        // Time bucket resolution (default: 60)
  
  // Alerting
  enableAlerting: true,              // Enable real-time alerts
  alertWebhooks: [                   // Webhook URLs (Slack, Discord, etc.)
    'https://hooks.slack.com/services/YOUR/WEBHOOK',
  ],
  alertOnCritical: true,             // Alert on CRITICAL errors
  alertOnErrorSpike: true,           // Alert on error spikes
  errorSpikeThreshold: 5,            // Errors in 5min window to trigger alert
  
  // Error Clustering
  enableErrorClustering: true,       // Group similar errors
  maxClusters: 50,                   // Max clusters to maintain
});

🔍 Root Cause Detection

The package automatically identifies the most likely cause of failures:

| Scenario | Root Cause | |---|---| | Response 500 + Redis failed | Redis GET failed: connection refused | | DB query 900ms + total latency high | Slow PostgreSQL query: SELECT orders (900ms) | | Step timeout after 2s | Payment API timed out after 2000ms |

📡 Subpath Imports

// Only import what you need
import { flowDebugger } from 'flow-debugger/express';
import { mongoTracer } from 'flow-debugger/mongo';
import { mysqlTracer } from 'flow-debugger/mysql';
import { pgTracer } from 'flow-debugger/postgres';
import { redisTracer } from 'flow-debugger/redis';

🐍 Python Support (FastAPI)

Flow Debugger now supports Python applications with FastAPI integration:

from fastapi import FastAPI
from flow_debugger.fastapi_middleware import FlowDebuggerFastAPI

app = FastAPI()

# Initialize debugger
debugger = FlowDebuggerFastAPI(
    app=app,
    debugger_url="http://localhost:3500/__debugger",
    enabled=True,
    slow_threshold=300
)

@app.get("/api/users")
async def get_users(request):
    tracer = request.state.tracer  # Available via middleware
    
    # Manual tracing
    users = await trace_async_step(
        request,
        "DB query users", 
        lambda: database.fetch_users(),
        service="postgres"
    )
    
    return {"users": users}

Python Manual Tracing

from flow_debugger.fastapi_middleware import traced_step

@app.post("/api/orders")
async def create_order(request):
    with traced_step(request.state.tracer, "Validate user", "auth"):
        user = await validate_user(request.user_id)
    
    with traced_step(request.state.tracer, "Create order", "postgres"):
        order = await create_order_in_db(request.order_data)
    
    return {"order": order}

📱 React Native Integration

For React Native applications, use the React Native SDK:

import { DebuggerProvider, useDebugger, traceAsyncOperation } from 'flow-debugger/react_native';

// Wrap your app with the provider
export default function App() {
  return (
    <DebuggerProvider debuggerUrl="http://localhost:3500/__debugger">
      <YourAppContent />
    </DebuggerProvider>
  );
}

// Use in your components
function MyComponent() {
  const { step, startTrace } = useDebugger();
  
  const fetchData = async () => {
    const traceId = startTrace('/api/data', 'GET');
    
    try {
      // Trace an async operation
      const result = await step(
        traceId,
        'API call',
        () => fetch('https://api.example.com/data').then(r => r.json()),
        { service: 'external' }
      );
      
      // Send trace to dashboard
      await sendTrace(traceId);
      
      return result;
    } catch (error) {
      await sendTrace(traceId);
      throw error;
    }
  };
  
  // Or use the utility function
  const fetchWithTrace = async () => {
    return await traceAsyncOperation(
      '/api/data',
      'Fetch data',
      () => fetch('https://api.example.com/data').then(r => r.json()),
      'external'
    );
  };
  
  return <View>...</View>;
}

React Native Fetch Interceptor

import { createFetchInterceptor } from 'flow-debugger/react_native';

// Replace global fetch with intercepted version
global.fetch = createFetchInterceptor("http://localhost:3500/__debugger");

🏗 Architecture

core/
  TraceEngine         — request trace lifecycle
  Classifier          — INFO/WARN/ERROR/CRITICAL
  RootCause           — failure origin detection
  Analytics           — endpoint-level aggregation
  HealthMonitor       — dependency health tracking
  Timeline            — console renderer
  DistributedTracing  — W3C Trace Context (NEW v2.0)
  Metrics             — Counter/Gauge/Histogram/Summary (NEW v2.0)
  Alerting            — Webhooks & Slack integration (NEW v2.0)
  AnomalyDetection    — Statistical anomaly detection (NEW v2.0)
  ErrorClustering     — Group similar errors (NEW v2.0)
  TrendAnalysis       — Pattern detection (NEW v2.0)
  LogCorrelation      — Trace ID injection (NEW v2.0)
  DependencyGraph     — Service mapping (NEW v2.0)

integrations/
  mongo           — Mongoose auto-tracing
  mysql           — mysql2 auto-tracing
  postgres        — pg auto-tracing
  redis           — Redis auto-tracing
  fetch           — Global fetch tracing
  axios           — Axios interceptor

middleware/
  express         — Express middleware + dashboard

python-sdk/
  fastapi_middleware — FastAPI integration
  flow_debugger_core — Core Python classes

react-native/
  react_native    — React Native integration

🌐 Enhanced API Endpoints (v2.0)

In addition to the original endpoints, v2.0 adds:

| Endpoint | Method | Description | |----------|--------|-------------| | GET /__debugger | GET/POST | Original analytics API | | GET /__debugger/dashboard | GET | Live HTML dashboard | | GET /__debugger/health | GET | Service health status | | GET /__debugger/alerts | GET | Recent alerts | | GET /__debugger/alerts/rules | GET | Alert rules configuration | | GET /__debugger/metrics | GET | Prometheus-compatible metrics | | GET /__debugger/clusters | GET | Error clusters | | GET /__debugger/trends | GET | Trend analysis results | | GET /__debugger/dependency-graph | GET | Service dependency graph |

🛡 Production Rules

Debugger never blocks requests — all logic is async & non-blocking
Debugger never crashes your app — every operation is try/catch wrapped
Use sampling in high traffic — set samplingRate: 0.1 for 10% sampling
Errors always get traced — even when sampling drops a request
Enable features gradually — start with core, add observability features as needed
Monitor overhead — total overhead should be < 10ms per request

📊 Performance Impact

| Feature | Overhead | Recommendation | |---------|----------|----------------| | Core Tracing | ~2ms | Always enable | | Distributed Tracing | < 1ms | Enable in production | | Metrics | < 0.5ms | Enable in production | | Log Correlation | < 0.5ms | Enable in production | | Anomaly Detection | < 1ms | Enable in production | | Error Clustering | < 2ms | Enable in production | | Trend Analysis | < 1ms | Enable in production | | Alerting | < 5ms (async) | Enable with sampling | | Dependency Graph | < 1ms | Enable in production |

Total overhead: ~5-10ms per request with all features enabled.

📄 License

MIT