Observability Framework

A comprehensive, configurable observability framework built on OpenTelemetry with semantic wide events, automatic middleware, intelligent error categorization, and utility functions for common patterns.

Installation

npm install @delta-base/observability

Quick Start

import { observabilityMiddleware } from '@delta-base/observability';
import { BUILD_INFO } from './build-info'; // Generated by collect-build-info script

const app = new OpenAPIHono();

// Add observability middleware - automatically handles tracing, HTTP tracking, 
// intelligent error detection, and build info initialization
app.use('*', observabilityMiddleware({
  buildInfo: BUILD_INFO // Automatically initializes build info
}));

// Your routes now have automatic observability with smart error categorization
app.post('/users', async (c) => {
  const tracker = c.get('wideEvent');
  
  // Set operation and track user data being created
  tracker
    .setOperation('user.create', { domain: 'user-management', layer: 'api' })
    .trackUser({
      email: '[email protected]',
      actorType: 'public'
    });
  
  // Your business logic here
  const result = await createUser(userData);
  
  // Track the created entity
  tracker.trackEntity({
    type: 'user',
    id: result.id
  });
  
  return c.json(result, 201);
});

💡 Tip: Add "prebuild": "collect-build-info --package-json-dir . --output-dir src" to your package.json scripts to automatically generate build info before each build.

Configuration

The framework now supports comprehensive configuration to customize behavior for different domains and use cases.

Simple Configuration

import { configureObservability, observabilityMiddleware } from '@delta-base/observability';
import { BUILD_INFO } from './build-info';

// Option 1: Configure globally
configureObservability({
  serviceName: 'my-service',
  autoTrackHttp: true,
  autoTrackErrorResponses: true,
  debug: false
});

// Option 2: Configure via middleware (recommended for build info)
app.use('*', observabilityMiddleware({
  serviceName: 'my-service',
  autoTrackHttp: true,
  autoTrackErrorResponses: true,
  debug: false,
  buildInfo: BUILD_INFO  // Automatically initializes build info
}));

Advanced Configuration

import { configureObservabilityAdvanced } from '@delta-base/observability';

// Configure with comprehensive options
configureObservabilityAdvanced({
  tracing: {
    enabled: true,
    serviceName: 'my-service',
    serviceVersion: '1.0.0',
    environment: 'production'
  },
  http: {
    trackRequests: true,
    trackResponses: true,
    trackBodies: false,
    excludeHeaders: ['authorization', 'cookie']
  },
  errors: {
    autoTrackErrorResponses: true,
    autoCategorizeErrors: true,
    defaultUnexpected: true
  },
  performance: {
    trackTiming: true,
    categorizePerformance: true,
    performanceThresholds: {
      fast: 100,
      slow: 1000
    }
  },
  contextSchema: {
    user: ['id', 'email', 'role', 'organization_id'],
    operation: ['type', 'domain', 'layer', 'tenant_id'],
    // ... more categories
  }
});

Build Information Integration

The framework includes comprehensive build information tracking with multiple initialization methods:

Method 1: Automatic via Middleware (Recommended)

The simplest approach is to pass build info directly to the middleware:

import { observabilityMiddleware } from '@delta-base/observability';
import { BUILD_INFO } from './build-info'; // Generated by collect-build-info script

const app = new OpenAPIHono();

// Automatically initializes build info when middleware is set up
app.use('*', observabilityMiddleware({ 
  buildInfo: BUILD_INFO 
}));

// Your routes now have build info automatically available
app.post('/users', async (c) => {
  const tracker = c.get('wideEvent');
  // Build info is already initialized - no manual setup needed
});

Method 2: Manual Initialization

If you need more control or want to initialize build info elsewhere:

import { initializeBuildInfo } from '@delta-base/observability';

// Initialize build info manually in your application
initializeBuildInfo({
  'service.name': 'my-service',
  'service.version': '1.0.0',
  'service.environment': 'production',
  'git.commit_hash': 'abc123',
  'git.branch': 'main',
  'build.timestamp': new Date().toISOString()
});

// Then use middleware without buildInfo parameter
app.use('*', observabilityMiddleware());

Method 3: Build Info Collection Script

Generate build info automatically using the CLI tool:

# Add to your package.json scripts
"prebuild": "collect-build-info --package-json-dir . --output-dir src"

This creates a src/build-info.ts file with comprehensive build information:

// Auto-generated build info - do not edit manually
export const BUILD_INFO = {
  "service.name": "@my-org/my-service",
  "service.version": "1.0.0",
  "service.environment": "production",
  "git.commit_hash": "abc123def456",
  "git.branch": "main",
  "git.commit_message": "feat: add user management",
  "git.commit_author": "developer",
  "build.timestamp": "2024-01-01T12:00:00.000Z",
  "deployment.user": "ci-cd",
  "deployment.trigger": "push"
} as const;

Then use it in your middleware:

import { BUILD_INFO } from './build-info';

app.use('*', observabilityMiddleware({ buildInfo: BUILD_INFO }));

Build Info Benefits

With build info properly initialized, your observability data includes:

• Service identification: service.name, service.version, service.environment
• Git context: git.commit_hash, git.branch, git.commit_author
• Build context: build.timestamp, build.id, deployment.user
• Deployment context: deployment.trigger, deployment.ci_build_url

This appears in all traces and logs:

{
  "service.name": "@my-org/my-service",
  "service.version": "1.0.0",
  "git.commit_hash_short": "abc123d",
  "git.branch": "main",
  "operation": "user.create",
  "request_id": "uuid-here",
  // ... other tracking data
}

Context Schema Customization

The framework uses a configurable context schema to determine which fields are tracked and extracted from spans. You can customize this for your domain:

import { addContextFields } from '@delta-base/observability';

// Add custom user fields
addContextFields('user', ['role', 'organization_id', 'permissions']);

// Add custom operation fields  
addContextFields('operation', ['tenant_id', 'workspace_id']);

// Add custom infrastructure fields
addContextFields('infrastructure', ['queue.name', 'cache.hit_rate']);

Custom Error Patterns

Extend the automatic error detection with domain-specific patterns:

import { addErrorPattern } from '@delta-base/observability';

// Add custom error pattern for your API
addErrorPattern({
  name: 'CustomBusinessError',
  matcher: (body) => 
    body?.error?.type === 'BUSINESS_RULE_VIOLATION' && 
    body?.error?.code,
  tracker: (wideEvent, body) => {
    wideEvent.trackError({
      type: 'BusinessRuleViolation',
      message: body.error.message,
      businessRule: {
        rule: body.error.code,
        violation: body.error.details
      },
      unexpected: false // Expected - business rule violation
    });
  }
});

Core Concepts

Wide Events

Wide events are context-rich, structured logs that capture comprehensive information about operations. Instead of multiple log lines, you emit a single event per service hop with all relevant context.

Intelligent Error Categorization

The framework automatically categorizes errors into expected and unexpected types:

Expected Errors (unexpected: false)

• Validation errors - User submitted invalid data
• Business rule violations - Duplicate email, insufficient permissions
• Authentication/authorization failures - Wrong password, missing permissions
• Rate limiting - User exceeded API limits
• Resource not found - User requested non-existent data

Unexpected Errors (unexpected: true)

• System failures - Database down, network issues
• Programming errors - Null pointer exceptions, type errors
• Infrastructure issues - Service unavailable, timeouts
• Unhandled edge cases - Code paths that shouldn't be reached

This categorization enables:

• Smart alerting - Only alert on unexpected errors
• SLA tracking - Only count unexpected errors against service level agreements
• User experience monitoring - Track expected errors separately for UX insights
• Debugging prioritization - Unexpected errors need immediate attention

Semantic Tracking Methods

The framework provides semantic methods for tracking different types of data:

• setOperation() - Set operation type and context (domain, layer, etc.)
• trackUser() - User and actor information
• trackEntity() - Primary resource being operated on
• trackInfrastructure() - Technical infrastructure details
• trackError() - Error information with automatic categorization
• recordEvent() - Business events during the operation

Configuration API Reference

ObservabilityConfig

The comprehensive configuration interface for the observability framework.

interface ObservabilityConfig {
  contextSchema: ContextSchemaConfig;   // Defines which fields are tracked
  tracing: TracingConfig;               // OpenTelemetry tracing configuration  
  http: HttpTrackingConfig;             // HTTP tracking configuration
  errors: ErrorTrackingConfig;          // Error detection and categorization
  performance: PerformanceConfig;       // Performance tracking and thresholds
  debug: boolean;                       // Debug logging
}

ContextSchemaConfig

Defines which fields are tracked for each category of context data.

interface ContextSchemaConfig {
  user: string[];           // User-related fields (id, email, role, etc.)
  entity: string[];         // Entity-related fields (type, id, status, etc.)
  operation: string[];      // Operation-related fields (type, domain, layer, etc.)
  infrastructure: string[]; // Infrastructure fields (database, event_store, etc.)
  error: string[];          // Error-related fields (type, message, unexpected, etc.)
  http: string[];           // HTTP-related fields (method, path, status_code, etc.)
  performance: string[];    // Performance fields (duration_ms, category, etc.)
  service: string[];        // Service-related fields (name, version, environment, etc.)
}

ErrorResponsePattern

Defines a pattern for detecting and categorizing error responses.

interface ErrorResponsePattern {
  name: string;                              // Human-readable pattern name
  matcher: (body: any) => boolean;           // Test if response matches pattern
  tracker: (wideEvent: WideEventsWrapper, body: any) => void; // Track the error
}

Configuration Functions

observabilityMiddleware(options?: ObservabilityOptions)

Create observability middleware with configuration options.

interface ObservabilityOptions {
  /** Custom service name (defaults to environment variable SERVICE_NAME) */
  serviceName?: string;
  
  /** Whether to automatically track HTTP request/response (defaults to true) */
  autoTrackHttp?: boolean;
  
  /** Whether to automatically detect and track error responses (defaults to true) */
  autoTrackErrorResponses?: boolean;
  
  /** Whether to enable debug logging (defaults to false) */
  debug?: boolean;
  
  /** Build info to initialize (optional) */
  buildInfo?: Record<string, string | number | boolean>;
}

// Simple usage
app.use('*', observabilityMiddleware());

// With configuration
app.use('*', observabilityMiddleware({
  serviceName: 'my-service',
  autoTrackHttp: true,
  autoTrackErrorResponses: true,
  debug: false,
  buildInfo: BUILD_INFO
}));

configureObservability(options: ObservabilityOptions)

Simple configuration for basic use cases.

configureObservability({
  serviceName: 'my-service',
  autoTrackHttp: true,
  autoTrackErrorResponses: true,
  debug: false
});

configureObservabilityAdvanced(config: Partial)

Advanced configuration with full control over all options.

configureObservabilityAdvanced({
  tracing: {
    serviceName: 'my-service',
    environment: 'production'
  },
  contextSchema: {
    user: ['id', 'email', 'role'],
    operation: ['type', 'domain', 'tenant_id']
  }
});

addContextFields(category: keyof ContextSchemaConfig, fields: string[])

Add custom fields to a context category.

addContextFields('user', ['organization_id', 'permissions']);
addContextFields('operation', ['workspace_id', 'feature_flag']);

addErrorPattern(pattern: ErrorResponsePattern)

Add a custom error detection pattern.

addErrorPattern({
  name: 'RateLimitError',
  matcher: (body) => body?.error?.code === 'RATE_LIMIT_EXCEEDED',
  tracker: (wideEvent, body) => {
    wideEvent.trackError({
      type: 'RateLimitError',
      message: body.error.message,
      unexpected: false // Expected - rate limiting
    });
  }
});

API Reference

WideEventsWrapper

The main class for tracking operation context.

import { WideEventsWrapper } from '@delta-base/observability';

const tracker = new WideEventsWrapper('Create User', 'user.create');

await tracker.execute(async (t) => {
  // Set operation and track different types of context
  t.setOperation('user.create', { domain: 'user-management', layer: 'api' })
   .trackUser({ id: '123', email: '[email protected]' })
   .trackEntity({ type: 'user', id: '123' });
   
  // Your business logic here
});

trackUser(userData: UserData)

Track user and actor information.

tracker.trackUser({
  id: '123',
  email: '[email protected]',
  firstName: 'John',
  lastName: 'Doe',
  actorType: 'authenticated_user'
});

trackEntity(entityData: EntityData)

Track the primary entity/resource being operated on.

tracker.trackEntity({
  type: 'user',
  id: 'user-123',
  status: 'active'
});

setOperation(operationType: string, context?: OperationData)

Set the operation type and context for this operation.

// Simple operation
tracker.setOperation('user.create');

// With context
tracker.setOperation('user.create', {
  domain: 'user-management',
  layer: 'api'
});

// With custom properties
tracker.setOperation('user.change-password', {
  domain: 'user-management',
  layer: 'business-logic',
  trigger: 'user-initiated'
});

trackInfrastructure(infrastructureData: InfrastructureData)

Track infrastructure and technical details.

tracker.trackInfrastructure({
  database: {
    operation: 'INSERT',
    table: 'users',
    duration: 45
  },
  eventStore: {
    streamId: 'user-123',
    expectedVersion: 0,
    nextVersion: 1
  },
  externalService: {
    name: 'notification-service',
    operation: 'send-welcome-email',
    duration: 120
  }
});

trackError(errorData: ErrorData)

Track error information with automatic categorization.

// Expected error - validation failure
tracker.trackError({
  type: 'ValidationError',
  message: 'Email is required',
  validation: {
    issueCount: 2,
    fields: ['email', 'firstName']
  },
  unexpected: false // Expected - user submitted invalid data
});

// Expected error - business rule violation
tracker.trackError({
  type: 'BusinessRuleViolation',
  message: 'User already exists',
  businessRule: {
    rule: 'unique_email',
    violation: 'already_exists'
  },
  unexpected: false // Expected - business constraint violation
});

// Unexpected error - system failure
tracker.trackError({
  type: 'DatabaseConnectionError',
  message: 'Connection pool exhausted',
  unexpected: true // Unexpected - infrastructure issue
});

recordEvent(eventName: string, eventData?: object)

Record business events during the operation.

tracker.recordEvent('user.created', {
  userId: '123',
  source: 'api'
});

tracker.recordEvent('validation.completed', {
  validation_type: 'input'
});

Middleware

observabilityMiddleware(options?)

Automatic observability middleware that:

• Initializes OpenTelemetry tracing
• Creates a WideEventsWrapper for the request
• Automatically tracks HTTP request/response details
• Intelligently detects and categorizes error responses using pattern matching
• Makes the tracker available via c.get('wideEvent')
• Provides simple fallback operation type derivation

import { observabilityMiddleware } from '@delta-base/observability';

// Basic usage
app.use('*', observabilityMiddleware());

// With custom options
app.use('*', observabilityMiddleware({
  serviceName: 'my-custom-service',
  autoTrackHttp: true,
  autoTrackErrorResponses: true
}));

The middleware includes intelligent error response detection using pattern matching:

// Automatic detection of OpenAPIHono validation errors
{
  name: 'OpenAPIHono_ZodValidation',
  matcher: (body) => 
    body?.success === false &&
    body?.error?.name === 'ZodError' &&
    Array.isArray(body?.error?.issues),
  tracker: (wideEvent, body) => {
    wideEvent.trackError({
      type: 'ValidationError',
      message: 'Request validation failed',
      validation: {
        issueCount: body.error.issues.length,
        fields: body.error.issues.map(issue => issue.path.join('.'))
      },
      unexpected: false // Expected - user submitted invalid data
    });
  }
}

createObservabilityErrorHandler(options?)

Error handler that integrates with observability tracking for exceptions that bypass normal response flow.

import { observabilityMiddleware, createObservabilityErrorHandler } from '@delta-base/observability';

const app = new OpenAPIHono();
app.use('*', observabilityMiddleware());
app.onError(createObservabilityErrorHandler());

extractUserContext(request: Request)

Extract user context from HTTP request.

const userContext = extractUserContext(c.req.raw);
if (userContext) {
  tracker.trackUser(userContext);
}

Utility Functions

withDomainTracking

Track domain operations with automatic span management.

import { withDomainTracking } from '@delta-base/observability';

const result = await withDomainTracking(tracker, 'user', 'create', async () => {
  // Domain logic here
  return await createUser(data);
});

withEventStoreTracking

Track event store operations with automatic span management.

import { withEventStoreTracking } from '@delta-base/observability';

const result = await withEventStoreTracking(tracker, 'user-123', 'append', async () => {
  return await eventStore.append(streamId, events);
});

withValidationTracking

Track validation operations with automatic error handling and categorization.

import { withValidationTracking } from '@delta-base/observability';

const validatedData = await withValidationTracking(tracker, 'input', async () => {
  return await validateUserInput(data);
});

withExternalServiceTracking

Track external service calls with automatic timing.

import { withExternalServiceTracking } from '@delta-base/observability';

const result = await withExternalServiceTracking(tracker, 'user-service', 'create', async () => {
  return await userService.createUser(data);
});

withDatabaseTracking

Track database operations with automatic timing.

import { withDatabaseTracking } from '@delta-base/observability';

const result = await withDatabaseTracking(tracker, 'INSERT', 'users', async () => {
  return await db.insert(userData);
});

trackUserFromSources

Automatically extract and track user context from various sources.

import { trackUserFromSources } from '@delta-base/observability';

trackUserFromSources(tracker, {
  request: c.req.raw, // HTTP request
  body: requestBody,  // Request body
  headers: customHeaders, // Custom headers
});

Build Information System

The framework includes a comprehensive build information system that automatically collects and tracks build metadata:

Automatic Build Info Collection

# Use the CLI tool to collect build info
npx collect-build-info --package-json-dir . --output-dir src --service-name my-service

# Or programmatically
import { collectBuildInfo } from '@delta-base/observability';

const buildInfo = collectBuildInfo({
  packageJsonDir: process.cwd(),
  outputDir: 'src',
  defaultServiceName: 'my-service'
});

Build Info in Package.json

Add the build info collection to your package.json:

{
  "scripts": {
    "prebuild": "collect-build-info --package-json-dir . --output-dir src --service-name my-service",
    "build": "npm run prebuild && your-build-command"
  }
}

Build Info Integration

The framework automatically includes build information in all wide events:

import { initializeBuildInfo } from '@delta-base/observability';

// Initialize with your build info
initializeBuildInfo({
  'service.name': 'my-service',
  'service.version': '1.0.0',
  'git.commit_hash': 'abc123',
  'git.branch': 'main',
  'build.timestamp': new Date().toISOString()
});

Advanced Usage

Error Categorization in Practice

The framework automatically categorizes errors, but you can also manually track errors with explicit categorization:

const tracker = c.get('wideEvent');

try {
  // Your operation
  await performOperation();
} catch (error) {
  if (error instanceof ValidationError) {
    // Expected error - user input issue
    tracker.trackError({
      type: 'ValidationError',
      message: error.message,
      validation: {
        issueCount: 1,
        fields: ['email']
      },
      unexpected: false // Expected - user submitted invalid data
    });
  } else if (error instanceof DatabaseConnectionError) {
    // Unexpected error - system issue
    tracker.trackError({
      type: 'DatabaseConnectionError',
      message: error.message,
      unexpected: true // Unexpected - infrastructure issue
    });
  }
  throw error;
}

Alerting Based on Error Categories

// Example alerting logic
if (error.unexpected === true) {
  await sendAlert({
    severity: 'high',
    message: `Unexpected error in ${operation}`,
    error: error
  });
} else {
  // Expected errors might be tracked for user experience metrics
  await trackUserExperienceMetric({
    type: 'validation_error',
    operation: operation,
    error: error
  });
}

SLA and Error Budget Tracking

// Example SLA tracking
const errorBudget = {
  recordRequest: (success: boolean, unexpected: boolean) => {
    if (!success && unexpected) {
      // Only count unexpected errors against SLA
      errorBudget.recordFailure();
    } else {
      errorBudget.recordSuccess();
    }
  }
};

// In your monitoring system
if (event.success === false && event.error?.unexpected === true) {
  slaTracker.recordViolation();
}

Custom Error Response Patterns

You can extend the error response pattern matching by adding custom patterns:

// Example of adding custom error patterns
const customErrorPatterns = [
  {
    name: 'CustomBusinessError',
    matcher: (body) => 
      body?.error?.type === 'BUSINESS_RULE_VIOLATION' &&
      body?.error?.code,
    tracker: (wideEvent, body) => {
      wideEvent.trackError({
        type: 'BusinessRuleViolation',
        message: body.error.message,
        businessRule: {
          rule: body.error.code,
          violation: body.error.details
        },
        unexpected: false // Expected - business rule violation
      });
    }
  }
];

Creating Child Spans

const validationSpan = tracker.createOperationSpan('validation', {
  'operation.layer': 'validation'
});

try {
  // Validation logic
  validationSpan.setStatus({ code: SpanStatusCode.OK });
} catch (error) {
  validationSpan.recordException(error);
} finally {
  validationSpan.end();
}

Comprehensive Route Example

app.post('/users', async (c) => {
  const tracker = c.get('wideEvent'); // From middleware
  const body = c.req.valid('json');
  
  try {
    // Set operation and track user data being created
    tracker
      .setOperation('user.create', {
        domain: 'user-management',
        layer: 'api'
      })
      .trackUser({
        email: body.email,
        firstName: body.firstName,
        lastName: body.lastName,
        actorType: 'public'
      });

    // Use utility for domain tracking
    const result = await withDomainTracking(tracker, 'user', 'create', async () => {
      return await createUserHandler(eventStore, actor, body, tracker);
    });

    // Track created entity and infrastructure
    tracker
      .trackEntity({
        type: 'user',
        id: result.id,
        status: 'created'
      })
      .trackInfrastructure({
        eventStore: {
          streamId: result.id,
          expectedVersion: 0,
          nextVersion: result.nextExpectedStreamVersion,
          createdNewStream: true
        }
      });

    return c.json({
      success: true,
      message: 'User created successfully',
      user: result.user,
      nextExpectedStreamVersion: result.nextExpectedStreamVersion
    }, 201);

  } catch (error) {
    // Error tracking is automatic via middleware pattern matching
    // But you can add specific context if needed
    if (error instanceof IllegalStateError && error.message.includes('already exists')) {
      tracker.trackError({
        type: 'BusinessRuleViolation',
        message: error.message,
        businessRule: {
          rule: 'unique_email',
          violation: 'already_exists'
        },
        unexpected: false // Expected - business constraint violation
      });
    }
    throw error;
  }
});

Output Format

The framework generates structured wide events with the following format:

{
  "request_id": "8bfdf7ecdd485694",
  "timestamp": "2024-09-08T06:14:05.680Z",
  "operation": "user.create",
  "success": true,
  
  "http.method": "POST",
  "http.path": "/users",
  "http.status_code": 201,
  
  "user.email": "[email protected]",
  "user.first_name": "John",
  "user.last_name": "Doe",
  "actor.type": "public",
  
  "entity.type": "user",
  "entity.id": "user-123",
  
  "operation.domain": "user-management",
  "operation.layer": "api",
  
  "event_store.stream_id": "user-123",
  "event_store.expected_version": 0,
  "event_store.next_version": 1,
  "event_store.created_new_stream": true,
  
  "timing.domain_ms": 45,
  "timing.event_store_ms": 67,
  "timing.total_ms": 124,
  
  "performance_category": "fast",
  "error_category": null,
  
  "spans": [
    {
      "name": "validation",
      "duration_ms": 12,
      "status": "ok"
    },
    {
      "name": "domain",
      "duration_ms": 45,
      "status": "ok"
    },
    {
      "name": "event_store",
      "duration_ms": 67,
      "status": "ok"
    }
  ],
  
  "service.name": "my-service",
  "service.version": "1.0.0",
  "git.commit_hash": "abc123",
  "git.branch": "main",
  "build.timestamp": "2024-09-08T06:14:05.680Z"
}

Error Output Format

When errors occur, the framework adds comprehensive error context:

{
  "request_id": "8bfdf7ecdd485694",
  "timestamp": "2024-09-08T06:14:05.680Z",
  "operation": "user.create",
  "success": false,
  
  "http.method": "POST",
  "http.path": "/users",
  "http.status_code": 400,
  
  "error.type": "ValidationError",
  "error.message": "Request validation failed",
  "error.unexpected": false,
  "error.validation.issue_count": 2,
  "error.validation.fields": ["email", "firstName"],
  
  "user.email": "invalid-email",
  "user.first_name": "John",
  "actor.type": "public",
  
  "operation.domain": "user-management",
  "operation.layer": "api",
  
  "timing.total_ms": 12,
  "performance_category": "fast",
  "error_category": "expected",
  
  "service.name": "my-service",
  "service.version": "1.0.0",
  "git.commit_hash": "abc123",
  "git.branch": "main",
  "build.timestamp": "2024-09-08T06:14:05.680Z"
}

Environment Variables

• SERVICE_NAME - Name of your service (used in spans and events)
• SERVICE_VERSION - Version of your service
• ENVIRONMENT_NAME - Environment (development, staging, production)

Error Response Pattern Matching

The framework uses an extensible pattern matching system to intelligently detect and categorize different types of error responses:

interface ErrorResponsePattern {
  name: string;                          // Human-readable name
  matcher: (body: any) => boolean;       // Pattern matching function
  tracker: (wideEvent: WideEventsWrapper, body: any) => void; // Error tracking function
}

Built-in patterns include:

• OpenAPIHono ZodValidation: Detects Zod validation errors from OpenAPIHono
• Business Validation: Detects business rule violations
• Generic HTTP Errors: Fallback for unmatched error responses

Best Practices

Framework Usage

1. Use Middleware: Always use observabilityMiddleware() for automatic setup and intelligent error detection
2. Configure Early: Configure observability at application startup before registering middleware
3. Leverage Automatic Categorization: Trust the framework's error categorization - it properly distinguishes expected vs unexpected errors
4. Semantic Methods: Use the semantic tracking methods (trackUser, trackEntity, etc.) instead of generic setters
5. Utility Functions: Use utility functions like withDomainTracking for common patterns

Configuration

6. Context Schema: Customize the context schema for your domain - add fields that are important for your use case
7. Error Patterns: Add custom error patterns for domain-specific error types
8. Environment Specific: Use different configurations for different environments (development, staging, production)
9. Performance Thresholds: Adjust performance thresholds based on your service's characteristics

Error Handling

10. Error Context: The middleware automatically adds error context, but you can add specific business context when needed
11. Business Events: Record significant business events with recordEvent()
12. Infrastructure Details: Track infrastructure operations for performance analysis
13. Alerting Strategy: Only alert on unexpected errors (unexpected: true)
14. SLA Tracking: Only count unexpected errors against service level agreements
15. User Experience: Track expected errors separately for UX insights

Development Workflow

16. Debug Mode: Enable debug mode in development to see detailed logging
17. Test Error Patterns: Test your custom error patterns with different error scenarios
18. Validate Context: Regularly review the context fields being tracked to ensure they're useful
19. Monitor Performance: Use the performance categorization to identify slow operations

Extending the Framework

Creating Custom Tracking Utilities

You can create domain-specific utilities that wrap the framework's functionality:

// Custom utility for tracking database operations
export async function withDatabaseQuery<T>(
  tracker: WideEventsWrapper | undefined,
  query: string,
  table: string,
  fn: () => Promise<T>
): Promise<T> {
  if (!tracker) return await fn();
  
  return await withDatabaseTracking(tracker, 'SELECT', table, async () => {
    tracker.recordEvent('database.query.started', { query, table });
    const result = await fn();
    tracker.recordEvent('database.query.completed', { query, table });
    return result;
  });
}

// Custom utility for tracking external API calls
export async function withExternalAPI<T>(
  tracker: WideEventsWrapper | undefined,
  apiName: string,
  endpoint: string,
  fn: () => Promise<T>
): Promise<T> {
  if (!tracker) return await fn();
  
  return await withExternalServiceTracking(tracker, apiName, endpoint, async () => {
    tracker.recordEvent('external_api.call.started', { api: apiName, endpoint });
    const result = await fn();
    tracker.recordEvent('external_api.call.completed', { api: apiName, endpoint });
    return result;
  });
}

Domain-Specific Configuration Presets

Create configuration presets for different types of services:

// User management service configuration
export const userServiceConfig = {
  contextSchema: {
    user: ['id', 'email', 'role', 'organization_id'],
    operation: ['type', 'domain', 'layer', 'user_id'],
    infrastructure: ['database.operation', 'cache.hit_rate'],
    error: ['type', 'message', 'unexpected', 'validation_error']
  },
  performance: {
    performanceThresholds: {
      fast: 100,
      slow: 1000
    }
  }
};

// API gateway service configuration  
export const apiGatewayServiceConfig = {
  contextSchema: {
    user: ['id', 'email', 'role', 'organization_id'],
    operation: ['type', 'domain', 'layer', 'route', 'method'],
    infrastructure: ['upstream.service', 'upstream.duration_ms'],
    error: ['type', 'message', 'unexpected', 'auth_failure', 'rate_limit']
  },
  performance: {
    performanceThresholds: {
      fast: 100,
      slow: 2000  // API calls can be slower
    }
  }
};

// Apply preset configuration
configureObservabilityAdvanced(userServiceConfig);

Troubleshooting

Common Issues

Context Fields Not Appearing

// Problem: Custom context fields not showing in wide events
// Solution: Ensure fields are added to the context schema
addContextFields('user', ['custom_field']);

// Or check if the field name matches what you're tracking
tracker.trackUser({ customField: 'value' }); // This becomes 'user.custom_field'

Error Patterns Not Matching

// Problem: Custom error patterns not being detected
// Solution: Add debug logging to test your matcher
addErrorPattern({
  name: 'CustomError',
  matcher: (body) => {
    console.log('Testing error pattern:', body); // Debug logging
    return body?.error?.type === 'CUSTOM_ERROR';
  },
  tracker: (wideEvent, body) => {
    // ... tracker logic
  }
});

Performance Issues

// Problem: Observability causing performance degradation
// Solution: Disable expensive features in production
configureObservabilityAdvanced({
  http: {
    trackBodies: false,        // Disable body tracking
    maxBodySize: 1024         // Limit body size
  },
  performance: {
    autoCreateSpans: false     // Disable automatic spans
  }
});

Debug Mode

Enable debug mode to see detailed information about what the framework is doing:

configureObservability({
  debug: true
});

// You'll see console output like:
// [Observability] Detected OpenAPIHono_ZodValidation error response, tracking...
// [Observability] Extracted business context: user.id, user.email, operation.type
// [Observability] Wide event emitted: {...}

Summary of Features

This enhanced observability framework provides:

✅ Configurable Context Schema

• No more hard-coded context fields
• Customize what fields are tracked for your domain
• Add fields incrementally with addContextFields()

✅ Pluggable Error Patterns

• Extend automatic error detection
• Domain-specific error categorization
• Proper expected vs unexpected error handling

✅ Comprehensive Configuration

• Simple and advanced configuration options
• Environment-specific settings
• Performance tuning options

✅ Build Information Integration

• Automatic build info collection from git and package.json
• CLI tool for build pipeline integration
• Comprehensive metadata tracking

✅ Developer Experience

• Clear TypeScript interfaces
• Extensive documentation and examples
• Debug mode for troubleshooting

✅ Production Ready

• Comprehensive error handling
• Performance optimizations
• Environment-specific configurations

This framework provides a comprehensive, configurable observability solution that can be adapted to any domain while maintaining high-quality error categorization and debugging capabilities.