@bernierllc/backoff-retry

Portable async retry utility with exponential backoff, jitter, and error filtering for API and webhook management.

Features

• Exponential Backoff: Configurable backoff strategies with jitter
• Persistent State: Track retry attempts across application restarts
• Multiple Storage Backends: Memory, Redis with automatic fallback
• Memory Protection: Built-in memory usage monitoring and protection
• Flexible Error Handling: Custom retry conditions and error filtering
• Metrics & Monitoring: Built-in retry metrics and performance tracking
• TypeScript Support: Full TypeScript definitions included

Installation

npm install @bernierllc/backoff-retry

Quick Start

Basic Usage

import { createRetryManager } from '@bernierllc/backoff-retry';

const retryManager = createRetryManager();

// Simple retry with backoff
const result = await retryManager.retryWithBackoff(
  async () => {
    const response = await fetch('https://api.example.com/data');
    if (!response.ok) throw new Error('API request failed');
    return response.json();
  },
  {
    maxRetries: 5,
    initialDelayMs: 1000,
    maxDelayMs: 30000,
    backoffFactor: 2,
    jitter: true
  }
);

Persistent Retry State

// Retry with persistent state (survives app restarts)
const result = await retryManager.executeWithRetry(
  'api:user:123', // Unique ID for this operation
  async () => {
    return await updateUserProfile(userId, data);
  },
  {
    maxRetries: 3,
    initialDelayMs: 2000
  }
);

if (result.success) {
  console.log('Operation completed:', result.data);
} else {
  console.log('Operation failed after', result.attempts, 'attempts');
}

Configuration

Environment Variables

# Storage type: 'memory' or 'redis'
RETRY_STORAGE_TYPE=redis

# Redis connection URL (if using Redis storage)
REDIS_URL=redis://localhost:6379

# Enhancement configuration
NEVERADMIN_API_KEY=your_admin_api_key
NEVERHUB_API_KEY=your_hub_api_key
[email protected],[email protected]
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/YOUR/WEBHOOK
PAGERDUTY_KEY=your_pagerduty_key

Custom Configuration

import { createRetryManager } from '@bernierllc/backoff-retry';
import { MemoryStorageAdapter } from '@bernierllc/backoff-retry';

const retryManager = createRetryManager({
  storage: new MemoryStorageAdapter(1000), // Max 1000 retry states
  memoryProtection: {
    maxMemoryUsage: 50 * 1024 * 1024, // 50MB
    maxRetryStates: 500,
    cleanupInterval: 60000, // 1 minute
    evictionPolicy: 'lru'
  },
  defaultOptions: {
    maxRetries: 3,
    initialDelayMs: 1000,
    maxDelayMs: 10000,
    backoffFactor: 2,
    jitter: true
  },
  enhancements: {
    storage: {
      enabled: true,
      type: 'memory',
      autoUpgrade: true,
      thresholdNotifications: true,
      memoryThreshold: 80
    },
    notifications: {
      enabled: true,
      email: ['[email protected]'],
      thresholds: {
        memoryUsage: 80,
        retryFailures: 10,
        performanceDegradation: 20
      }
    }
  }
});

API Reference

RetryManager

retryWithBackoff<T>(fn, options): Promise<T>

Execute a function with retry logic.

const result = await retryManager.retryWithBackoff(
  async () => fetch('/api/data'),
  {
    maxRetries: 5,
    initialDelayMs: 1000,
    maxDelayMs: 30000,
    backoffFactor: 2,
    jitter: true,
    shouldRetry: (error) => error.status !== 404,
    onRetry: (attempt, delay, error) => {
      console.log(`Retry ${attempt} in ${delay}ms`);
    },
    onFailure: (error) => {
      console.error('Final failure:', error);
    }
  }
);

executeWithRetry<T>(id, fn, options): Promise<RetryResult<T>>

Execute a function with persistent retry state.

const result = await retryManager.executeWithRetry(
  'unique-operation-id',
  async () => processData(),
  { maxRetries: 3 }
);

if (result.success) {
  console.log('Success:', result.data);
  console.log('Attempts:', result.attempts);
  console.log('Total time:', result.totalTime);
} else {
  console.log('Failed:', result.error);
}

getRetryState(id): Promise<RetryState | null>

Get the current state of a retry operation.

listRetryStates(prefix?): Promise<RetryState[]>

List all retry states, optionally filtered by prefix.

cancelRetry(id): Promise<void>

Cancel a retry operation.

clearAll(): Promise<void>

Clear all retry states.

getMetrics(): RetryMetrics

Get retry performance metrics.

getMemoryStats(): MemoryStats

Get memory protection statistics.

Storage Adapters

Memory Storage

Default in-memory storage with LRU eviction:

import { MemoryStorageAdapter } from '@bernierllc/backoff-retry';

const storage = new MemoryStorageAdapter(1000); // Max 1000 items

Redis Storage

Redis storage with automatic fallback to memory:

import { RedisStorageAdapter } from '@bernierllc/backoff-retry';
import { createClient } from 'redis';

const redisClient = createClient({ url: 'redis://localhost:6379' });
const storage = new RedisStorageAdapter(redisClient, 'retry:');

Enhancement System

The package includes an opt-in enhancement system that provides advanced features with automatic upgrades and admin notifications:

Available Enhancements

• Storage Enhancements: Auto-upgrade from memory to Redis/PostgreSQL/MongoDB
• Memory Protection: Advanced analytics, leak detection, predictive forecasting
• NeverAdmin Integration: Dashboard widgets, real-time metrics, configuration management
• NeverHub Integration: Event streaming, webhook management, cross-service sync
• Machine Learning: Predictive retry optimization, error classification
• Performance: Multi-level caching, concurrency management, load balancing
• Security: Encryption, rate limiting, audit logging, access control
• Notifications: Email, Slack, PagerDuty, webhook notifications

Basic Enhancement Setup

const retryManager = createRetryManager({
  enhancements: {
    storage: {
      enabled: true,
      type: 'memory',
      autoUpgrade: true, // Automatically upgrade when Redis/DB available
      thresholdNotifications: true,
      memoryThreshold: 80 // Notify when 80% memory usage reached
    },
    notifications: {
      enabled: true,
      email: ['[email protected]'],
      thresholds: {
        memoryUsage: 80,
        retryFailures: 10,
        performanceDegradation: 20
      }
    }
  }
});

Environment-Based Auto-Configuration

The enhancement system automatically detects and enables features based on environment variables:

# Auto-enable Redis storage
REDIS_URL=redis://localhost:6379

# Auto-enable NeverAdmin integration
NEVERADMIN_API_KEY=your_key
NEVERADMIN_DASHBOARD_URL=https://admin.example.com

# Auto-enable NeverHub integration
NEVERHUB_API_KEY=your_key
NEVERHUB_URL=https://hub.example.com

# Auto-enable notifications
[email protected],[email protected]
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/YOUR/WEBHOOK
PAGERDUTY_KEY=your_pagerduty_key

Dynamic Enhancement Management

// Check if enhancement is available
if (retryManager.isEnhancementAvailable('neverAdmin')) {
  console.log('NeverAdmin integration is enabled');
}

// Enable enhancement dynamically
retryManager.enableEnhancement('ml', {
  predictiveRetry: true,
  errorClassification: true
});

// Get enhancement recommendations
const recommendations = retryManager.getEnhancementRecommendations();
console.log('Recommendations:', recommendations);

// Get enhancement metrics
const enhancementMetrics = retryManager.getEnhancementManager().getMetrics();
console.log('Performance score:', enhancementMetrics.performanceScore);

Memory Protection

The package includes built-in memory protection to prevent runaway memory usage:

const retryManager = createRetryManager({
  memoryProtection: {
    maxMemoryUsage: 100 * 1024 * 1024, // 100MB
    maxRetryStates: 1000,
    cleanupInterval: 30000, // 30 seconds
    evictionPolicy: 'lru' // 'lru', 'fifo', or 'random'
  }
});

Error Handling

Custom Retry Conditions

const shouldRetry = (error: any) => {
  // Retry on network errors, but not on validation errors
  if (error.code === 'ECONNRESET') return true;
  if (error.status === 400) return false;
  if (error.status >= 500) return true;
  return false;
};

await retryManager.retryWithBackoff(fn, {
  maxRetries: 5,
  shouldRetry
});

Retry Callbacks

await retryManager.retryWithBackoff(fn, {
  maxRetries: 3,
  onRetry: (attempt, delay, error) => {
    console.log(`Retry ${attempt} after ${delay}ms delay`);
    // Log to monitoring service
    analytics.track('retry_attempt', { attempt, delay, error: error.message });
  },
  onFailure: (error) => {
    console.error('Operation failed permanently:', error);
    // Alert operations team
    alerting.sendAlert('retry_failure', { error: error.message });
  }
});

Examples

API Request Retry

const apiClient = {
  async request(endpoint: string, options: RequestInit = {}) {
    return retryManager.retryWithBackoff(
      async () => {
        const response = await fetch(endpoint, {
          ...options,
          headers: {
            'Content-Type': 'application/json',
            ...options.headers
          }
        });
        
        if (!response.ok) {
          throw new Error(`HTTP ${response.status}: ${response.statusText}`);
        }
        
        return response.json();
      },
      {
        maxRetries: 3,
        initialDelayMs: 1000,
        shouldRetry: (error) => {
          // Retry on server errors and network issues
          return error.message.includes('500') || 
                 error.message.includes('502') ||
                 error.message.includes('503') ||
                 error.message.includes('504');
        }
      }
    );
  }
};

Webhook Delivery

const webhookManager = {
  async deliverWebhook(webhookId: string, payload: any) {
    return retryManager.executeWithRetry(
      `webhook:${webhookId}`,
      async () => {
        const response = await fetch(webhookUrl, {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify(payload)
        });
        
        if (!response.ok) {
          throw new Error(`Webhook delivery failed: ${response.status}`);
        }
        
        return response.json();
      },
      {
        maxRetries: 5,
        initialDelayMs: 5000, // Start with 5 second delay
        maxDelayMs: 300000,   // Max 5 minute delay
        backoffFactor: 2,
        jitter: true
      }
    );
  }
};

Monitoring and Metrics

// Get retry metrics
const metrics = retryManager.getMetrics();
console.log('Total retries:', metrics.totalRetries);
console.log('Success rate:', metrics.successfulRetries / metrics.totalRetries);
console.log('Average retry time:', metrics.averageRetryTime);

// Get memory protection stats
const memoryStats = retryManager.getMemoryStats();
console.log('Memory usage:', memoryStats.usagePercentage.toFixed(2) + '%');
console.log('Memory trend:', memoryStats.trend > 0 ? 'increasing' : 'decreasing');

License

ISC - See LICENSE file for details.

Contributing

This package is part of the Bernier LLC tools monorepo. Please refer to the main repository for contribution guidelines.