smart-compress-middleware

Intelligent Express middleware developed my Maheen Meshram for dynamic compression with pluggable AI/ML strategies. Automatically selects the best compression algorithm (gzip, Brotli, or none) based on payload size, content type, and request headers.

Features

• 🚀 Dynamic compression algorithm selection
• 🧠 Pluggable decision strategies (rule-based, AI, adaptive, hybrid)
• 📊 Real-time dashboard with WebSocket updates
• 🌊 Streaming compression (memory-efficient for large payloads)
• 📦 Buffered compression for small/medium responses
• ⚙️ Configurable size thresholds
• 📈 Performance feedback loop for learning
• 🛡️ Graceful error handling with automatic fallback
• 🔌 Zero dependencies (peer dependency: Express)
• 📝 TypeScript definitions included

Installation

npm install smart-compress-middleware

Quick Start

const express = require('express');
const smartCompress = require('smart-compress-middleware');

const app = express();

// Basic usage with defaults
app.use(smartCompress());

// With custom options
app.use(smartCompress({
  enabled: true,
  smallThreshold: 1024,      // Don't compress < 1KB
  largeThreshold: 5120,       // Use Brotli for > 5KB
  strategy: 'adaptive',       // 'rule-based', 'ai', 'adaptive', 'hybrid'
  logging: true,
  benchmark: true
}));

app.get('/api/data', (req, res) => {
  res.json({ message: 'This will be compressed automatically!' });
});

app.listen(3000);

Dashboard Setup

Add a real-time compression dashboard to monitor performance:

const express = require('express');
const http = require('http');
const smartCompress = require('smart-compress-middleware');

const app = express();
const server = http.createServer(app);

// Apply middleware
app.use(smartCompress());

// Setup dashboard (requires HTTP server for WebSocket)
smartCompress.setupDashboard(app, server, {
  path: '/dashboard'  // Access at http://localhost:3000/dashboard
});

server.listen(3000);

Configuration Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | enabled | boolean | true | Enable/disable compression | | smallThreshold | number | 1024 | Min size (bytes) for compression | | largeThreshold | number | 5120 | Size (bytes) to prefer Brotli | | strategy | string | 'rule-based' | Decision strategy to use | | aiModelEndpoint | string | null | External AI model endpoint | | logging | boolean | true | Enable console logging | | benchmark | boolean | true | Enable benchmark logging |

Decision Strategies

1. Rule-Based (Default)

Traditional size-based thresholds. Fast and predictable.

app.use(smartCompress({ strategy: 'rule-based' }));

• < 1KB → no compression
• 1KB-5KB → gzip
• > 5KB → Brotli (if supported)

2. AI-Based

Machine learning predictions with historical learning.

app.use(smartCompress({ 
  strategy: 'ai',
  aiModelEndpoint: 'http://your-ml-service/predict'  // Optional
}));

Features:

• Learns from compression performance
• Considers multiple factors (size, content type, path)
• Automatic fallback to rule-based on failure
• Mock implementation ready for real ML models

3. Adaptive

Learns from real-time performance metrics.

app.use(smartCompress({ strategy: 'adaptive' }));

Features:

• Tracks compression ratio and speed by payload size
• Adjusts decisions based on historical performance
• No external dependencies
• Self-optimizing over time

4. Hybrid

Combines multiple strategies with voting.

app.use(smartCompress({ strategy: 'hybrid' }));

Features:

• Weighted, unanimous, or majority voting
• Leverages strengths of different approaches
• Maximum reliability

Runtime Strategy Switching

Switch strategies without restarting:

const smartCompress = require('smart-compress-middleware');

// Switch strategy
smartCompress.engine.setStrategy('adaptive');

// Get current strategy
const current = smartCompress.engine.getCurrentStrategyName();

// List available strategies
const strategies = smartCompress.engine.listStrategies();

// Get statistics
const stats = smartCompress.engine.getStats();

Custom Strategies

Create your own decision strategy:

const { strategies } = require('smart-compress-middleware');

class CustomStrategy extends strategies.BaseStrategy {
  async decide(context) {
    const { size, acceptEncoding, contentType, path, isStreaming } = context;
    
    // Your custom logic here
    
    return {
      algorithm: 'gzip',  // 'gzip', 'brotli', or 'none'
      reason: 'Custom decision logic',
      confidence: 0.85    // Optional: 0-1 confidence score
    };
  }
  
  // Optional: Implement learning
  learn(feedback) {
    // Update internal state based on performance
  }
  
  getName() {
    return 'custom';
  }
}

// Register and use
const smartCompress = require('smart-compress-middleware');
smartCompress.engine.registerStrategy('custom', new CustomStrategy());
smartCompress.engine.setStrategy('custom');

API Routes

When dashboard is enabled, these routes are available:

# Get current strategy
GET /smart-compress-api/strategy/current

# Get strategy statistics
GET /smart-compress-api/strategy/stats

# Switch strategy
POST /smart-compress-api/strategy/switch
Body: { "strategy": "adaptive" }

# List all strategies
GET /smart-compress-api/strategy/list

# Dashboard stats
GET /smart-compress-api/dashboard/stats

TypeScript Support

Full TypeScript definitions included:

import smartCompress, { 
  CompressionOptions, 
  DecisionEngine,
  strategies 
} from 'smart-compress-middleware';

const options: CompressionOptions = {
  enabled: true,
  strategy: 'adaptive',
  smallThreshold: 1024
};

app.use(smartCompress(options));

Performance

• Non-blocking streaming compression
• Minimal memory overhead (no full payload buffering)
• Efficient stream handling with backpressure support
• Automatic detection of buffered vs streaming responses

Compression Logic

The middleware automatically:

• Respects Accept-Encoding headers
• Sets Content-Encoding headers
• Adds X-Compression-Algorithm header
• Handles errors gracefully (falls back to uncompressed)
• Never crashes your server

Dashboard Features

The live dashboard shows:

1. Real-time Statistics

   • Total requests processed
   • Average compression ratio
   • Average compression time
   • Total bytes saved

2. Visual Charts

   • Compression ratio by algorithm
   • Request count per algorithm

3. Live Activity Log

   • Recent compression events
   • Color-coded by algorithm
   • Detailed metrics per request

4. Interactive Controls

   • Clear logs
   • Test endpoints
   • Auto-reconnecting WebSocket

Examples

See the /example directory for a complete working example:

git clone https://github.com/yourusername/smart-compress-middleware.git
cd smart-compress-middleware
npm install
npm start

Visit http://localhost:3000/dashboard to see the dashboard in action.

Testing

npm test

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT © 2026

Links

• GitHub Repository
• npm Package
• Issue Tracker

Changelog

1.0.1

• Initial release
• Rule-based, AI, adaptive, and hybrid strategies
• Real-time dashboard
• Streaming compression support
• TypeScript definitions