@cook-step/cache-system

High-performance multi-level cache system with automatic immutability protection, LRU eviction, and memory management. Designed for DAG execution engines and high-performance applications.

Features

• 🚀 Multi-level caching - L1/L2/L3 cache with automatic promotion/demotion
• 🔒 Immutability protection - Automatic protection using Immer (with binary data support)
• 💾 Multiple eviction policies - LRU, LFU, and FIFO strategies
• 📊 Comprehensive statistics - Hit/miss rates, memory usage, performance metrics
• 🔄 Pattern-based invalidation - Invalidate by key patterns or tags
• ⚡ Hash-based keys - Support for complex objects as cache keys
• 🎯 TTL support - Time-to-live for cache entries
• 🧮 Smart memory management - Size-aware caching with automatic eviction
• ✅ Well tested - 93.7% code coverage with 61+ tests

Installation

npm install @cook-step/cache-system
# or
pnpm add @cook-step/cache-system
# or
yarn add @cook-step/cache-system

Quick Start

import { CacheSystem } from "@cook-step/cache-system";

// Create cache with configuration
const cache = new CacheSystem({
  maxMemory: 100 * 1024 * 1024, // 100MB
  maxEntries: 10000,
  enableImmutability: true,
  defaultTTL: 60000, // 1 minute
});

// Store data (automatically protected)
const data = { items: [1, 2, 3] };
cache.set("my-key", data);

// Retrieve data (immutable)
const cached = cache.get("my-key");
// cached.items.push(4); // TypeError: Cannot add property

// Cache with TTL
cache.set("temp-data", value, { ttl: 5000 }); // 5 seconds

// Pattern invalidation
cache.invalidate("user:*"); // Invalidate all user keys

// Get statistics
const stats = cache.getStats();
console.log(`Hit rate: ${stats.hitRate}%`);

API Reference

CacheSystem

class CacheSystem<K = string, V = any> {
  constructor(options?: CacheOptions);

  // Basic operations
  get(key: K): V | undefined;
  set(key: K, value: V, options?: SetOptions): void;
  has(key: K): boolean;
  delete(key: K): boolean;
  clear(): void;

  // Pattern operations
  invalidate(pattern: string): number;
  keys(pattern?: string): K[];

  // Statistics
  getStats(): CacheStats;
  resetStats(): void;

  // Memory management
  getMemoryUsage(): MemoryInfo;
  evict(count?: number): number;

  // Persistence
  save(): SerializedCache;
  load(data: SerializedCache): void;
}

Types

interface CacheOptions {
  maxMemory?: number; // Max memory in bytes
  maxEntries?: number; // Max number of entries
  enableImmutability?: boolean; // Auto-protect cached values
  defaultTTL?: number; // Default time-to-live in ms
  evictionPolicy?: "lru" | "lfu" | "fifo";
  hashFunction?: (value: any) => string;
}

interface SetOptions {
  ttl?: number; // Time-to-live in ms
  size?: number; // Size in bytes (auto-calculated if not provided)
  immutable?: boolean; // Override immutability setting
  tags?: string[]; // Tags for grouped invalidation
}

interface CacheStats {
  hits: number;
  misses: number;
  hitRate: number;
  evictions: number;
  size: number;
  memoryUsage: number;
  avgGetTime: number;
  avgSetTime: number;
}

Advanced Usage

Multi-Level Cache

import { MultiLevelCache } from "@cook-step/cache-system";

const cache = new MultiLevelCache({
  levels: [
    { name: "l1", maxEntries: 100, ttl: 1000 }, // Hot cache
    { name: "l2", maxEntries: 1000, ttl: 10000 }, // Warm cache
    { name: "l3", maxEntries: 10000, ttl: 60000 }, // Cold cache
  ],
});

// Automatically promotes/demotes between levels
const value = cache.get("key"); // Checks L1 → L2 → L3

Hash-Based Keys

import { HashKeyCache } from "@cook-step/cache-system";

const cache = new HashKeyCache();

// Complex object as key
const key = {
  nodeId: "node1",
  inputs: { a: 1, b: 2 },
  params: { threshold: 0.5 },
};

// Automatically hashes to consistent string key
cache.set(key, computeExpensiveResult(key));

// Same object structure = same hash
const result = cache.get(key); // Cache hit!

Tagged Invalidation

// Set with tags
cache.set("user:123", userData, { tags: ["user", "session"] });
cache.set("user:456", userData, { tags: ["user"] });
cache.set("config", config, { tags: ["system"] });

// Invalidate by tag
cache.invalidateTag("user"); // Removes user:123 and user:456

Memory-Aware Caching

const cache = new CacheSystem({
  maxMemory: 50 * 1024 * 1024, // 50MB
  onMemoryPressure: (usage, limit) => {
    console.log(`Memory pressure: ${usage}/${limit}`);
  },
});

// Large data with size hint
const largeData = new Float32Array(1000000);
cache.set("mesh-data", largeData, {
  size: largeData.byteLength, // Help cache manage memory
});

Integration Examples

With Engine Runtime

import { CacheSystem } from "@cook-step/cache-system";

class RuntimeEngine {
  private cache = new CacheSystem({
    enableImmutability: true,
    maxMemory: 100 * 1024 * 1024,
  });

  executeNode(nodeId: string, inputs: any) {
    const cacheKey = this.generateKey(nodeId, inputs);

    // Check cache first
    const cached = this.cache.get(cacheKey);
    if (cached) return cached;

    // Execute and cache
    const result = this.compute(inputs);
    this.cache.set(cacheKey, result);

    return result;
  }
}

With React

import { useCacheSystem } from "@cook-step/cache-system/react";

function DataComponent() {
  const cache = useCacheSystem({
    maxEntries: 100,
    ttl: 5000,
  });

  const fetchData = async (id: string) => {
    const cached = cache.get(id);
    if (cached) return cached;

    const data = await api.getData(id);
    cache.set(id, data);
    return data;
  };
}

Performance

| Operation | Time | Memory | | ------------------- | -------- | ------ | | Set (small object) | ~0.01ms | O(1) | | Get (cache hit) | ~0.001ms | O(1) | | Get (cache miss) | ~0.002ms | O(1) | | Invalidate pattern | ~0.1ms | O(n) | | LRU eviction | ~0.05ms | O(1) | | Hash key generation | ~0.02ms | O(k) |

Testing

# Run tests
pnpm test

# Run tests with coverage
pnpm test:coverage

# Watch mode
pnpm test:watch

Current test coverage: 93.7% with 61 tests covering all major functionality.

Contributing

Contributions are welcome! Please ensure:

• All tests pass (pnpm test)
• Type checking passes (pnpm typecheck)
• Coverage remains above 90%

License

MIT