@asamuzakjp/generational-cache

v2.0.2

Published

15 days ago

A generational pseudo-LRU cache with strict maximum size limits.

0High
0Medium
0Low

asamuzakjp

generational-cache

A lightweight, generational pseudo-LRU (Least Recently Used) cache with strict maximum size limits.

How it Works

GenerationalCache maintains two internal Map objects: current and old.

Insertion: New items are always added to the current generation.
Promotion: If you get an item that exists in the old generation, it is promoted to the current generation to ensure it stays in the cache longer.
Generation Swapping: Once the current generation reaches the boundary size ($max / 2$), the old generation is discarded, the current generation becomes the old generation, and a new empty current generation is created.

This "pseudo-LRU" approach avoids the overhead of updating timestamps or linked list pointers on every access. While not a drop-in replacement for standard LRU caches, it prioritizes raw throughput over strict eviction ordering.

Installation

npm i @asamuzakjp/generational-cache

Usage

import { GenerationalCache } from '@asamuzakjp/generational-cache';

// Initialize with a max capacity of 1024 items
const cache = new GenerationalCache(1024);

API

`new GenerationalCache(max)`

Creates a new cache instance.

max (number): The maximum number of items the cache can hold. If the specified value is less than 4, or if an invalid value is specified, the default value of 4 will be used.

Properties

cache.size (number, read-only): Returns the total number of entries currently in the cache. Note: To optimize for write speed, this library allows temporary key duplication between generations. Therefore, this value may not always reflect the exact count of unique keys.
cache.max (number): Gets or sets the maximum capacity. Note: Updating this property dynamically will invoke cache.clear() to safely recalculate boundaries.

Methods

cache.get(key) Retrieves an item. If the item is found in the older generation, it is automatically promoted to the current generation to prevent it from being evicted during the next swap.
- Returns: The value associated with the key, or undefined.
cache.set(key, value) Adds or updates an item. If adding this item pushes the current generation's size to the boundary threshold (max / 2), a generation swap is triggered, and the old generation is discarded.
- Returns: The cache instance itself (allows chaining).
cache.has(key) Checks if a key exists in the cache (in either generation).
- Returns: true if the key exists, otherwise false.
cache.delete(key) Removes an item from the cache.
- Returns: true if the item existed and was removed, otherwise false.
cache.clear() Empties all items from the cache by dropping references to the internal Maps.

Performance

Benchmarks are divided into two states to simulate real-world conditions:

Cold State: Measured with aggressive internal Garbage Collection to observe performance before full V8 TurboFan optimizations.
Warm State: Measured after sufficient warmup, representing sustained throughput under optimal JIT compilation.

The results below reflect the sustained operations per second (ops/sec), calculated from the average latency (ns/iter). Higher values indicate better performance.

Benchmark Environment

Engine: Node.js v24.x (V8)
Measurement: mitata.
Comparison: LRUCache (v11.x), QuickLRU (v7.x), Mnemonist (v0.40.x)

1. Small Cache (Max Size = 512)

2. Medium Cache (Max Size = 2,048)

3. Large Cache (Max Size = 8,192)

4. Cyclic Access (Max Size = 8,192 / Working Set = 5,000)

| Metric | GenerationalCache | LRUCache | QuickLRU | Mnemonist | | :--- | :--- | :--- | :--- | :--- | | Hit Rate | 78.30% | 100.00% | 100.00% | 100.00% | | Throughput | 10,365,916 ops/sec | 40,832,993 ops/sec | 40,950,040 ops/sec | 48,426,150 ops/sec |

5. Cyclic Access (Max Size = 4,096 / Working Set = 5,000)

| Metric | GenerationalCache | LRUCache | QuickLRU | Mnemonist | | :--- | :--- | :--- | :--- | :--- | | Hit Rate | 18.06% | 18.06% | 99.98% | 18.06% | | Throughput | 13,192,612 ops/sec | 9,672,115 ops/sec | 10,188,487 ops/sec | 7,564,868 ops/sec |

Key Characteristics

High Eviction Efficiency: GenerationalCache demonstrates strong throughput during high-turnover workloads, maintaining a performance margin compared to standard LRU designs in large-scale eviction scenarios.
Predictable Scalability: While other libraries may experience performance degradation as cache size increases, GenerationalCache maintains consistent throughput due to its generational swap mechanism.
Balanced Read/Write: It provides stable and competitive performance across all basic operations (get, set), making it suitable for both read-heavy and write-heavy environments.
Trade-offs: In cyclic access patterns where the working set is greater than max / 2 but smaller than max, GenerationalCache will experience frequent generation swaps and cache misses. To maximize the performance benefits of GenerationalCache, it is often better to keep the max size small enough to allow some evictions, rather than trying to fit the entire working set (See 4 & 5).

License

MIT