@asamuzakjp/generational-cache

v3.0.1

Published

25 days ago

A generational cache with strict entry-count limits and payload validation.

0High
0Medium
0Low

asamuzakjp

generational-cache

A lightweight, generational cache with strict entry-count limits and payload validation.

How it Works

GenerationalCache maintains two internal Map objects: current and old. It uses an internal boundary of ($max / 2$), allowing both generations to remain bounded while keeping generation swaps inexpensive.

Insertion & Validation: New items are validated against byte size limits before being added to the current generation.
Promotion: If you get an item that exists in the old generation, it is promoted to the current generation to ensure it stays in the cache longer.
Generation Swapping: Once the current generation's size meets or exceeds the boundary threshold ($max / 2$), a generation swap is triggered: the existing old generation is discarded, the current generation becomes the new old generation, and a new empty current generation is created.

This two-generation approach avoids the overhead of updating timestamps or linked list pointers on every access. While not a drop-in replacement for standard LRU caches, it prioritizes raw throughput over strict eviction ordering.

When NOT to Use

GenerationalCache may not be a good fit when:

You require strict LRU eviction ordering.
Cache hit rate is more important than insertion/eviction throughput.
You already know your working set size and can provision a sufficiently large LRU cache.

Installation

npm i @asamuzakjp/generational-cache

Usage

import { GenerationalCache } from '@asamuzakjp/generational-cache';

// Initialize with a max capacity of 1024 items and custom size limits
const cache = new GenerationalCache(1024, {
  maxKeySize: 4096,          // 4 KB limit for keys
  maxValueSize: 1024 * 1024, // 1 MB limit for values
  strictValidate: true       // Enable strict deep validation for objects (default)
});

API

`new GenerationalCache(max, opt)`

Creates a new cache instance with a maximum capacity of max entries.

max (number): The maximum number of items the cache can hold. If the specified value is less than 4, or if an invalid value is specified, the default value of 4 will be used.
opt (object, optional):
- cacheFunction (boolean): Caches functions if true. Defaults to false.
- cacheSymbol (boolean): Caches symbols if true. Defaults to false.
- maxKeySize (number): Maximum allowed size for a key in bytes. Defaults to 8192 (8 KB).
- maxValueSize (number): Maximum allowed size for a value in bytes. Defaults to 1048576 (1 MB).
- strictValidate (boolean): Strictly validate object payload structures and sizes if true. Defaults to true.

Properties

cache.entryCount (number, read-only): Returns the total number of underlying entries currently stored across both generations. Note: To maximize write throughput, this library allows temporary key duplication between the current and old generations (e.g., when an item exists in both generations simultaneously). The reported count may exceed the number of unique keys, but remains bounded by the cache's internal capacity.
cache.max (number): Gets or sets the maximum item capacity. Note: Updating this property dynamically will clear all existing cached items (it implicitly invokes cache.clear() to safely recalculate boundaries).

Methods

cache.get(key) Retrieves an item. If the item is found in the old generation, it is automatically promoted to the current generation to prevent it from being evicted during the next swap.
- Returns (any | undefined): The value associated with the key, or undefined if the key is not found.
cache.set(key, value) Adds or updates an item. It validates the byte size of both the key and value (if they are strings) before inserting. If this operation causes the current generation's size to meet or exceed the boundary threshold, a generation swap is triggered.
- Note: Storing undefined as a value is not supported, as cache.get(key) treats undefined as a cache miss.
- Returns: The cache instance itself (allows chaining).
cache.has(key) Checks if a key exists in the cache (in either generation).
- Returns: true if the key exists, otherwise false.
cache.delete(key) Removes an item from the cache.
- Returns: true if the item existed and was removed, otherwise false.
cache.clear() Empties all items from the cache.

Performance

Benchmarks are divided into two states to simulate real-world conditions:

Cold State: Measured with aggressive internal Garbage Collection to observe performance before full V8 TurboFan optimizations.
Warm State: Measured after sufficient warmup, representing sustained throughput under optimal JIT compilation.

The results below reflect the sustained operations per second (ops/sec), calculated from the average latency (ns/iter). Higher values indicate better performance.

Benchmark Environment

Engine: Node.js v24.x (V8)
Measurement: mitata.
Comparison: LRUCache (v11.x), QuickLRU (v7.x), Mnemonist (v0.40.x)

1. Small Cache (Max Size = 512)

2. Medium Cache (Max Size = 2,048)

3. Large Cache (Max Size = 8,192)

4. Non-Primitive Payload (Max Size = 4,096 / strictValidate = true)

5. Non-Primitive Payload (Max Size = 4,096 / strictValidate = false)

6. Cyclic Access (Max Size = 8,192 / Working Set = 5,000)

| Metric | GenerationalCache | LRUCache | QuickLRU | Mnemonist | | :---- | :---- | :---- | :---- | :---- | | Hit Rate | 78.30% | 100.00% | 100.00% | 100.00% | | Throughput | 8,783,487 ops/sec | 37,778,617 ops/sec | 38,550,501 ops/sec | 44,742,729 ops/sec |

7. Cyclic Access (Max Size = 4,096 / Working Set = 5,000)

| Metric | GenerationalCache | LRUCache | QuickLRU | Mnemonist | | :---- | :---- | :---- | :---- | :---- | | Hit Rate | 0.00% | 0.00% | 78.30% | 0.00% | | Throughput | 9,931,472 ops/sec | 6,396,724 ops/sec | 8,509,189 ops/sec | 6,078,288 ops/sec |

Key Characteristics

High Eviction Efficiency: GenerationalCache demonstrates strong throughput during high-turnover workloads, maintaining a performance margin compared to standard LRU designs in large-scale eviction scenarios.
Predictable Scalability: While other libraries may experience performance degradation as cache size increases, GenerationalCache maintains consistent throughput due to its generational swap mechanism.
Balanced Read/Write: It provides stable and competitive performance across all basic operations (get, set), making it suitable for both read-heavy and write-heavy environments.
Strict Validation Toggle: By default, non-primitive payloads undergo deep validation to prevent memory exhaustion from oversized objects, which impacts write throughput. Disabling strictValidate restores write performance, provided that payload sizes are managed externally (See 4 & 5).
Trade-offs: In cyclic access patterns where the working set is greater than max / 2 but smaller than max, GenerationalCache will experience frequent generation swaps and cache misses. To maximize the performance benefits of GenerationalCache, it is often better to keep the max size small enough to allow some evictions, rather than trying to fit the entire working set (See 6 & 7).

License

MIT