@omnifolio/tiered-cache

Two-tier caching with L1 in-memory + L2 Redis, stale-while-revalidate, circuit breaker state, request deduplication, and LRU eviction. Zero dependencies. Built for multi-instance deployments on Cloud Run, Kubernetes, etc.

Built by OmniFolio — Financial Intelligence Platform.

The Problem

When you auto-scale to N instances (Cloud Run, K8s, ECS), each instance has its own in-memory cache. Without a shared L2 layer:

• 20 instances × 30 API calls = 600 external calls/minute instead of 30
• API rate limits blown, bills explode, users get errors

The Solution

Request → L1 (in-memory, 0ms) → L2 (Redis, ~2ms) → Origin

First instance fetches from origin and writes to Redis. All other instances read from Redis — zero redundant external calls.

Graceful degradation: No Redis? Falls back to per-instance Map. Redis down? Falls back to per-instance Map. Zero crashes.

Install

npm install @omnifolio/tiered-cache

Quick Start

Shared Cache (L1 + L2)

import { configureL2, cacheGet, cacheSet } from '@omnifolio/tiered-cache';
import { Redis } from '@upstash/redis';

// Configure L2 (optional — works without it in dev)
const redis = new Redis({ url: '...', token: '...' });
configureL2({
  get: (key) => redis.get(key),
  set: (key, value, ttl) => redis.set(key, value, { ex: ttl }),
  del: (key) => redis.del(key).then(() => {}),
  ping: () => redis.ping().then(() => {}),
});

// Use it
const data = await cacheGet<MyData>('api:prices');
if (!data) {
  const fresh = await fetchPrices();
  await cacheSet('api:prices', fresh, 300); // 5min TTL
}

SWR Cache (Stale-While-Revalidate)

import { SWRCache, SWRPresets } from '@omnifolio/tiered-cache';

const cache = new SWRCache(500); // max 500 entries

// Returns stale data instantly, refreshes in background
const prices = await cache.swr(
  'prices:BTC',
  () => fetchBTCPrice(),
  SWRPresets.frequent, // 30s fresh, 5min max
);

// Subscribe to updates
const unsub = cache.subscribe('prices:BTC', (data) => {
  console.log('Price updated:', data);
});

// Optimistic mutation
cache.mutate('prices:BTC', (current) => ({
  ...current,
  price: newPrice,
}));

Circuit Breaker

import { getCircuitBreakerState, setCircuitBreakerState } from '@omnifolio/tiered-cache';

const state = await getCircuitBreakerState('coinbase-api');
if (state.open) {
  return cachedFallback();
}

try {
  const data = await fetchFromCoinbase();
  await setCircuitBreakerState('coinbase-api', {
    open: false, openedAt: 0, consecutiveErrors: 0,
  });
  return data;
} catch {
  await setCircuitBreakerState('coinbase-api', {
    open: state.consecutiveErrors >= 4,
    openedAt: Date.now(),
    consecutiveErrors: state.consecutiveErrors + 1,
  });
}

SWR Presets

| Preset | TTL (fresh) | Max Age | Use Case | |--------|------------|---------|----------| | realtime | 15s | 1min | Live prices | | frequent | 30s | 5min | Dashboards | | standard | 1min | 10min | General data | | slow | 5min | 1hr | Reference data | | static | 1hr | 24hr | Config/metadata |

API Reference

Shared Cache

| Function | Description | |----------|-------------| | configureL2(adapter, l1Ttl?) | Set up the Redis adapter | | cacheGet<T>(key) | Get from L1 → L2 | | cacheSet<T>(key, value, ttlSeconds) | Write to L1 + L2 | | cacheDel(key) | Delete from L1 + L2 | | healthCheck() | Check L2 connectivity + latency | | clearL1() | Clear L1 (for tests) |

SWR Cache

| Method | Description | |--------|-------------| | swr(key, fetcher, options?) | Get with stale-while-revalidate | | set(key, data, options?) | Direct set | | get(key) | Direct get (no revalidation) | | invalidate(key) | Remove a key | | invalidatePattern(pattern) | Remove by glob pattern | | mutate(key, mutator) | Optimistic update | | subscribe(key, callback) | Listen for updates | | getStats() | Cache statistics |

Circuit Breaker

| Function | Description | |----------|-------------| | getCircuitBreakerState(provider) | Read state from shared cache | | setCircuitBreakerState(provider, state) | Update state |

License

MIT — see LICENSE.

Built with ❤️ by OmniFolio