@layrx/ai-cache

AI Cache Manager for LayrX. Caches AI request and response data based on user queries and context packages, independently of any LLM provider.

Architecture

User Query + Context Package
            ↓
      CacheHasher (deterministic key)
            ↓
       CacheStore (SQLite)
            ↓
     Hit? → Return response
     Miss? → Ready for future LLM call
            ↓
       Store response (set)

| File | Responsibility | |------|----------------| | AICacheManager.ts | Orchestrator and public API | | CacheStore.ts | SQLite persistence | | CacheHasher.ts | Key generation and context hashing | | CachePolicy.ts | TTL and eviction policy | | CacheCleaner.ts | Expired entry cleanup | | CacheStatistics.ts | Aggregate metrics | | CacheTypes.ts | Shared types |

Key generation

Cache keys are SHA-256 digests of:

• Normalised user query
• Context package hash (canonical JSON)
• Model name
• Model version
• Prompt version

Equivalent context packages produce identical keys regardless of property order.

TTL strategy

• Default TTL: 24 hours
• Configurable per manager (policy.defaultTtlMs) or per entry (ttlMs on set)
• Expired entries are treated as cache misses
• Optional eviction on lookup (evictOnLookup: true, default)
• clearExpired() removes all expired rows

Public API

import {
  get,
  set,
  deleteCache,
  clearExpired,
  clearAll,
  getStatistics,
} from '@layrx/ai-cache';

const context = {
  repositoryId: 'repo-id',
  summary: 'Relevant code context',
  entities: [{ entityId: '1', entityType: 'Function', name: 'Widget' }],
};

// Lookup
const cached = get({
  query: 'Explain Widget',
  context,
  model: 'gpt-4',
  modelVersion: '2024-01',
  promptVersion: 'v1',
});

if (cached) {
  console.log(cached.response);
} else {
  // Future: call LLM, then store response
  set({
    query: 'Explain Widget',
    context,
    model: 'gpt-4',
    modelVersion: '2024-01',
    promptVersion: 'v1',
    response: 'Widget is a React component...',
    tokenUsage: { promptTokens: 100, completionTokens: 50, totalTokens: 150 },
  });
}

Class API: new AICacheManager({ databasePath, policy })

Schema

ai_cache

cache_key, query, context_hash, model, model_version, prompt_version, response, token_usage, created_at, expires_at, last_accessed, hit_count

cache_statistics

Singleton row tracking cache_hits and cache_misses

Statistics

• totalEntries
• cacheHits / cacheMisses
• hitRate
• expiredEntries
• databaseSize

Logging

Cache lookup → Cache hit / Cache miss → Entry created → Entry expired → Cache cleaned → Errors

Future distributed cache support

• Stable cache_key maps directly to Redis/Memcached keys
• context_hash enables cache invalidation when context changes
• CacheStore can be swapped for a DistributedCacheStore adapter
• TTL maps to Redis EXPIRE; hit counts via HINCRBY
• SQLite remains local L1; Redis as shared L2 across agents

Tests

npm run test --workspace=@layrx/ai-cache

Flow

User Query → Context Package → Generate Cache Key → Lookup SQLite
  → Hit: Return Response
  → Miss: Ready for LLM → Store Response

Do not use this package for LLM calls, HTTP APIs, or security scanning.