Integration Coins Cache - E2E Cache Validation

Comprehensive end-to-end validation of the unified cache system.

Overview

This dedicated test package validates all aspects of the @pioneer-platform/pioneer-cache module:

• Displays all cached entries in a table with TTLs
• Shows cache health status
• Tests cache behavior (miss, hit, stale, refresh)
• Clears cache and validates empty state
• Tests from fresh state

Running the Test

# Start the pioneer-server first
cd /Users/highlander/WebstormProjects/keepkey-stack/projects/pioneer
make start

# In another terminal, run the cache validation test
cd e2e/wallets/intergration-coins-cache
bun run dev

What the Test Does

1. Cache Health Check

• Queries /cache/health endpoint
• Displays overall status (healthy/degraded/unhealthy)
• Shows individual cache health (balance, price, transaction)
• Displays worker status and queue length
• Lists any issues or warnings

2. Cache Table Display

• Scans Redis for all cache entries
• Groups by type: balance, price, transaction
• Displays in formatted table with:
  • Key/CAIP identifier
  • Cached value (balance, price)
  • TTL remaining (seconds)
  • Age (seconds since cached)
  • Status (fresh vs stale)
• Shows summary statistics

3. Cache Behavior Tests

Tests the 5 critical fixes:

TEST 1: Cache Miss (First Request)

• Request price for an asset
• Expected: Blocks and fetches from source (Fix #1)
• Validates blocking behavior on cache miss
• Records response time

TEST 2: Cache Hit (Second Request)

• Request same asset again
• Expected: < 100ms from cache
• Validates stale-while-revalidate pattern
• Should be 50%+ faster than first request

TEST 3: Force Refresh

• Manually trigger refresh via /cache/refresh/{caip}
• Expected: Blocks and returns fresh data
• Validates manual refresh capability

4. Cache Clear

• Deletes all cache entries from Redis
• Validates all keys removed
• Shows before/after counts

5. Fresh State Testing

• Repeats cache behavior tests from empty cache
• Validates cache miss → cache hit flow
• Confirms TTLs are properly set (Fix #2)

Expected Output

╔═══════════════════════════════════════════════════════════════════════════════╗
║                     PIONEER CACHE SYSTEM AUDIT                                ║
╚═══════════════════════════════════════════════════════════════════════════════╝

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🏥 CACHE HEALTH CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  Overall Status: ✅ HEALTHY
  Queue Initialized: ✅
  Redis Connected: ✅

  Individual Cache Health:
    - Balance Cache: ✅ healthy
    - Price Cache: ✅ healthy
    - Transaction Cache: ✅ healthy
    - Background Worker: ✅ RUNNING
      Queue Length: 0 jobs

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 CACHE AUDIT - All Cached Entries
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

💰 BALANCE CACHE (TTL: 5min, Stale: >2min)
────────────────────────────────────────────────────────────────────────────────
  Key                                            Balance       TTL       Age       Status
────────────────────────────────────────────────────────────────────────────────
  balance_v2:eip155:1/slip44:60:xpub123...       1.50000000    285s      15s       ✅ FRESH
  balance_v2:eip155:1/slip44:0:xpub456...        0.05000000    290s      10s       ✅ FRESH

💵 PRICE CACHE (TTL: 1hr, Stale: >30min)
────────────────────────────────────────────────────────────────────────────────
  CAIP                                    Price USD     TTL       Age       Status
────────────────────────────────────────────────────────────────────────────────
  eip155:1/slip44:60                      $2500.00      3450s     150s      ✅ FRESH
  eip155:1/slip44:0                       $45000.00     3470s     130s      ✅ FRESH

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 CACHE SUMMARY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Total Entries: 4
  - Balances: 2 (0 stale)
  - Prices: 2 (0 stale)
  - Transactions: 0 (permanent)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🧪 TESTING CACHE BEHAVIOR
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

TEST 1: Cache Miss (first request)
────────────────────────────────────────────────────────────────────────────────
  Price: $2500.00
  Response time: 450ms
  Expected: Blocking, fetch from source, then cache
  ✅ TEST 1 PASSED

TEST 2: Cache Hit (second request - should be instant)
────────────────────────────────────────────────────────────────────────────────
  Price: $2500.00
  Response time: 45ms
  Expected: < 100ms (from cache)
  ✅ TEST 2 PASSED (90% faster)

TEST 3: Force Refresh (manual trigger)
────────────────────────────────────────────────────────────────────────────────
  Price: $2500.00
  Fresh: YES
  Response time: 425ms
  Expected: Blocking, fetch fresh data
  ✅ TEST 3 PASSED

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ ALL CACHE BEHAVIOR TESTS PASSED
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🗑️  CLEARING ALL CACHE
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✅ Cleared 2 entries matching balance_v2:*
  ✅ Cleared 2 entries matching price_v2:*

  Total entries cleared: 4

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ CACHE CLEARED VERIFICATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Entries before clear: 4
  Entries cleared: 4
  Entries after clear: 0
  ✅ Cache successfully cleared!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

[Tests repeat from fresh state...]

╔═══════════════════════════════════════════════════════════════════════════════╗
║                        CACHE AUDIT COMPLETE                                   ║
╠═══════════════════════════════════════════════════════════════════════════════╣
║  ✅ All 5 Critical Fixes Validated:                                           ║
║  1. Cache Miss Blocking - VERIFIED                                            ║
║  2. Redis TTL Management - VERIFIED                                           ║
║  3. Loud Queue Failures - VERIFIED                                            ║
║  4. Synchronous Fallback - VERIFIED                                           ║
║  5. Health Monitoring - VERIFIED                                              ║
╚═══════════════════════════════════════════════════════════════════════════════╝

Validation Checklist

• [ ] Cache health endpoint returns healthy status
• [ ] All caches show in health check (balance, price, transaction)
• [ ] Background worker is running
• [ ] Cache table displays entries with correct TTLs
• [ ] Balance cache shows 5min TTL
• [ ] Price cache shows 1hr TTL
• [ ] Transaction cache shows PERMANENT (no TTL)
• [ ] Cache miss blocks and fetches from source
• [ ] Cache hit returns instantly (<100ms)
• [ ] Force refresh works and returns fresh data
• [ ] Cache clear removes all entries
• [ ] Fresh state tests work after clear

Troubleshooting

If health check fails:

• Ensure pioneer-server is running: make start
• Check Redis is running: redis-cli ping
• Verify server logs for errors

If cache entries don't appear:

• Make sure to run some API requests first to populate cache
• Try the behavior tests which will populate cache automatically

If TTLs are wrong:

• This indicates Fix #2 is not working
• Check server logs for cache write operations
• Verify Redis version supports EX parameter

If tests fail:

• Check server is accessible at http://127.0.0.1:9001
• Verify cache endpoints are registered
• Check Redis connectivity

Standalone Package

This is a standalone e2e test package focused solely on cache validation. It can be run independently without requiring the full integration-coins test suite.

CI/CD Integration

Add to GitHub Actions or CI pipeline:

- name: Run cache validation
  run: |
    cd e2e/wallets/intergration-coins-cache
    bun run test