ecoai

Stop paying for the same AI response twice.

EcoAI is an open-source caching SDK for AI-powered applications. It intercepts repetitive LLM calls, stores the responses, and returns them instantly — saving you token costs, slashing latency, and reducing the environmental footprint of your AI apps.

Before EcoAI  →  38 identical calls  →  $4.26 spent  →  142,000 tokens used
After EcoAI   →  1 real call + 37 cached  →  $0.11 spent  →  97% saved

Install

npm install eco-ai

Optional peer dependencies (install whichever provider(s) you use):

npm install openai                  # OpenAI
npm install @anthropic-ai/sdk       # Anthropic
npm install @google/generative-ai   # Google Gemini
npm install ioredis                 # Redis storage (optional)

Quickstart

OpenAI

import { EcoAI } from 'eco-ai';
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const eco = new EcoAI({ client: openai, mode: 'dev' });

// Use eco exactly like your regular OpenAI client.
const response = await eco.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Summarise the water cycle.' }],
});

// Second call with the same prompt? Returns from cache. Instantly. Free.

Anthropic

import { EcoAI } from 'eco-ai';
import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
const eco = new EcoAI({ client: anthropic, mode: 'dev' });

const response = await eco.messages.create({
  model: 'claude-3-5-sonnet-20241022',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Explain quantum entanglement.' }],
});

Google Gemini

import { EcoAI } from 'eco-ai';
import { GoogleGenerativeAI } from '@google/generative-ai';

const gemini = new GoogleGenerativeAI(process.env.GEMINI_API_KEY!);
const eco = new EcoAI({ client: gemini, mode: 'dev' });

const response = await eco.generateContent({
  model: 'gemini-2.5-flash',
  prompt: 'What is photosynthesis?',
});

That's the entire integration. No new infrastructure. No config files. No account needed. EcoAI stores responses in a local SQLite file and serves them from cache on subsequent calls.

How It Works

Your app code
     │
     ▼
┌─────────────────────────────────────┐
│            EcoAI SDK                │
│                                     │
│  1. Hash prompt + model + params    │
│  2. Check cache (SQLite / Redis)    │
│                                     │
│  Cache HIT  ────────────────────►  Return stored response (0ms, $0)
│                                     │
│  Cache MISS ────────────────────►  Forward to AI provider
│                                     │        │
│  3. Store response in cache  ◄──────┘        │
│  4. Log usage (tokens, cost, CO₂)            │
│  5. Return response to your app  ◄───────────┘
└─────────────────────────────────────┘

EcoAI uses exact-match caching (Phase 1): a SHA-256 hash of the full request (model, messages, parameters) as the cache key. Identical requests return instantly from cache.

Configuration

const eco = new EcoAI({
  client: openai,         // Your existing AI client (required)
  mode: 'dev',            // 'dev' (default) | 'prod'
  storage: 'sqlite',      // 'sqlite' (default) | 'redis' | 'memory'
  sqlitePath: '.ecoai/cache.db',   // SQLite file path (default shown)
  redisUrl: 'redis://...',         // Required if storage: 'redis'
  ttl: 3600,              // Cache TTL in seconds (prod mode only)
  ttlByModel: {           // Per-model TTL overrides
    'gpt-4o': 7200,
    'gpt-4o-mini': 1800,
  },
  logUsage: true,         // Enable usage logging (default: true)
  logPath: '.ecoai/usage.db',      // Usage log file path (default shown)
});

Environment variables

ECOAI_MODE=dev            # 'dev' | 'prod'
ECOAI_STORAGE=sqlite      # 'sqlite' | 'redis' | 'memory'
ECOAI_REDIS_URL=redis://...
ECOAI_TTL=3600

API Reference

new EcoAI(config)

Instantiates the caching client. Detects the AI provider from the client you pass.

Provider methods

eco.chat.completions.create(params)  // OpenAI — same signature as openai.chat.completions.create
eco.messages.create(params)          // Anthropic — same signature as anthropic.messages.create
eco.generateContent({ model, prompt }) // Gemini — unified interface

Streaming calls (stream: true) bypass the cache and pass through directly to the provider.

Cache controls

await eco.cache.flush()                        // Clear all cached responses
await eco.cache.flush({ model: 'gpt-4o' })     // Clear by model
await eco.cache.flush({ pattern: 'summarise*' }) // Clear by prompt glob pattern

Mode switching

eco.setMode('prod')  // Switch to prod mode (TTL-based expiry)
eco.setMode('dev')   // Switch back to dev mode (cache forever)

Usage statistics

const stats = await eco.usage.summary();
// {
//   totalCalls: 142,
//   cachedCalls: 137,
//   hitRate: 0.965,
//   tokensSaved: 89420,
//   costSaved: 4.15,
//   co2Saved: 0.00179   // kg CO₂
// }

const history = await eco.usage.history({ from: '2025-01-01', limit: 100 });
// UsageRecord[]

Storage backends

| Backend | Use case | Config | |---|---|---| | sqlite (default) | Local dev, single-process production | sqlitePath | | memory | Tests, ephemeral processes | — | | redis | Production, shared cache across processes | redisUrl, requires ioredis |

Dev vs Prod mode

| | dev mode | prod mode | |---|---|---| | Cache expiry | Never (infinite TTL) | Respects ttl (default 3600s) | | Best for | Local development, CI | Production deployments |

Requirements

• Node.js 18+
• better-sqlite3 (bundled as a dependency)
• Optional: ioredis for Redis storage
• Optional: openai, @anthropic-ai/sdk, @google/generative-ai — whichever provider you use

License

MIT — see LICENSE.