@nodellmcache/langchain

LangChain integration for NodeLLMCache. NodeLLMCache is a LangChain BaseCache backed by any NodeLLMCache StorageAdapter, so you can cache LangChain LLM/chat responses across the whole backend ecosystem (in-memory, Redis, tiered, encrypted, ...).

Install

npm install @nodellmcache/langchain @nodellmcache/core @langchain/core
# plus whichever adapter you want, e.g. @nodellmcache/redis

Quick start

import { ChatOpenAI } from '@langchain/openai'
import { NodeLLMCache } from '@nodellmcache/langchain'
import { RedisAdapter } from '@nodellmcache/redis'

const model = new ChatOpenAI({
  model: 'gpt-4o',
  cache: new NodeLLMCache({
    adapter: new RedisAdapter({ host: 'localhost', port: 6379 }),
    ttl: 24 * 60 * 60 * 1000, // optional
  }),
})

await model.invoke('Explain Redis')  // hits the API
await model.invoke('Explain Redis')  // served from your NodeLLMCache backend

Any LangChain model that accepts a cache works the same way. Because the cache is just a StorageAdapter, you get Redis, multi-tier, and at-rest encryption for free by swapping the adapter.

Options

| Option | Default | Description | |--------|---------|-------------| | adapter | — (required) | Any @nodellmcache storage adapter holding Generation[] | | ttl | none | Relative TTL (ms) for cached generations | | namespace | langchain | Key namespace (isolate multiple apps on one backend) |

Keys are built from both the prompt and LangChain's llmKey (model + params), hashed via KeyBuilder — raw prompt text never lands in a key.

License

MIT