@nodellmcache/vercel-ai

Vercel AI SDK caching middleware for NodeLLMCache. cacheMiddleware is a LanguageModelV1Middleware you wrap any model with, so generateText, generateObject, streamText, and streamObject are cached across any NodeLLMCache backend.

Install

npm install @nodellmcache/vercel-ai @nodellmcache/core ai
# plus an adapter, e.g. @nodellmcache/redis

Quick start

import { wrapLanguageModel, generateText } from 'ai'
import { openai } from '@ai-sdk/openai'
import { cacheMiddleware } from '@nodellmcache/vercel-ai'
import { RedisAdapter } from '@nodellmcache/redis'

const model = wrapLanguageModel({
  model: openai('gpt-4o'),
  middleware: cacheMiddleware({
    adapter: new RedisAdapter({ host: 'localhost', port: 6379 }),
    ttl: 60 * 60 * 1000, // optional
  }),
})

await generateText({ model, prompt: 'Explain Redis' }) // hits the API
await generateText({ model, prompt: 'Explain Redis' }) // served from cache

What's cached

• wrapGenerate — generateText / generateObject results are cached by a hash of the full call params (prompt + settings).
• wrapStream — streamText / streamObject: the stream parts are captured on a miss (forwarded live) and replayed as a stream on a hit. An interrupted stream is not cached.

Options

| Option | Default | Description | |--------|---------|-------------| | adapter | — (required) | Any @nodellmcache storage adapter | | ttl | none | Relative TTL (ms) for cached results | | namespace | vercel-ai | Key namespace |

Because it is just a StorageAdapter, you get in-memory, Redis, multi-tier, and at-rest encryption for free by swapping the adapter. Note that with a JSON-serializing backend (e.g. Redis), only JSON-safe fields of a result survive a round-trip; the in-memory adapter is lossless.

License

MIT