PromptCacheAI SDK

Provider-agnostic TypeScript SDK for adding PromptCacheAI semantic caching to AI applications.

PromptCacheAI sits before your model provider. It checks for exact or high-confidence similar requests, returns a cached response when appropriate, and lets your app call the model normally on a miss.

Install

npm install @promptcacheai/sdk

Requires Node.js 18 or later.

Recommended Workflow

1. Create a PromptCacheAI API key.
2. Create a namespace in test mode.
3. Wrap your existing model call with withCache.
4. Review cached responses and similar prompt variants in the dashboard.
5. Switch the namespace to live when you trust the cache behavior.

Test mode lets you see what PromptCacheAI would have reused without serving cached responses to users. Your app still calls your model provider, while the dashboard shows would-hits, reusable responses, and prompt variants to review.

5-Minute Quickstart

Use withCache around your existing model call. On a cache hit, PromptCacheAI returns the saved response. On a miss, your callModel function runs and the SDK saves the model response for future reuse when recommended.

import OpenAI from "openai";
import { PromptCacheAI } from "@promptcacheai/sdk";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY!,
});

const pcai = new PromptCacheAI({
  apiKey: process.env.PROMPTCACHEAI_API_KEY!,
});

export async function answerSupportQuestion(prompt: string) {
  const result = await pcai.withCache({
    prompt,
    namespace: "support-faq",
    provider: "openai",
    model: "gpt-4o-mini",
    callModel: async () => {
      const completion = await openai.chat.completions.create({
        model: "gpt-4o-mini",
        messages: [{ role: "user", content: prompt }],
      });

      return completion.choices[0]?.message?.content ?? "";
    },
  });

  return result.response;
}

Example result:

{
  response: "You have 30 days to return items with a receipt.",
  cached: false,
  source: "model",
  promptHash: "82d2bf...",
  cacheMode: "test",
  saveRecommended: true,
  cacheDecision: "miss_no_candidate",
  canonicalPromptHash: undefined,
  saved: true
}

Provider Agnostic

The SDK does not call OpenAI, Anthropic, Gemini, or any other model provider directly. You provide the model call with callModel, so PromptCacheAI can wrap the code you already have.

const result = await pcai.withCache({
  prompt,
  namespace: "docs-bot",
  provider: "custom",
  model: "my-internal-model",
  callModel: async () => myInternalModel(prompt),
});

Production-Safe Defaults

withCache fails open by default:

• If /chat fails, your model call still runs.
• If there is a cache miss, your model call runs.
• If /cache/save fails after the model responds, the model response is still returned.
• If PromptCacheAI says save_recommended: false, the SDK skips saving.

This keeps PromptCacheAI from blocking your application unless you explicitly choose stricter behavior.

Use strict: true only if you want PromptCacheAI errors to throw:

const pcai = new PromptCacheAI({
  apiKey: process.env.PROMPTCACHEAI_API_KEY!,
  strict: true,
});

Low-Level API

You can also control the workflow manually with chat and save.

const chat = await pcai.chat({
  prompt,
  namespace: "support-faq",
  provider: "openai",
  model: "gpt-4o-mini",
});

if (chat.cached) {
  return chat.response;
}

const response = await callModel(prompt);

if (chat.saveRecommended !== false) {
  await pcai.save({
    promptHash: chat.promptHash,
    response,
    namespace: "support-faq",
  });
}

return response;

Client Options

const pcai = new PromptCacheAI({
  apiKey: process.env.PROMPTCACHEAI_API_KEY!,
  baseUrl: "https://api.prompt-cache.ai/v1",
  timeoutMs: 10_000,
  strict: false,
  fetch: globalThis.fetch,
  onEvent: (event) => console.log(event.type),
});

| Option | Required | Description | | --- | --- | --- | | apiKey | Yes | Your PromptCacheAI API key. Keep this server-side. | | baseUrl | No | PromptCacheAI API base URL. Defaults to https://api.prompt-cache.ai/v1. | | timeoutMs | No | Request timeout for PromptCacheAI API calls. Defaults to 10000. | | strict | No | When false, withCache fails open. When true, PromptCacheAI errors throw. Defaults to false. | | fetch | No | Custom fetch implementation. Useful for custom runtimes or tests. | | onEvent | No | Callback for cache, model, and save events. Useful for logs or tracing. |

Method Reference

withCache(options)

Recommended integration path. Checks PromptCacheAI, calls your model on a miss, and saves the model response when appropriate.

const result = await pcai.withCache({
  prompt: "What is your refund policy?",
  namespace: "support-faq",
  provider: "openai",
  model: "gpt-4o-mini",
  callModel: async () => "You have 30 days to return items with a receipt.",
});

Returns:

{
  response: string;
  cached: boolean;
  source: "cache" | "model";
  promptHash?: string;
  cacheMode?: string;
  saveRecommended?: boolean;
  cacheDecision?: string;
  canonicalPromptHash?: string;
  saved: boolean;
  chatError?: unknown;
  saveError?: unknown;
}

chat(request)

Checks PromptCacheAI for an exact or semantic cache hit.

const chat = await pcai.chat({
  prompt: "What is your return policy?",
  namespace: "support-faq",
  provider: "openai",
  model: "gpt-4o-mini",
});

Returns:

{
  cached: boolean;
  response: string;
  promptHash: string;
  cacheMode?: string;
  saveRecommended?: boolean;
  cacheDecision?: string;
  canonicalPromptHash?: string;
  raw: unknown;
}

save(request)

Saves a model response after a miss.

await pcai.save({
  promptHash: chat.promptHash,
  response: "You have 30 days to return items with a receipt.",
  namespace: "support-faq",
});

Events

Use onEvent to observe cache behavior without changing your application flow.

const pcai = new PromptCacheAI({
  apiKey: process.env.PROMPTCACHEAI_API_KEY!,
  onEvent: (event) => {
    if (event.type === "model_call") {
      console.log("Calling model because of", event.reason);
    }
  },
});

Event types include:

• chat_success
• chat_error
• model_call
• save_skipped
• save_success
• save_error

Optional Metadata

temperature is available as optional metadata on chat and withCache, but most integrations can omit it. It is included for compatibility with applications that want to pass model settings through the cache lookup.

Server-Side Use

Keep your PromptCacheAI API key on the server. Do not expose it in browser code, mobile apps, or public client bundles.

For frontend apps, call PromptCacheAI from your server route, API route, backend service, or edge/server runtime where secrets are protected.