@redis-ai/langcache

Developer-friendly & type-safe Typescript SDK specifically catered to leverage @redis-ai/langcache API.

Summary

LangCache: API for managing a Redis LangCache service.

Table of Contents

• @redis-ai/langcache
  • SDK Installation
  • Requirements
  • SDK Example Usage
  • Available Resources and Operations
  • Standalone functions
  • Retries
  • Error Handling
  • Custom HTTP Client
  • Debugging
• Development
  • Maturity

SDK Installation

The SDK can be installed with either npm, pnpm, bun or yarn package managers.

NPM

npm add @redis-ai/langcache

PNPM

pnpm add @redis-ai/langcache

Bun

bun add @redis-ai/langcache

Yarn

yarn add @redis-ai/langcache

[!NOTE] This package is published with CommonJS and ES Modules (ESM) support.

Requirements

For supported JavaScript runtimes, please consult RUNTIMES.md.

SDK Example Usage

Save an entry

Save an entry to the cache

import { LangCache } from "@redis-ai/langcache";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  const result = await langCache.set({
    prompt: "How does semantic caching work?",
    response:
      "Semantic caching stores and retrieves data based on meaning, not exact matches.",
  });

  console.log(result);
}

run();

Search for entries

Search for entries in the cache

import { LangCache } from "@redis-ai/langcache";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  const result = await langCache.search({
    prompt: "How does semantic caching work?",
  });

  console.log(result);
}

run();

Delete an entry

Delete an entry from the cache by id

import { LangCache } from "@redis-ai/langcache";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  await langCache.deleteById("<id>");
}

run();

Delete entries

Delete entries based on attributes

import { LangCache } from "@redis-ai/langcache";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  const result = await langCache.deleteQuery({
    attributes: {
      "language": "en",
      "topic": "ai",
    },
  });

  console.log(result);
}

run();

Flush cache

Flush all entries from the cache

import { LangCache } from "@redis-ai/langcache";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  await langCache.flush();
}

run();

Use exact search

Search for entries in the cache using both exact and semantic search

import { LangCache } from "@redis-ai/langcache";
import { SearchStrategy } from '@redis-ai/langcache/models/searchstrategy.js';

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  const result = await langCache.search({
    prompt: "How does semantic caching work?",
    searchStrategies: [SearchStrategy.Exact, SearchStrategy.Semantic],
  });

  console.log(result);
}

run();

Available Resources and Operations

LangCache SDK

• deleteQuery - Deletes multiple cache entries based on specified attributes. If no attributes are provided, all entries in the cache are deleted.
• set - Adds an entry to the cache with a prompt and response.
• search - Searches the cache for entries that match the prompt and attributes. If no entries are found, this endpoint returns an empty array.
• deleteById - Deletes a single cache entry by the entry ID.
• flush - Flushes all entries from the cache.

Standalone functions

All the methods listed above are available as standalone functions. These functions are ideal for use in applications running in the browser, serverless runtimes or other environments where application bundle size is a primary concern. When using a bundler to build your application, all unused functionality will be either excluded from the final bundle or tree-shaken away.

To read more about standalone functions, check FUNCTIONS.md.

• deleteById - Deletes a single cache entry by the entry ID.
• deleteQuery - Deletes multiple cache entries based on specified attributes. If no attributes are provided, all entries in the cache are deleted.
• flush - Flushes all entries from the cache.
• search - Searches the cache for entries that match the prompt and attributes. If no entries are found, this endpoint returns an empty array.
• set - Adds an entry to the cache with a prompt and response.

Retries

Some of the endpoints in this SDK support retries. If you use the SDK without any configuration, it will fall back to the default retry strategy provided by the API. However, the default retry strategy can be overridden on a per-operation basis, or across the entire SDK.

To change the default retry strategy for a single API call, simply provide a retryConfig object to the call:

import { LangCache } from "@redis-ai/langcache";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  const result = await langCache.deleteQuery({
    attributes: {
      "language": "en",
      "topic": "ai",
    },
  }, {
    retries: {
      strategy: "backoff",
      backoff: {
        initialInterval: 1,
        maxInterval: 50,
        exponent: 1.1,
        maxElapsedTime: 100,
      },
      retryConnectionErrors: false,
    },
  });

  console.log(result);
}

run();

If you'd like to override the default retry strategy for all operations that support retries, you can provide a retryConfig at SDK initialization:

import { LangCache } from "@redis-ai/langcache";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  retryConfig: {
    strategy: "backoff",
    backoff: {
      initialInterval: 1,
      maxInterval: 50,
      exponent: 1.1,
      maxElapsedTime: 100,
    },
    retryConnectionErrors: false,
  },
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  const result = await langCache.deleteQuery({
    attributes: {
      "language": "en",
      "topic": "ai",
    },
  });

  console.log(result);
}

run();

Error Handling

LangCacheError is the base class for all HTTP error responses. It has the following properties:

| Property | Type | Description | | ------------------- | ---------- | --------------------------------------------------------------------------------------- | | error.message | string | Error message | | error.statusCode | number | HTTP response status code eg 404 | | error.headers | Headers | HTTP response headers | | error.body | string | HTTP body. Can be empty string if no body is returned. | | error.rawResponse | Response | Raw HTTP response | | error.data$ | | Optional. Some errors may contain structured data. See Error Classes. |

Example

import { LangCache } from "@redis-ai/langcache";
import * as errors from "@redis-ai/langcache/models/errors";

const langCache = new LangCache({
  serverURL: "https://api.example.com",
  cacheId: "<id>",
  apiKey: "<LANGCACHE_API_KEY>",
});

async function run() {
  try {
    const result = await langCache.deleteQuery({
      attributes: {
        "language": "en",
        "topic": "ai",
      },
    });

    console.log(result);
  } catch (error) {
    // The base class for HTTP error responses
    if (error instanceof errors.LangCacheError) {
      console.log(error.message);
      console.log(error.statusCode);
      console.log(error.body);
      console.log(error.headers);

      // Depending on the method different errors may be thrown
      if (error instanceof errors.BadRequestErrorResponseContent) {
        console.log(error.data$.title); // string
        console.log(error.data$.status); // number
        console.log(error.data$.detail); // string
        console.log(error.data$.instance); // string
        console.log(error.data$.type); // models.BadRequestErrorURI
      }
    }
  }
}

run();

Error Classes

Primary errors:

• LangCacheError: The base class for HTTP error responses.
  • BadRequestErrorResponseContent: BadRequestError 400 response. Status code 400.
  • AuthenticationErrorResponseContent: AuthenticationError 401 response. Status code 401.
  • ForbiddenErrorResponseContent: ForbiddenError 403 response. Status code 403.
  • NotFoundErrorResponseContent: NotFoundError 404 response. Status code 404.
  • PayloadTooLargeErrorResponseContent: PayloadTooLargeError 413 response. Status code 413.
  • ResourceUnavailableErrorResponseContent: ResourceUnavailableError 424 response. Status code 424.
  • TooManyRequestsErrorResponseContent: TooManyRequestsError 429 response. Status code 429.
  • UnexpectedErrorResponseContent: UnexpectedError 500 response. Status code 500.

Network errors:

• ConnectionError: HTTP client was unable to make a request to a server.
• RequestTimeoutError: HTTP request timed out due to an AbortSignal signal.
• RequestAbortedError: HTTP request was aborted by the client.
• InvalidRequestError: Any input used to create a request is invalid.
• UnexpectedClientError: Unrecognised or unexpected error.

Inherit from LangCacheError:

• ResponseValidationError: Type mismatch between the data returned from the server and the structure expected by the SDK. See error.rawValue for the raw value and error.pretty() for a nicely formatted multi-line string.

Custom HTTP Client

The TypeScript SDK makes API calls using an HTTPClient that wraps the native Fetch API. This client is a thin wrapper around fetch and provides the ability to attach hooks around the request lifecycle that can be used to modify the request or handle errors and response.

The HTTPClient constructor takes an optional fetcher argument that can be used to integrate a third-party HTTP client or when writing tests to mock out the HTTP client and feed in fixtures.

The following example shows how to use the "beforeRequest" hook to to add a custom header and a timeout to requests and how to use the "requestError" hook to log errors:

import { LangCache } from "@redis-ai/langcache";
import { HTTPClient } from "@redis-ai/langcache/lib/http";

const httpClient = new HTTPClient({
  // fetcher takes a function that has the same signature as native `fetch`.
  fetcher: (request) => {
    return fetch(request);
  }
});

httpClient.addHook("beforeRequest", (request) => {
  const nextRequest = new Request(request, {
    signal: request.signal || AbortSignal.timeout(5000)
  });

  nextRequest.headers.set("x-custom-header", "custom value");

  return nextRequest;
});

httpClient.addHook("requestError", (error, request) => {
  console.group("Request Error");
  console.log("Reason:", `${error}`);
  console.log("Endpoint:", `${request.method} ${request.url}`);
  console.groupEnd();
});

const sdk = new LangCache({ httpClient: httpClient });

Debugging

You can setup your SDK to emit debug logs for SDK requests and responses.

You can pass a logger that matches console's interface as an SDK option.

[!WARNING] Beware that debug logging will reveal secrets, like API tokens in headers, in log messages printed to a console or files. It's recommended to use this feature only during local development and not in production.

import { LangCache } from "@redis-ai/langcache";

const sdk = new LangCache({ debugLogger: console });

Development

Maturity

This SDK is in beta, and there may be breaking changes between versions without a major version update. Therefore, we recommend pinning usage to a specific package version. This way, you can install the same version each time without breaking changes unless you are intentionally looking for the latest version.