@cryptoteep/llm-retry

Tiny, zero-dependency retry/backoff wrapper for LLM API calls. Works with the Anthropic SDK, OpenAI SDK, Vercel AI SDK, or any fetch-based client.

Handles the thing every LLM app eventually needs: rate limits (429) and transient server errors (5xx/network) retried with exponential backoff and jitter, respecting Retry-After when the API sends one.

Install

npm install @cryptoteep/llm-retry

Usage

import { withRetry } from "@cryptoteep/llm-retry";
import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic();

const message = await withRetry(() =>
  anthropic.messages.create({
    model: "claude-sonnet-5",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello" }],
  }),
);

Wrap a function once, retry every call

import { retryable } from "@cryptoteep/llm-retry";

const createMessage = retryable(anthropic.messages.create.bind(anthropic.messages));

const message = await createMessage({ model: "claude-sonnet-5", max_tokens: 1024, messages: [...] });

Options

await withRetry(fn, {
  maxAttempts: 5,      // including the first attempt
  baseDelayMs: 500,
  maxDelayMs: 30_000,
  factor: 2,            // exponential growth factor
  jitter: true,          // randomize delay to avoid thundering herd
  shouldRetry: (error, attempt) => true, // custom retry predicate
  onRetry: (error, attempt, delayMs) => console.warn(`retry #${attempt} in ${delayMs}ms`),
  signal: abortController.signal,
});

By default, retries trigger on 429, any 5xx, and errors with no HTTP status (network failures, timeouts). Everything else — 400, 401, 404, etc. — fails immediately since retrying won't help.

Why not just use the SDK's built-in retries?

Most SDKs retry internally, but you often need this at a higher level: across multiple calls in an agent loop, with custom backoff for your own rate limits, or with a client that has no retry logic at all (raw fetch, a self-hosted model gateway, etc.). withRetry is provider-agnostic — it only looks at error.status / error.statusCode / error.response.status, which covers the Anthropic SDK, OpenAI SDK, Vercel AI SDK, and axios-style errors out of the box.

License

MIT