llm-guard

A production-ready middleware layer for LLM calls, adding validation, retries, fallback models, and safety.

Features

• 🛡️ Schema Validation: Built-in support for Zod schemas to ensure LLM outputs match your expectations.
• 🔄 Retry Mechanism: Configurable retries with exponential backoff.
• 🪜 Fallback Models: Automatically switch to secondary models if the primary one fails.
• 🧹 Output Formatting: Automatic JSON extraction and parsing from LLM responses.
• 📝 Logging System: Detailed console logs for monitoring attempts, failures, and model usage.
• ❌ Custom Errors: Specific error classes for validation and LLM failures.

Installation

npm install llm-guard zod

Usage

Basic Example

import { llmGuard } from "llm-guard";
import { z } from "zod";

const schema = z.object({
  name: z.string(),
  age: z.number()
});

const guard = llmGuard({
  model: "gpt-4o",
  schema,
  retries: 2,
  fallbackModels: ["claude-3", "mistral"]
});

const result = await guard.run("Generate user JSON for John Doe, age 30");
console.log(result);
// Output: { name: "John Doe", age: 30 }

API Reference

llmGuard(options)

Creates a new instance of the LLM Guard.

Options:

• model (string): The primary model to use.
• fallbackModels (string[]): Optional list of models to try if the primary fails.
• retries (number): Number of times to retry a failed validation or call.
• schema (ZodSchema): Optional Zod schema for output validation.
• temperature (number): Optional temperature setting for the LLM.

guard.run(prompt)

Executes the LLM call with the configured middleware.

Example Output

[LLM Guard] Using model: fail-model
[LLM Guard] ERROR: Model fail-model failed.
[LLM Guard] Using model: claude-3
Result: { name: 'John Doe', age: 30, status: 'success', modelUsed: 'claude-3' }

Roadmap

• [ ] Support for streaming responses.
• [ ] Integration with popular LLM providers (OpenAI, Anthropic, Google).
• [ ] Token usage tracking.
• [ ] Custom validation functions.

Contributing

Contributions are welcome! Please open an issue or submit a pull request.

License

MIT