agentic-gate
v1.5.0
Published
Deterministic schema validation gates and circuit breakers for LLM function calling
Maintainers
Readme
🛡️ Agentic Deterministic State Engine
A production-ready, model-agnostic proxy engine that enforces strict runtime execution bounds, schema validation gates, and self-correction loops for LLM Function Calling.
📌 Problem Statement
LLM Agentic Function Calling (Tools) is inherently non-deterministic:
- Hallucinated Arguments: Models frequently output invalid parameters (e.g., non-existent regions, malformed UUIDs, out-of-bound dates).
- Infinite Loops: When a tool call fails, agents often re-issue the exact same broken payload indefinitely.
- Unsafe Execution: Executing unvalidated tool calls directly against downstream infrastructure (AWS, SQL databases, payment gateways) risks severe operational failure.
💡 Solution Architecture
This repository implements a Deterministic State Engine sitting between the user/application and downstream infrastructure tools. It intercepts every tool invocation requested by the LLM, passes it through a local Zod Schema Validation Gate, and manages a stateful retry loop with error feedback injection.
┌─────────────────────────────────┐
│ User Input / System Prompt │
└────────────────┬────────────────┘
│
▼
┌─────────────────────────────────┐
│ LLM Provider (Bedrock/OpenAI) │
└────────────────┬────────────────┘
│
▼ (Tool Use Requested)
┌─────────────────────────────────┐
│ DETERMINISTIC STATE ENGINE │
│ │
│ ┌─────────────────────────┐ │
│ │ Zod Validation Gate │ │
│ └────────────┬────────────┘ │
└────────────────┼────────────────┘
│
┌─────────────────┴─────────────────┐
│ │
[Schema PASSED] [Schema FAILED]
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ Execute Downstream API │ │ Inject Error Feedback │
│ (AWS EC2, DB, etc.) │ │ into Message History │
┌─────────────────────────┐ └────────────┬────────────┘
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ Return Success │ │ Loop (Max Retries Check)│
└─────────────────────────┘ └─────────────────────────┘flowchart TD
A[LLM Provider<br/>Bedrock / OpenAI / Anthropic / Gemini] -->|Tool Use Requested| B[Interceptor Gate]
B --> C{Zod Schema<br/>Validation}
C -->|PASSED| D[Execute Downstream Tool<br/>AWS API / DB / Payment Gateway]
C -->|FAILED| E[Inject Error Feedback<br/>into Message History]
E -->|Retry within maxRetries| A
D --> F[Return Result to Caller]Key Features
- 🛡️ Zero-Trust Validation Gate: Uses Zod schemas to intercept and validate parameters locally before hitting downstream APIs.
- 🔄 Self-Correction Feedback Loop: Feeds exact schema validation error traces back into the conversation history, allowing the LLM to self-correct in subsequent attempts.
- ⚡ Circuit Breaker: After
maxConsecutiveFailures(default3) failures in a row for the same tool, the gate trips and rejects further calls before touching the schema orexecute()— stopping runaway LLM retry loops without your own bookkeeping. Reset withgate.resetCircuit(toolName). - 📡 Telemetry Hooks:
onGateSuccess/onGateFailurecallbacks fire on every call, so you can export metrics to OpenTelemetry, Datadog, CloudWatch, or plain logs. - 🔍 Async External-State Validation: Add an optional
validate()hook per tool to check real external state (e.g. does this EC2 instance ID actually exist) beforeexecute()runs — a rejectedvalidate()counts as a gate failure just like a schema mismatch. - 🔌 Provider-Agnostic Design: Decoupled architecture support for AWS Bedrock, OpenAI, Anthropic, and Google Gemini.
- 🔑 Zero Secrets Leakage: Native integration with local AWS credentials (
aws configure) or environment variables.
🏗️ Engine Sequence Diagram
User Engine Zod Gate Bedrock / LLM AWS API
│ │ │ │ │
│─ Send Prompt ─────>│ │ │ │
│ │── Converse API ───────────────────────>│ │
│ │<── Tool Request ("restart_ec2") ───────│ │
│ │ │ │ │
│ │── Validate Input >│ │ │
│ │<─ FAILED (Region)─│ │ │
│ │ │ │ │
│ │── Append Tool Error & Retry Loop ─────>│ │
│ │<── Self-Corrected Text Response ───────│ │
│ │ │ │ │
│<─ Return Result ───│ │ │ │🛠️ Multi-AI Provider Extensibility
While the core implementation uses AWS Bedrock Converse API, the state engine and validation gates are completely provider-agnostic.
Below is how the exact same validation engine wraps other AI providers:
1. AWS Bedrock (Default Implementation)
import { BedrockRuntimeClient, ConverseCommand } from "@aws-sdk/client-bedrock-runtime";
const response = await bedrockClient.send(new ConverseCommand({
modelId: "us.anthropic.claude-haiku-4-5-20251001-v1:0",
messages: messages,
toolConfig: toolConfig
}));2. OpenAI (openai Package)
import OpenAI from "openai";
const openai = new OpenAI();
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: messages,
tools: toolsSpec
});
const toolCall = response.choices[0].message.tool_calls?.[0];
const validation = EC2RestartSchema.safeParse(JSON.parse(toolCall.function.arguments));3. Anthropic Native (@anthropic-ai/sdk)
import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic();
const response = await anthropic.messages.create({
model: "claude-3-5-sonnet-20241022",
messages: messages,
tools: toolsSpec
});
const toolCall = response.content.find(c => c.type === "tool_use");
const validation = EC2RestartSchema.safeParse(toolCall.input);4. Google Gemini (@google/genai)
import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
const response = await ai.models.generateContent({
model: "gemini-3.6-flash",
contents,
config: { tools: [{ functionDeclarations: [restartEc2Declaration] }] }
});
const call = response.functionCalls[0];
const validation = EC2RestartSchema.safeParse(call.args);🚀 Quick Start
agentic-gate ships for both JavaScript/TypeScript and Python — same engine,
same guarantees, native API for each.
1. Install
npm install agentic-gate zod # JS/TS
pip install agentic-gate # Python2. Register a tool and intercept a call
This is the core API — provider-agnostic, no AWS/network calls required.
JavaScript / TypeScript:
import { AgenticGate } from "agentic-gate";
import { z } from "zod";
const gate = new AgenticGate();
gate.registerTool({
name: "restart_ec2_instance",
schema: z.object({
instanceId: z.string().regex(/^i-[a-f0-9]{8,17}$/, "Invalid AWS EC2 Instance ID format"),
region: z.enum(["us-east-1", "us-west-2", "ap-south-1"]),
}),
execute: async ({ instanceId, region }) => {
// Your real downstream call (AWS SDK, DB, HTTP, etc.)
return { status: "success", instanceId, region };
},
});
// Feed it raw, untrusted arguments straight from the LLM's tool-call payload
const result = await gate.interceptAndExecute("restart_ec2_instance", {
instanceId: "i-0123456789abcdef0",
region: "eu-central-1", // not in the enum -> gate rejects before execute() runs
});
if (!result.success) {
console.log(result.error);
// "[Validation Gate Failed]: region: Invalid enum value..."
// Feed this string back into the LLM's message history so it can self-correct.
}Python:
from typing import Literal
from pydantic import BaseModel
from agentic_gate import AgenticGate
class RestartEc2Args(BaseModel):
instance_id: str
region: Literal["us-east-1", "us-west-2", "ap-south-1"]
gate = AgenticGate()
async def restart_ec2(args: RestartEc2Args):
# Your real downstream call (boto3, an HTTP client, etc.)
return {"status": "success", "instance_id": args.instance_id, "region": args.region}
gate.register_tool("restart_ec2_instance", RestartEc2Args, execute=restart_ec2)
# Feed it raw, untrusted arguments straight from the LLM's tool-call payload
result = await gate.intercept_and_execute("restart_ec2_instance", {
"instance_id": "i-0123456789abcdef0",
"region": "eu-central-1", # not in the enum -> gate rejects before execute() runs
})
if not result.success:
print(result.error)
# "[Validation Gate Failed]: region: Input should be 'us-east-1', 'us-west-2' or 'ap-south-1'"
# Feed this string back into the LLM's message history so it can self-correct.Full API reference, circuit breaker/telemetry examples, and provider adapters
for Python are in python/README.md.
3. Circuit breaker + telemetry
Configure both when constructing the gate:
const gate = new AgenticGate({
maxConsecutiveFailures: 3, // 0 disables the breaker
onGateFailure: (e) => metrics.increment(`gate.failure.${e.reason}`, { tool: e.toolName }),
onGateSuccess: (e) => metrics.increment("gate.success", { tool: e.toolName }),
});
// After 3 consecutive failures for "restart_ec2_instance", the gate short-circuits:
// { success: false, error: "[Circuit Breaker OPEN]: Tool 'restart_ec2_instance' has failed 3 consecutive times..." }
gate.resetCircuit("restart_ec2_instance"); // once the underlying issue is fixed4. Async external-state validation
Zod only checks the shape of the arguments — it can't tell you whether
i-0123456789abcdef0 is an EC2 instance that actually exists. For that, add
a validate() hook: it runs after the schema passes and before execute(),
and throwing rejects the call exactly like a schema failure (same circuit
breaker, same telemetry, reason "async-validation"):
import { EC2Client, DescribeInstancesCommand } from "@aws-sdk/client-ec2";
const ec2 = new EC2Client({});
gate.registerTool({
name: "restart_ec2_instance",
schema: z.object({
instanceId: z.string().regex(/^i-[a-f0-9]{8,17}$/),
region: z.enum(["us-east-1", "us-west-2", "ap-south-1"]),
}),
validate: async ({ instanceId, region }) => {
const { Reservations } = await ec2.send(
new DescribeInstancesCommand({ InstanceIds: [instanceId] }, { region })
);
if (!Reservations?.length) {
throw new Error(`Instance '${instanceId}' does not exist in ${region}`);
}
},
execute: async ({ instanceId, region }) => {
// Safe to restart — we already confirmed the instance is real.
},
});If the lookup can hang, set new AgenticGate({ validateTimeoutMs: 2000 }) (Python: validate_timeout_s=2.0). A timeout is rejected as reason "validation-unavailable" and execute() never runs. The error tells the model the check could not finish, not that the resource is invalid.
Arguments that arrive as a JSON string (OpenAI-style endpoints) can be passed to interceptAndExecute as-is. Truncated or malformed JSON is rejected with reason "parse", separate from schema failures.
5. Less boilerplate with an adapter
Every provider example above ends up writing the same ~30 lines: pull the
tool call(s) out of the response, parse the arguments, run the gate, and
rebuild a provider-shaped result message. agentic-gate/adapters/* does that
for you — one adapter per provider, since each has its own response and
message shape:
import { AgenticGate } from "agentic-gate";
import { handleResponse } from "agentic-gate/adapters/openai"; // or /anthropic, /bedrock, /gemini
for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
const response = await openai.chat.completions.create({ model, messages, tools });
const outcome = await handleResponse(gate, response);
messages.push(...outcome.messages); // ready-to-append message(s), gate already applied
if (outcome.done) {
console.log(outcome.text);
break;
}
}Each adapter only depends on the shape of its provider's response object —
not the provider's SDK — so none of them add a dependency. All four handle
multiple simultaneous tool calls in one response correctly (Anthropic/Bedrock
bundle every result into a single following turn, as required). Because the
openai adapter only cares about the response shape, it also works against
any OpenAI-compatible local server — verified live against a Docker-hosted
Ollama running a 397MB model, see
examples/ollama-adapter.mjs.
6. See it wired into a real provider loop
The snippet above validates a single call. For the full retry loop — sending the
validation error back to the model and looping until it self-corrects or hits
maxRetries — see the runnable examples in examples/. Each
provider has both a raw version (manual parsing, to see the mechanics) and an
-adapter version (using agentic-gate/adapters/*, to see the boilerplate
disappear):
| Example | Provider | Domain |
|---|---|---|
| examples/bedrock-converse.mjs / -adapter | AWS Bedrock Converse API | EC2 instance restart |
| examples/openai.mjs / -adapter | OpenAI Function Calling | EC2 instance restart |
| examples/anthropic.mjs / -adapter | Anthropic Messages API | EC2 instance restart |
| examples/gemini.mjs / -adapter | Google Gemini API | Satellite launch scheduling (non-infra) |
| examples/pizza-order.mjs | AWS Bedrock Converse API | Pizza ordering (non-infra, to show the gate isn't AWS-specific) |
| examples/langchain.mjs | LangChain (@langchain/google-genai) | Pet adoption — gate wrapped inside a LangChain tool() handler |
| examples/ollama-adapter.mjs | Local model via Ollama (OpenAI-compatible endpoint) | EC2 instance restart |
Each example requires only its provider's SDK and credentials — see
examples/README.md for setup.
🐍 Python details
See the Quick Start above for the Python install +
core-API snippet. agentic-gate for Python also has the full feature set —
circuit breaker, telemetry hooks, async external-state validation, and all
four provider adapters (agentic_gate.adapters.openai/anthropic/bedrock/gemini)
mirroring the JS ones above — see examples/ollama-adapter.mjs
for the local-model story, verified live in both languages. Source
lives in python/ in this same repo; see
python/README.md for the full API reference and
python/examples/ for runnable scripts.
🧪 Real Execution Logs
Here is unedited output from examples/pizza-order.mjs — a
domain with nothing to do with AWS, chosen to show the gate validates whatever
Zod schema you give it. The prompt deliberately breaks three constraints at
once (size, quantity, and toppings), and telemetry hooks are enabled so
you can see them fire alongside each attempt:
🚀 Starting Engine Execution for Prompt: "I'm throwing a huge party — order me 50 party-size pizzas loaded with every topping you can think of: pepperoni, mushroom, onion, olives, extra-cheese, pineapple, and jalapenos."
📡 Using Bedrock Model: us.anthropic.claude-haiku-4-5-20251001-v1:0
--- Loop Attempt 1/3 ---
[Bedrock Tool Requested]: "order_pizza" with input: {
size: 'party',
quantity: 50,
toppings: [
'pepperoni',
'mushroom',
'onion',
'olives',
'extra-cheese',
'pineapple',
'jalapenos'
]
}
📉 telemetry: validation failure #1 for "order_pizza"
❌ [Validation Gate Failed]: size: size must be one of: small, medium, large; quantity: quantity cannot exceed 10 pizzas per order; toppings: you can pick at most 5 toppings; toppings.5: Invalid enum value. Expected 'pepperoni' | 'mushroom' | 'onion' | 'olives' | 'extra-cheese', received 'pineapple'; toppings.6: Invalid enum value. Expected 'pepperoni' | 'mushroom' | 'onion' | 'olives' | 'extra-cheese', received 'jalapenos'
--- Loop Attempt 2/3 ---
🤖 [Model Response]:
I see there are some limitations with the pizza ordering system:
1. **Size**: Only small, medium, or large sizes are available (not party-size)
2. **Quantity**: Maximum of 10 pizzas per order
3. **Toppings**: Maximum of 5 toppings, and only these are available:
- pepperoni
- mushroom
- onion
- olives
- extra-cheese
Unfortunately, pineapple and jalapenos aren't available options.
To help you with your party, here's what I can do:
**Option 1**: I can place **5 separate orders of 10 large pizzas each** (50 pizzas total) with your choice of up to 5 of the available toppings.
**Option 2**: Order a smaller quantity with the available toppings.
Which would you prefer? And which 5 toppings would you like me to use from: pepperoni, mushroom, onion, olives, or extra-cheese?Note the gate rejected the call — hallucinated size, over-limit quantity, and two invalid toppings, five violations across three fields — in one pass, before any downstream code ran, and the model didn't just apologize: it proposed a working alternative on its own.
Scope
agentic-gate checks the arguments of a tool call before it runs, feeds validation errors back to the model, and stops repeated failures with a circuit breaker. It is one layer in an agent's safety design, and it works alongside the things you already have: authorization and permissions on the tools themselves, rate limits, and review of high-impact actions. A call that passes the schema is valid in shape; whether it is the right action for the user is still your application's decision.
What it does not do:
- It is a wrapper, not a sandbox. The check only applies to calls routed through the gate. If code can call the tool function directly, it bypasses the gate, so register the gate at the point where the tool actually runs.
- No idempotency. The gate rejects calls before they run. If a call passes, executes, and then times out on the response, a model retry is a second execution unless the tool uses idempotency keys.
- No caller identity. There is no built-in per-user authorization. Do that inside
validate()or in the tool itself. - The circuit breaker is per tool. It counts consecutive failures, so many different bad calls to one tool also trip it. Use
resetCircuit()after fixing the cause.
📄 License
This project is licensed under the MIT License.
