agent-gateway 🌐

Self-Hosted, Privacy-First LLM Provider Router

A Project by SILICON VALLEY GLOBAL PH INC

agent-gateway is a self-hosted, embeddable router that gives you one unified API across OpenAI, Anthropic, Google Gemini, and more — with automatic fallback and cost/latency-based routing — without a third-party server ever sitting between you and the provider. Created and maintained by SILICON VALLEY GLOBAL PH INC.

⚠️ The Problem

Hosted routing services solve the "one API for many providers" problem well, but every request has to pass through their servers first. For teams with compliance requirements, sensitive codebases, or a simple policy of not routing traffic through a third party, that's a real blocker — not a minor inconvenience.

agent-gateway solves the same problem as a library that runs entirely on your own machine. Requests go straight from your code to the actual provider. Nothing is logged, proxied, or billed by a middleman — because there isn't one.

🧩 Key Architecture Pillars

1. Provider Adapters (src/adapters/)

   • Normalize OpenAI, Anthropic, and Google Gemini's differing request/response formats into one common shape, with more providers addable without touching core logic.

2. Routing Engine (src/routing/)

   • Manual priority chains, cost-based routing, and latency-based routing, with automatic failover to the next configured provider on error or rate-limit.

3. Unified Streaming (src/streaming/)

   • One consistent async-iterator streaming interface regardless of which provider is actually serving the request underneath.

4. Local-Only Guarantee (src/core/no-telemetry.ts)

   • Zero analytics, zero logging to any third party, zero gateway server — verified by tests asserting no network calls happen except directly to the selected provider.

5. Suite Integration (src/integrations/)

   • Pulls provider keys from agent-config-sync, scans outgoing prompts through agent-leak-guard before they leave the machine, and exposes routing-decision hooks for agent-permit.

📊 Supported Providers

| Provider | Status | |---|---| | OpenAI | Supported | | Anthropic | Supported | | Google Gemini | Supported | | Others (Cohere, Mistral, local models) | Planned — contributions welcome |

💻 Quickstart

npm install @siliconvalleyglobal/agent-gateway

Basic usage:

import { AgentGateway } from "@siliconvalleyglobal/agent-gateway";

const gateway = new AgentGateway({
  strategy: "fallback",
  providers: ["anthropic", "openai", "gemini"],
});

const response = await gateway.send({
  prompt: "Summarize this changelog...",
});

Check provider status:

npx agent-gateway status

Test the routing chain without a real workload:

npx agent-gateway route-test

⚙️ Configuration

~/.agent-gateway/config.json:

{
  "strategy": "cost",
  "providers": ["anthropic", "openai", "gemini"],
  "fallbackChain": ["anthropic", "openai", "gemini"],
  "useAgentConfigSync": true
}

🔌 Suite Integration Example

import { AgentGateway } from "@siliconvalleyglobal/agent-gateway";
import { scan } from "@siliconvalleyglobal/agent-leak-guard";

const gateway = new AgentGateway({ strategy: "fallback" });

const findings = scan(prompt);
if (findings.length === 0) {
  await gateway.send({ prompt });
}

🗺️ Roadmap

• [ ] Local/self-hosted model support (Ollama, vLLM)
• [ ] Built-in cost tracking export compatible with agent-rate-limit-monitor
• [ ] Response caching layer for duplicate requests

🤝 Contributing

See CONTRIBUTING.md for how to add a new provider adapter.

License

MIT