Caduceus

One shape for every model. Caduceus is the open, local-first, provider-agnostic transport layer for Massive Intelligence (IM). It normalizes OpenAI, Anthropic, and Gemini, routes each request across a pool of heterogeneous models by cost, quality, or latency, fails over when a provider is down, holds every call to a hard USD ceiling, and emits an auditable trace that says which model answered. Zero runtime dependencies, a node-free core, TypeScript-first.

It is the peripheral nervous system of the @takk/coryphaeus orchestrator, and it works perfectly well on its own.

Where the market moved from prompt to agent, the next category is the non-human entity: MAIC, HIMs, and NHEs. Caduceus is the transport such entities route over, Massive Intelligence (IM) infrastructure for automation beyond OpenClaw, Hermes Agent, and Claude Code, run on your own infrastructure with your own keys, fully auditable.

Caduceus is the package (display name). caduceus is the npm scope token, the CLI binary, and the import path. The core never speaks HTTP; the network call lives in an injectable dispatch, so the routing logic is provider-agnostic, runs on the edge, and is testable offline.

Why

Every provider has a different request body, a different streaming format, and a different tool-call shape. Routing dynamically by task, cost, latency, and quality, with automatic failover when one falls over, is usually done ad hoc and is fragile. The gateways that promise to solve it (a hosted single endpoint) hide which model answered, can't be configured, price at the ceiling of the pool they route, and are blocked in some regions. Caduceus is the opposite: it runs on your infrastructure with your keys, every routing decision is visible, the strategies are yours to choose, and dollar ceilings are hard.

• Provider-agnostic. OpenAI, Anthropic, and Gemini request, response, streaming, and tool calls normalized to one typed vocabulary. An OpenAI-compatible shape covers Ollama, vLLM, Together, Groq, and any /v1/chat/completions gateway.
• Resilient. Per-model circuit breakers (closed, open, half-open), rate-limit recovery that honors Retry-After and retries before failover, and automatic failover, so one provider going down or rate-limiting you does not take your request with it.
• Auditable. A route trace names every model tried, its breaker state, its cost and latency, and which one answered. It renders to DOT or Mermaid and seals with SHA-256.
• Cost-controlled. USD forecasts before every call and a hard per-request ceiling that raises a typed error instead of a surprise bill.
• Portable. A node-free core runs on Cloudflare Workers, Vercel Edge, Bun, Deno, and the browser. The one bundled real transport (fetch) lives in the Node entry.

Install

npm install @takk/caduceus
# or: pnpm add @takk/caduceus

Node 20+. Ships ESM and CJS with types for every subpath.

Quickstart

import { Caduceus, defineProvider, defineModel } from "@takk/caduceus";
import { createFetchDispatch } from "@takk/caduceus/node";

const providers = [
  defineProvider({ id: "openai", shape: "openai", baseUrl: "https://api.openai.com/v1" }),
  defineProvider({ id: "anthropic", shape: "anthropic", baseUrl: "https://api.anthropic.com/v1" }),
  defineProvider({ id: "google", shape: "gemini", baseUrl: "https://generativelanguage.googleapis.com/v1beta" }),
];

const models = [
  defineModel({ id: "gpt", provider: "openai", model: "gpt-5", cost: { inputPerMTok: 5, outputPerMTok: 15 }, quality: 0.9 }),
  defineModel({ id: "sonnet", provider: "anthropic", model: "claude-sonnet-4-6", cost: { inputPerMTok: 3, outputPerMTok: 15 }, quality: 0.85 }),
  defineModel({ id: "flash", provider: "google", model: "gemini-flash", cost: { inputPerMTok: 0.3, outputPerMTok: 1.2 }, quality: 0.6 }),
];

// The dispatch carries your credentials; the core never sees a key.
const dispatch = createFetchDispatch({
  auth: (call) => {
    if (call.shape === "anthropic") return { "x-api-key": process.env.ANTHROPIC_API_KEY!, "anthropic-version": "2023-06-01" };
    if (call.shape === "gemini") return { "x-goog-api-key": process.env.GEMINI_API_KEY! };
    return { authorization: `Bearer ${process.env.OPENAI_API_KEY}` };
  },
});

const caduceus = new Caduceus({ dispatch, providers, models, strategy: "cost-then-quality", budgetUsd: 0.05 });

const result = await caduceus.complete({
  messages: [{ role: "user", content: "Name one routing strategy." }],
});

console.log(result.winner);          // { model: "flash", provider: "google" }
console.log(result.response.text);   // the answer, normalized
console.log(result.totalCostUsd);    // priced in USD
console.log(result.attempts);        // every model tried, with breaker state and cost

No keys to hand? Swap createFetchDispatch(...) for createEchoDispatch() from @takk/caduceus/codec and the whole thing runs offline and deterministically, the same client the tests and the CLI --mock use.

Streaming

const route = await caduceus.stream({ messages: [{ role: "user", content: "Stream a haiku." }] });
for await (const event of route.stream) {
  if (event.type === "text-delta") process.stdout.write(event.text);
}

OpenAI chunk deltas, Anthropic content-block events, and Gemini partial candidates all arrive as the same typed StreamEvent union (start, text-delta, tool-call-delta, finish, error).

The seven strategies

A strategy orders the candidate models; the router then tries them in order, skipping open breakers and failing over on error.

| Strategy | Orders by | | --- | --- | | cost-then-quality | cheapest first, ties broken by higher quality (the default) | | cost-first | cheapest first | | quality-first | highest quality first, ties broken by cost | | latency-first | lowest latency hint first | | weighted | highest selection weight first | | round-robin | rotates the starting model each request | | sequential-cascade | cheapest-and-weakest first, escalating to stronger models |

Pick a default at construction and override per request with complete(request, { strategy }). Inspect the ordering and forecast without spending anything:

const plan = caduceus.plan(request, { strategy: "quality-first" });
// plan.candidates -> ordered, each with an estimatedCostUsd; plan.estimatedCostUsd is the first attempt

Circuit breakers and budgets

const caduceus = new Caduceus({
  dispatch, providers, models,
  breaker: { failureThreshold: 3, cooldownMs: 30_000 }, // trip after 3 failures, probe after 30s
  budgetUsd: 0.05,                                       // a hard per-request ceiling
});

A model that fails three times in a row has its breaker tripped open; the router skips it (recording skipped-open) until the cooldown elapses, then lets a single probe through. A request whose cheapest affordable candidate is still over the ceiling raises a typed BUDGET_EXCEEDED rather than dispatching. caduceus.breakerSnapshot() reports the live state of every model.

Rate-limit recovery

In production a provider rarely goes down; it returns a 429. Caduceus does not fail over blindly on a rate-limit: it waits the provider's Retry-After (or an exponential backoff, both capped) and retries the same model before failing over, recording the retry count and wait in the trace.

const caduceus = new Caduceus({
  dispatch, providers, models,
  retry: { maxRetries: 2, baseDelayMs: 500, maxDelayMs: 30_000 },
});

Pair it with key rotation so a 429 on one key retries with the next, the @takk/keymesh seam without a hard dependency:

import { withKeyRotation } from "@takk/caduceus/adapter";
const dispatch = withKeyRotation((key) => createFetchDispatch({ auth: () => ({ authorization: `Bearer ${key}` }) }), keyPool);

The auditable trace

import { renderTraceDot, provenanceOf, sealTrace } from "@takk/caduceus/trace";

provenanceOf(result);     // { model, provider } that answered
renderTraceDot(result);   // a Graphviz transport graph of every attempt
await sealTrace(result);  // a SHA-256 integrity seal over the canonical trace

This is the answer to the gateway black box: you can see, render, and seal which model responded and why, on every request.

Drop it into what you already have

import { dispatchFromVercel } from "@takk/caduceus/adapter";
import { generateText } from "ai"; // the Vercel AI SDK

const dispatch = dispatchFromVercel(({ model, messages }) => generateText({ model: yourModel(model), messages }));
const caduceus = new Caduceus({ dispatch, providers, models });

Expose the router as a framework-agnostic tool for an MCP server or any tool-calling API:

import { createTransportTool } from "@takk/caduceus/adapter";
const tool = createTransportTool(caduceus); // name: "caduceus_route", with a JSON Schema and a JSON-safe handler

Feed the @takk/coryphaeus orchestrator's agent pool with no hard dependency:

import { agentClientFromCaduceus } from "@takk/caduceus/adapter";
const client = agentClientFromCaduceus(caduceus); // satisfies the @takk/coryphaeus AgentClient contract

Does failover actually help?

The benchmark (benchmarks/transport-benchmark.mjs) runs the built router over a controlled failure model on a fixed seed, 4000 requests per regime. It is a mechanism demonstration, not an LLM quality claim.

| Regime | Without Caduceus | With Caduceus | | --- | --- | --- | | Independent 40% provider failure | single provider 59.4% success | three-way failover 93.5% success (bound 1 − p³ = 93.6%) | | One provider fully down | 8000 dispatch calls | 4005 with the breaker (49.9% of wasted calls cut) |

Failover converts independent provider unreliability into near-certain success; the circuit breaker stops paying to call a provider that is down. The numbers above come from running the benchmark end to end, never invented.

CLI

caduceus providers --providers providers.json --models models.json
caduceus plan request.json --providers providers.json --models models.json --strategy quality-first
caduceus route request.json --providers providers.json --models models.json --mock --out trace.json
caduceus trace trace.json --format mermaid

Exit codes: 0 ok, 2 usage or input error, 21 no route, 30 budget exceeded. The CLI carries no provider credentials; route requires --mock, and real runs use the library API.

Subpath exports

| Import | What it gives you | | --- | --- | | @takk/caduceus | the barrel: Caduceus, providers, codec, strategies, breakers, budget, trace, adapters, errors | | @takk/caduceus/transport | the Caduceus router | | @takk/caduceus/provider | defineProvider, defineModel, ProviderRegistry | | @takk/caduceus/codec | the compatibility matrix and createEchoDispatch | | @takk/caduceus/strategy | the seven strategies | | @takk/caduceus/breaker | CircuitBreaker, BreakerRegistry | | @takk/caduceus/budget | token estimation, USD pricing, BudgetTracker | | @takk/caduceus/trace | provenance, DOT and Mermaid rendering, the SHA-256 seal | | @takk/caduceus/adapter | Vercel and OpenAI bridges, the @takk/coryphaeus AgentClient, the routing tool | | @takk/caduceus/node | the real fetch dispatch and the JSON loaders (the only entry touching a Node built-in) | | @takk/caduceus/edge | the node-free core for Workers, Edge, Bun, Deno, and the browser |

See SPEC.md for the full surface and CHANGELOG.md for what shipped.

Not in 1.0

A learned, quality-measured router (the 1.0 strategies are transparent and deterministic; a measured strategy with evaluators and bandits is planned behind the same Strategy seam); bundled provider SDKs; a hosted gateway; a runnable MCP server binary; multimodal content parts; and signed, timestamped trace seals. Each is planned for a later release.

License

Apache-2.0, © David C Cavalcante, Takk Innovate Studio. See NOTICE.