@cloudpeers-jkl/model-router

cloudpeers inference router — embedded library (Amendment A1.1: no central data plane). Each service embeds the router behind its existing seam; the sovereignty privacy gate runs in-process, where the data lives. Control plane (tier config, approved providers, gate stage, metering aggregation) distributes from the platform as deploy-time config.

Canonical spec: cloudpeers-github/docs/architecture/INFERENCE_ROUTER_SPEC.md (read Amendment A1 first). Pointer + contract map: mcp/docs/INFERENCE_ROUTER_SPEC.md.

Two APIs, one enforcement path

import {
  routeModel, routeMessages, defaultBackendAdapters,
} from '@cloudpeers-jkl/model-router';

const deps = {
  adapters: defaultBackendAdapters(),   // selfhosted, gemini, anthropic — no OpenAI (directive)
  meter,                                // the embedding service's token meter
  quota,                                // and quota check (token-tracker in mcp)
  hooks: {},                            // §7 mandate/policy/attribution — fail closed when named but unwired
};

// v1 convenience API — prompt in, text out
const r = await routeModel({
  taskClass: 'coaching',
  prompt: '…',
  sovereigntyClass: 'local_only',       // REQUIRED — the caller declares the data class
  operation: 'lab_coaching',
  userId, serviceId: 'experience-labs',
}, deps);

// Spec API — Anthropic Messages shape + cloudpeers envelope (§4)
const { response, usage } = await routeMessages({
  messages: [{ role: 'user', content: '…' }],
  max_tokens: 1000,
  cloudpeers: { v: 1, sovereignty_class: 'aggregate_only', task_class: 'reasoning' },
}, { userId, serviceId, operation: 'my_op' }, deps);

In mcp, don't construct deps yourself — import the prewired composition from server/lib/model-router.ts.

Behavior contracts

• Privacy gate stages (interim cloud-PHI policy 2026-07-09): MONITOR (default, logs [privacy-gate-monitor] would_block lines) → BAA egress (MODEL_ROUTER_BAA_PROVIDERS) → enforce (MODEL_PRIVACY_GATE_ENFORCE=true, fail closed). The monitor log lines are a stable interface — the Stage-3 flip decision is made from them.
• Fail closed: mandate_id/policy_id named with no wired hook → blocked (412). BYOK → 501 (deferred, A1.2). tool_use_strict with no capable backend → 422. Errors carry statusCode/code for HTTP mapping.
• Static routing (confidence v1): task-class tier chains in policy.ts; schema-validation failure escalates a tier; exhausted chain returns best-effort marked degraded.
• Uniform metering: every call reports through the injected meter with a router metadata block — provider, tier, taskClass, gate.

Env (deploy-time control plane)

MODEL_ROUTER_TIER0_MODEL / MODEL_ROUTER_TIER0_URL (Tier 0 self-hosted), MODEL_ROUTER_GEMINI_MODEL, MODEL_ROUTER_ANTHROPIC_MODEL, MODEL_ROUTER_BAA_PROVIDERS, MODEL_PRIVACY_GATE_ENFORCE, plus the provider keys (ANTHROPIC_API_KEY, GEMINI_SERVER_API_KEY/GEMINI_API_KEY/GOOGLE_AI_API_KEY).

Publishing

Published on npm since 2026-07-15 (@cloudpeers-jkl/model-router, current 0.2.1); carepeers consumes the npm package. mcp itself consumes this source in-repo via relative imports (see server/lib/model-router.ts) — source and package must not drift, so publish after any change here: npm publish --access public from this directory (prepublishOnly builds dist/; requires JKL's npm browser 2FA).